Spectroscopic imaging method of 3D electronic endoscope and 3D electronic endoscope

Abstract

The invention relates to the endoscope technology, and discloses a light splitting imaging method of a 3D electronic endoscope and the 3D electronic endoscope, and the method comprises the steps: when an endoscope lens at the top of an insertion part of the 3D electronic endoscope receives incident light, the incident light is converged to a beam splitter prism based on a first objective lens in each light group, so that fluorescent light in the incident light is transmitted to a second objective lens, and the second objective lens is used for receiving the incident light; the white light in the incident light is reflected to the white light sensor; the fluorescence is converged to the fluorescence sensor through the second objective lens; and the photoelectric conversion module is used for respectively carrying out photoelectric conversion on the received light rays based on the white light sensor and the fluorescent light sensor corresponding to each light group, and transmitting a video stream to a graphic processor at the rear end through high-speed SerDes, and the graphic processor generates a corresponding 3D image after various ISP processing and outputs the 3D image to a 3D display device for display. The invention aims to realize simultaneous fluorescence imaging and color white light imaging of the 3D electronic endoscope in space, and improve the sensitivity and high refresh frame rate in a fluorescence mode.

Description

Technical Field

[0001] The present application relates to the technical field of endoscopes, and in particular to a spectral imaging method of a 3D electronic endoscope and a 3D electronic endoscope. Background Art

[0002] 3D electronic endoscopes generally form images through two sets of lenses in the insertion part (usually a long and thin tube), and respectively collect left and right images through corresponding image sensors. Subsequent image processing processes the left and right images through ISP (Image Signal Processor), synthesizes them, and then outputs them using a 3D display device.

[0003] Due to space limitations, in any image acquisition optical group of a 3D electronic endoscope, fluorescence imaging and white light imaging generally share a color image sensor (such as Figure 1 Therefore, when fluorescence imaging and white light imaging are required, white light and fluorescence can only be imaged by staggered exposure in time. However, most current image sensors are in rolling shutter working mode. In staggered exposure imaging mode, both white light and fluorescence images are contaminated, resulting in low fluorescence sensitivity in color fluorescence mode. The actual fluorescence acquisition frame rate is only half of the maximum frame rate.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present application is to provide a spectral imaging method of a 3D electronic endoscope and a 3D electronic endoscope, so as to enable the 3D electronic endoscope to perform fluorescence imaging and color white light imaging simultaneously in space, and improve the sensitivity and high refresh frame rate in the fluorescence mode.

[0006] To achieve the above-mentioned purpose, the present application provides a spectral imaging method of a 3D electronic endoscope, in which two optical groups that are mirror images of each other are provided in the insertion part of the 3D electronic endoscope; in each optical group, a first objective lens, a beam splitter prism, and a second objective lens are provided in sequence from the light entrance of the optical group to the fluorescence sensor; the two beam splitting directions of the beam splitter prism are perpendicular to each other, one of which is directed toward the fluorescence sensor, and the other is directed toward the white light sensor; the second objective lens is provided with a first plano-convex lens, a second plano-convex lens, a first meniscus lens, and a second meniscus lens in sequence from the beam splitter prism to the fluorescence sensor; the spectral imaging method of the 3D electronic endoscope comprises: When the 3D electronic endoscope receives incident light, it converges the incident light to the beam splitter prism in each optical group based on the first objective lens in each optical group; Based on the beam splitter in each optical group, the fluorescence in the incident light is transmitted to the second objective lens in each optical group, and the white light in the incident light is reflected to the white light sensor corresponding to each optical group; and the fluorescence is focused to the fluorescence sensor corresponding to each optical group through the second objective lens in each optical group; Based on the white light sensor and fluorescence sensor corresponding to each light group, the received light is photoelectrically converted to obtain the corresponding white light image and fluorescence image; The video streams corresponding to the white light image and the fluorescent image are transmitted to the back-end image processor for processing, and the corresponding 3D images are generated and output to the 3D display device for display.

[0007] Optionally, the step of transmitting the video streams corresponding to the white light image and the fluorescent image to a back-end image processor for processing, generating a corresponding 3D image and outputting it to a 3D display device for display includes: Based on high-speed SerDes (SERializer / DESerializer), the corresponding video streams of white light images and fluorescent images are transmitted to the back-end image processor; After the image processor processes the video stream through various ISPs, it generates the corresponding 3D image and outputs it to the 3D display device for display.

[0008] To achieve the above-mentioned purpose, the present application also provides a 3D electronic endoscope, in which two light groups that are mirror images of each other are provided in the insertion part of the 3D electronic endoscope; in each light group, a first objective lens, a beam splitter prism and a second objective lens are provided in sequence from the light entrance of the light group to the fluorescence sensor; the two beam splitting directions of the beam splitter prism are perpendicular to each other, one of which is toward the fluorescence sensor, and the other is toward the white light sensor; the second objective lens is provided with a first plano-convex lens, a second plano-convex lens, a first meniscus lens and a second meniscus lens in sequence from the beam splitter prism to the fluorescence sensor; when the 3D electronic endoscope is in use, the beam splitting imaging method of the 3D electronic endoscope as described above is implemented.

[0009] Optionally, a first filter is further provided in the first objective lens; and a second filter is further provided between the first plano-convex lens and the second plano-convex lens.

[0010] The present application provides a 3D electronic endoscope with a spectroscopic imaging method and a 3D electronic endoscope. The 3D electronic endoscope adopts a design that separates white light and fluorescence in space, and performs secondary imaging on fluorescence. This can increase secondary fluorescence imaging without changing the white light path, and achieve the separation of fluorescence and white light paths in space, so that white light and fluorescence can be collected separately and simultaneously, without the need for staggered exposure in time, which greatly improves the efficiency of using a 3D electronic endoscope to perform fluorescence imaging and white light imaging simultaneously, improves the sensitivity and high refresh frame rate in the fluorescence mode, and correspondingly saves the cost of fluorescence and white light collection. Moreover, the insertion part of the 3D electronic endoscope does not need to be designed to be confocal for fluorescence and white light, that is, it can achieve separate focusing of fluorescence and white light, while ensuring the clarity of the two images, it also reduces the difficulty of equipment design. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of the optical group architecture of an existing 3D electronic endoscope; Figure 2 This is a schematic diagram of the internal structure of a 3D electronic endoscope in one embodiment of the present application; Figure 3 This is a schematic diagram of the steps of a spectral imaging method of a 3D electronic endoscope in one embodiment of the present application; Figure 4 This is a schematic diagram of the internal structure of the insertion portion of a 3D electronic endoscope in one embodiment of the present application; Figure 5 This is an MTF performance diagram of fluorescence secondary imaging of a 3D electronic endoscope in one embodiment of the present application; Figure 6 Schematic diagram of reflection and transmission of each surface of the beam splitter prism in a 3D electronic endoscope in one embodiment of the present application; Figure 7 This is a schematic diagram of the stroboscopic mode of each wavelength band of color light in a 3D electronic endoscope in an embodiment of the present application; Figure 8 This is a general schematic diagram of a 3D electronic endoscope and related equipment in one embodiment of the present application.

[0012] Description of reference numerals: 1. Fluorescence sensor; 2. First objective lens; 3. Beam splitter prism; 4. Second objective lens; 5. White light sensor; 6. Sapphire window; 7. Carrier board; 8. First protective glass; 9. Second protective glass; 21. First optical filter; 41. First plano-convex lens; 42. Second plano-convex lens; 43. First meniscus lens; 44. Second meniscus lens; 45. Second optical filter.

[0013] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0014] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0015] In addition, if the descriptions of "first", "second", etc. are involved in this application, they are only used for descriptive purposes (such as for distinguishing the same or similar features), and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0016] In one embodiment, a 3D electronic endoscope spectroscopic imaging method is provided, which is applied to a 3D electronic endoscope, wherein reference is made to Figure 2 The 3D electronic endoscope comprises a handle portion and an insertion portion (the outer shell of the insertion portion may be a slender metal tube), and in the insertion portion of the 3D electronic endoscope, two optical groups which are mirror images of each other are provided; in each optical group, a first objective lens 2, a beam splitter prism 3 and a second objective lens 4 are provided in sequence from the light entrance of the optical group to the fluorescence sensor 1; the two beam splitting directions of the beam splitter prism 3 are perpendicular to each other, one of which is directed toward the fluorescence sensor 1, and the other is directed toward the white light sensor 5; the second objective lens 4 is provided with a first plano-convex lens 41, a second plano-convex lens 42, a first meniscus lens 43 and a second meniscus lens 44 in sequence from the beam splitter prism 3 to the fluorescence sensor 1.

[0017] In this embodiment, the design of two mirror-image light groups can ensure that the same scene is captured from two different perspectives, which is conducive to the subsequent generation of accurate 3D images. Each light group is equipped with a corresponding fluorescence sensor 1 and white light sensor 5.

[0018] When the handle part and the insertion part of the 3D electronic endoscope are combined, the end of the light group close to the handle part can be defined as the proximal end, and the end of the light group away from the handle part can be defined as the distal end (correspondingly, the part of the insertion part away from the handle part is the top, and the part close to the handle part is the tail; and the endoscope lens corresponding to each light group is arranged at the top of the insertion part); wherein, the fluorescence sensor 1 corresponding to each light group is arranged behind the distal end of the light group, and the photosensitive surface of the fluorescence sensor 1 faces the light group.

[0019] The far end of the light group is the light group light entrance, and in each light group, a first objective lens 2, a beam splitter prism 3 and a second objective lens 4 are sequentially arranged from the light group light entrance to the fluorescence sensor 1; wherein the beam splitter prism 3 can transmit fluorescence and reflect white light, thereby dividing the incident light converged by the first objective lens 2 into two light paths, and the two beam splitting directions of the beam splitter prism 3 are perpendicular to each other, one of the beam splitting directions (fluorescence transmission direction) is toward the location of the fluorescence sensor 1, and the other beam splitting direction (white light reflection direction) is toward the location of the white light sensor 5, and the photosensitive surface of the white light sensor 5 is toward the beam splitter prism 3.

[0020] Optionally, the first objective lens 2 is an optical imaging lens with a positive total focal length, which can converge light in the object space to the image space to achieve the purpose of imaging. The first objective lens 2 can be a general 3D electronic endoscope imaging lens.

[0021] The second objective lens 4 is used to focus the fluorescence transmitted by the beam splitter prism 3 onto the photosensitive surface of the fluorescence sensor 1 .

[0022] In this embodiment, based on the above-mentioned 3D electronic endoscope, Figure 3 , the spectral imaging method of the 3D electronic endoscope includes: Step S10, when the 3D electronic endoscope receives incident light, based on the first objective lens in each optical group, the incident light is focused to the beam splitter prism in each optical group; Step S20: Based on the beam splitter in each light group, transmit the fluorescence in the incident light to the second objective lens in each light group, and reflect the white light in the incident light to the white light sensor corresponding to each light group; and converge the fluorescence to the fluorescence sensor corresponding to each light group through the second objective lens in each light group; Step S30, based on the white light sensor and the fluorescence sensor corresponding to each light group, respectively perform photoelectric conversion on the received light to obtain a corresponding white light image and fluorescence image; Step S40: Transmit the video streams corresponding to the white light image and the fluorescent image to the back-end image processor for processing, generate corresponding 3D images and output them to the 3D display device for display.

[0023] In this embodiment, the incident light is emitted from the edge of an object (generally the object to be measured) in the object space (such as the object reflects the searchlight). When the insertion part of the 3D electronic endoscope receives the incident light, the first objective lens 2 in each optical group in the insertion part will converge the incident light and emit the converged incident light to the dichroic prism 3 in each optical group.

[0024] Furthermore, the beam splitter prism 3 in each light group will split the incident light after convergence into two paths, one of which is fluorescent light and the other is white light.

[0025] In each light group, the beam splitter prism 3 reflects the white light in the incident light, and the reflected white light is focused onto the photosensitive surface of the white light sensor 5 for imaging.

[0026] At the same time, in each light group, the beam splitter prism 3 transmits the fluorescence in the incident light, focuses it in the space near the rear of the beam splitter prism 3 to form a primary image, and continues to emit it to the second objective lens 4. Then the fluorescence light will be converged again through the second objective lens 4, and the converged fluorescence light will be formed into a secondary image on the photosensitive surface of the fluorescence sensor 1 behind the second objective lens 4 (the primary image is an inverted real image relative to the object plane, and the secondary image is an upright real image relative to the object plane).

[0027] When the white light sensor 5 and the fluorescence sensor 1 corresponding to each light group receive the corresponding white light and fluorescence light, the received light is converted into an electrical signal, and the corresponding white light image and fluorescence image can be obtained.

[0028] By collecting white light images and fluorescence images in a certain time series, corresponding video stream data can be formed, and the video streams corresponding to the white light images and fluorescence images are further transmitted to the back-end image processor for processing.

[0029] The image processor combines the video streams corresponding to the white light images and fluorescence images collected based on different light groups to generate the corresponding 3D images.

[0030] Furthermore, these 3D images are transmitted to a 3D display device for display.

[0031] Optionally, the white light sensor 5 is a color image sensor; the fluorescence sensor 1 can be a black and white image sensor or a color graphic sensor, preferably a black and white image sensor.

[0032] In one embodiment, the 3D electronic endoscope adopts a design that separates white light and fluorescence in space, and performs secondary imaging of the fluorescence. This can increase the secondary imaging of fluorescence without changing the white light optical path, thereby realizing the separation of the fluorescence and white light optical paths in space, so that white light and fluorescence can be collected separately and simultaneously without the need for staggered exposure in time, thereby realizing the 3D electronic endoscope to perform fluorescence imaging and color white light imaging in space simultaneously, thereby improving the sensitivity and high refresh frame rate in the fluorescence mode.

[0033] In one embodiment, based on the above embodiment, the step of transmitting the video streams corresponding to the white light image and the fluorescent image to the back-end image processor for processing, generating the corresponding 3D image and outputting it to the 3D display device for display includes: Based on high-speed SerDes, the corresponding video streams of white light images and fluorescent images are transmitted to the back-end image processor; After the image processor processes the video stream through various ISPs, it generates the corresponding 3D image and outputs it to the 3D display device for display.

[0034] In this embodiment, high-speed SerDes is an efficient data transmission technology used to transmit the video streams of white light images and fluorescent images from the sensor to the image processor. High-speed SerDes increases the data transmission rate through serial transmission, is suitable for processing high-resolution, high-frame-rate video streams, and can process large-bandwidth data to ensure that the image data remains intact and accurate during high-speed transmission.

[0035] Optionally, the image processor uses various types of ISPs (Image Signal Processors) to process the video stream. The ISP is responsible for image noise reduction, sharpening, color correction, contrast adjustment, etc. to optimize image quality.

[0036] Optionally, the white light video stream is processed to generate a clear white light image; and the fluorescence video stream is processed to generate a clear fluorescence image.

[0037] The processed white light image and fluorescent image video stream data are fused to generate the corresponding 3D image. In particular, the white light image and the fluorescent image are synthesized from different perspectives through image registration and depth information reconstruction technology to generate the corresponding 3D view.

[0038] Furthermore, the generated 3D image is output to a 3D display device for display. The 3D display device can provide a stereoscopic effect, allowing the observer to perceive the depth and spatial structure of the image.

[0039] In one embodiment, based on the above embodiment, the second objective lens 4 is used to realize secondary imaging on the fluorescence sensor 1 .

[0040] In this embodiment, the size of the fluorescence image formed directly behind the beam splitter prism 3 is relatively large. If the second objective lens 4 is directly discarded and the fluorescence sensor 1 is arranged behind the beam splitter prism 3, a large-sized fluorescence sensor 1 is required to form an image. However, the insertion part of the 3D electronic endoscope generally has a small cross-sectional area for convenient detection, which is not conducive to the placement of a large-sized fluorescence sensor 1.

[0041] Therefore, by arranging the second objective lens 4 between the beam splitter prism 3 and the fluorescence sensor 1, and using the second objective lens 4 to lengthen the fluorescence optical path, the fluorescence light can realize secondary imaging on the fluorescence sensor 1, and at the same time, the second objective lens 4 can reduce the imaging size of the fluorescence on the fluorescence sensor 1, so that in the case of limited space, a smaller size fluorescence sensor 1 can be used, while still being able to obtain a high-quality fluorescence image. The problem of using a large-size sensor is avoided, making the insertion part design of the 3D electronic endoscope more compact and practical.

[0042] In one embodiment, based on the above embodiment, the dichroic prism 3 is used to reflect white light with a wavelength of 400nm~650nm and transmit dual fluorescence with a wavelength of 685~750nm & 810~1000nm.

[0043] In this embodiment, the beam splitter prism 3 is equivalent to a relatively thick flat glass, and its optical focal length is 0. It is formed by gluing two optical prisms together, and a dichroic film is coated on the gluing surface, which is a dielectric film.

[0044] The dichroic film can be coated by optical vacuum evaporation technology, and the film system is designed to reflect white light with a wavelength of 400nm~650nm and transmit dual fluorescence with a wavelength of 685~750nm&810~1000nm. Therefore, the beam splitter prism 3 can separate the white light imaging optical path (wavelength 400nm~650nm) and the fluorescence imaging optical path (wavelength 685~750nm&810~1000nm) in space, so that the white light and fluorescence can propagate in two different vertical directions.

[0045] In addition, a 3D electronic endoscope is also provided in the embodiment of the present application. Figure 2 In the insertion part of the 3D electronic endoscope, two optical groups are arranged in mirror image to each other; in each optical group, a first objective lens 2, a beam splitter prism 3 and a second objective lens 4 are arranged in sequence from the light entrance of the optical group to the fluorescence sensor 1; the two beam splitting directions of the beam splitter prism 3 are perpendicular to each other, one of which is toward the fluorescence sensor 1, and the other is toward the white light sensor 5; the second objective lens 4 is provided with a first plano-convex lens 41, a second plano-convex lens 42, a first meniscus lens 43 and a second meniscus lens 44 in sequence from the beam splitter prism 3 to the fluorescence sensor 1.

[0046] In this embodiment, the design of two optical groups arranged as mirrors can ensure that images of the same scene are captured from two different perspectives, which is beneficial for subsequent generation of accurate 3D images. Among them, each optical group is configured with a corresponding fluorescence sensor 1 and white light sensor 5.

[0047] When the handle part and the insertion part of the 3D electronic endoscope are combined, the end of the optical group close to the handle part can be defined as the proximal end, and the end of the optical group far from the handle part can be defined as the distal end; among them, the fluorescence sensor 1 corresponding to each optical group is arranged after the distal end of the optical group, and the photosensitive surface of the fluorescence sensor 1 faces the optical group.

[0048] The distal end of the optical group is the light inlet of the optical group, and in each optical group, a first objective lens 2, a beam splitter prism 3, and a second objective lens 4 are sequentially arranged between the light inlet of the optical group and the fluorescence sensor 1; among them, the beam splitter prism 3 can transmit fluorescence and reflect white light, so as to divide the incident light converged by the first objective lens 2 into two optical paths, and the two splitting directions of the beam splitter prism 3 are perpendicular to each other. One splitting direction (fluorescence transmission direction) faces the position where the fluorescence sensor 1 is located, and the other splitting direction (white light reflection direction) faces the position where the white light sensor 5 is located, and the photosensitive surface of the white light sensor 5 faces the beam splitter prism 3.

[0049] Optionally, the first objective lens 2 is an optical imaging lens with a positive total optical power, which can converge the light in the object space to the image space to achieve the purpose of imaging. Among them, the first objective lens 2 can adopt a general 3D electronic endoscope imaging lens.

[0050] Optionally, the value range of the total optical power of the first objective lens 2 can be (0.25, 0.5), and the value range of the effective focal length can be designed to be between 2 mm and 4 mm; the length from the vertex of the first optical surface (the surface closest to the light inlet of the optical group) of the first objective lens 2 to the vertex of the last optical surface is between 10 and 15 mm; in order to facilitate the insertion of the beam splitter prism 3 in front of the image plane, the design parameters of the overall optical length (TTL) and the back focal length (BFL) of the first objective lens 2 can be longer, such as setting 15 mm < TTL < 20 mm and 4 mm < BFL < 7 mm; in order to more easily realize the miniaturization of the lens, the aperture number (FNO.) can be set to be greater than 5.

[0051] Among them, the second objective lens 4 is used to converge the fluorescence transmitted by the beam splitter prism 3 onto the photosensitive surface of the fluorescence sensor 1.

[0052] When the 3D electronic endoscope is in use, it realizes a spectroscopic imaging method of a 3D electronic endoscope as described in the above embodiment.

[0053] Optionally, the total optical length of the second objective lens 4 cannot be too long, so the distance from the convex vertex of the first plano-convex lens 41 to the image plane is controlled within 10 mm; at the same time, the diameter of the optical lens cannot be too large, otherwise it cannot be loaded into the lens barrel of the 3D electronic endoscope, and the diameter of all lenses should be controlled within the range of less than or equal to Ø3 mm.

[0054] Optionally, the first plano-convex lens 41 has a positive optical power, the optical power is between 0.2 and 0.5, and the effective focal length is between 2 mm and 5 mm. In fact, the first plano-convex lens 41 is equivalent to a relay lens, which is used to converge the fluorescence transmitted by the beam splitter prism 3 and couple it into the second plano-convex lens 42 again.

[0055] Optionally, the second plano-convex lens 42 has a positive optical power, the optical power is between 0.2 and 0.5, and the effective focal length is between 2 mm and 5 mm.

[0056] Optionally, the first meniscus lens 43 has a negative optical focal length, close to a hyperhemispherical lens, with an optical focal length between -0.07 and -0.2 and an effective focal length between -5 mm and -15 mm.

[0057] The second meniscus lens 44 has positive optical power, the optical power is between 0.2 and 0.5, and the effective focal length is between 2 mm and 5 mm.

[0058] In one embodiment, the second lens is composed of the first plano-convex lens 41 , the second plano-convex lens 42 , the first meniscus lens 43 and the second meniscus lens 44 , so that the effect of converging the fluorescent light onto the fluorescent sensor 1 can be optimized.

[0059] It should be noted that the composition scheme of the second objective lens 4 (and Figure 2 , Figure 4 The composition of the second objective lens 4 shown in the figure is only a preferred exemplary structure, and the specific composition of the second objective lens 4 may not be limited to the above-mentioned embodiments; as long as the structural composition of the second objective lens 4 can lengthen the fluorescence optical path so that the fluorescence light can be imaged twice on the fluorescence sensor 1, while reducing the imaging size of the fluorescence on the fluorescence sensor 1 (such as using other types or numbers of lens composition schemes to achieve the same function of the second objective lens 4), it is within the scope of protection of the present application.

[0060] In one embodiment, based on the above embodiment, referring to Figure 4 A first filter 21 is also provided in the first objective lens 2 ; a second filter 45 is also provided between the first plano-convex lens 41 and the second plano-convex lens 42 .

[0061] In this embodiment, the first filter 21 is used to transmit white light of 400~650nm, dual fluorescence of 685~750nm & 810~1000nm, and cut off laser of 775~795nm; the second filter 45 is used to transmit dual fluorescence of 685~750nm & 810~1000nm, and cut off white light of 400~650nm and laser of 650~670nm.

[0062] Optionally, in order to avoid interference of 775~795nm laser (excitation light) on fluorescence imaging, it is necessary to cut off the 775~795nm laser. The first filter 21 can be a notch plate, located inside the first objective lens 2, and its main function is to cut off the 775~795nm laser (excitation light), and the cutoff depth must be above OD6; at the same time, the first filter 21 can transmit 400~650nm white light, 685~750nm & 810~1000n dual fluorescence, and the transmittance is >96%.

[0063] The specific coating method of the first filter 21 can be optical vacuum evaporation coating technology. The light incident surface (close to the object surface) of the first filter 21 is coated with an AR anti-reflection film to achieve a reflectivity of 400~650nm white light and 685~750nm&810~1000nm dual fluorescence <0.8%; the light exit surface of the first filter 21 (close to the fluorescence sensor 1) is coated with a cut-off film to cut off the 775~795nm laser, and the cut-off depth needs to reach OD6 or above; at the same time, it is necessary to transmit 400~650nm white light and 685~750nm&810~1000nm dual fluorescence, with a transmittance of >96%. Considering the cost of glass materials and the need for high light transmittance, H-K9L can be selected as the coating substrate of the first filter 21, and the thickness can be 0.4mm.

[0064] Optionally, in order to prevent the 650~670nm laser (excitation light) from interfering with the fluorescence imaging, the 650~670nm laser needs to be cut off. The second filter 45 can be a cut-off filter, located inside the second objective lens 4, and its main function is to cut off the 650~670nm laser and the 400~650nm white light at the same time, with a cut-off depth of more than OD6 to prevent the white light from affecting the fluorescence imaging; at the same time, it also needs to pass through the dual fluorescence of 685~750nm & 810~1000nm, with a transmittance of >96%. The specific coating method can be optical vacuum evaporation coating technology, and an AR anti-reflection film is coated on the light incident surface (close to the object surface) of the second filter 45 to achieve a reflectivity of 685~750nm&810~1000nm dual fluorescence <0.8%; a cut-off film is coated on the light exit surface (close to the fluorescence sensor 1) of the second filter 45 to cut off white light of 400~650nm and laser of 650~670nm, and the cut-off depth needs to reach OD6 or above, and at the same time, it needs to pass through 685~750nm&810~1000nm dual fluorescence, and the transmittance is>96%. It should be noted that in the actual coating area of ​​the filter, the central area has the best filtering effect, and the edge is relatively poor. Therefore, the position of the second filter 45 should be as close to the fluorescence sensor 1 as possible, so that the effective aperture of the second filter 45 can be smaller, and the light can be concentrated in the central area, so that the second filter 45 can achieve a better filtering effect. Considering the cost of glass materials and the need for high light transmittance, H-K9L can be selected as the coating substrate of the second filter 45, and the thickness can be 0.4 mm.

[0065] In one embodiment, based on the above embodiment, the gap between the first plano-convex lens 41 and the beam splitter prism 3 is in the range of (0 mm, 0.1 mm]; The gap between the second filter 45 and the first plano-convex lens 41 is in the range of [3.5 mm, 4.5 mm]; The gap between the second filter 45 and the second plano-convex lens 42 is in the range of [0.2 mm, 1 mm]; The gap between the second plano-convex lens 42 and the first meniscus lens 43 is in the range of [0.1 mm, 0.5 mm]; The gap between the first meniscus lens 43 and the second meniscus lens 44 is in the range of [0.1 mm, 0.8 mm].

[0066] In this embodiment, in order to realize the secondary imaging of the fluorescent part at the lowest possible cost, and at the same time control the total length of the secondary imaging optical system, and the resolution should also be close to the diffraction limit, the following specific preferred solutions are listed: Since the first plano-convex lens 41 acts as a relay lens, mainly coupling the primary image formed by the first objective lens 2 to the secondary optical system, and the first plano-convex lens 41 is just located at the position of the primary image, the first plano-convex lens 41 is also a field stop. The gap between the first plano-convex lens 41 and the beam splitter prism 3 should be as small as possible, but cannot be zero, and the gap range can be set to (0mm, 0.1mm], and the thickness of the first plano-convex lens 41 can also be set to between 0.5 and 1mm, which is also conducive to controlling the total length of the secondary optical system.

[0067] The second plano-convex lens 42 and the first plano-convex lens 41 are shared lenses with the same parameters. For cold processing, one less set of molds can be opened to achieve low cost. The main function of the second plano-convex lens 42 is to bear a part of the positive focal length of the first plano-convex lens 41, so that the focal length of the first plano-convex lens 41 is moderate and the focal length distribution of the entire system is more reasonable. Since the first plano-convex lens 41 is a relay lens, the gap between the second plano-convex lens 42 and the first plano-convex lens 41 will be larger, and the gap can be set between 3 and 5 mm.

[0068] The second filter 45 is between the first plano-convex lens 41 and the second plano-convex lens 42. Considering that the coating thickness of the second filter 45 is relatively thick, in order to avoid distortion of the filter caused by too thick film layer, the thickness of the second filter 45 should be set to be above 0.3 mm. In order to make the aperture of the filter as small as possible, the filter should be far away from the first plano-convex lens 41, and the gap between the filter and the first plano-convex lens 41 can be set between 3.5 and 4.5 mm, and the gap between the filter and the second plano-convex lens 42 can be set between 0.2 and 1 mm.

[0069] The first meniscus lens 43 has a negative focal length and can diverge light. It mainly forms a positive and negative lens group together with the second plano-convex lens 42, which plays a role in reducing aberrations and improving image quality. Therefore, the gap between the first meniscus lens 43 and the second plano-convex lens 42 is relatively small, and the gap can be set between 0.1 and 0.5, which is also conducive to controlling the total length of the secondary optical system.

[0070] The second meniscus lens 44 is a convex lens, and its main function is to converge the divergent light of the first meniscus lens 43 to the image plane to realize imaging. Therefore, the gap between the second meniscus lens 44 and the first meniscus lens 43 is also relatively small, and the gap can be set between 0.1 and 0.8, which is also conducive to controlling the total length of the secondary optical system.

[0071] In this way, the total length of the entire fluorescence secondary imaging optical system can be controlled between 6 and 10 mm, and fluorescence secondary imaging can be achieved with a relatively short total optical length. At the same time, a relatively high resolution of fluorescence secondary imaging is achieved. Figure 5This is the MTF performance diagram of fluorescence secondary imaging. It can be seen that the resolution reaches 140 lp / mm (MTF>0.2) in the whole field of view, which is close to the diffraction limit. The relatively short total optical length is used to achieve a higher resolution of fluorescence secondary imaging.

[0072] In one embodiment, based on the above embodiment, referring to Figure 4 A sapphire window 6 is provided at the light entrance of the light group.

[0073] In this embodiment, the sapphire window 6 is a flat glass with an optical focal length of 0 and a thickness that can be designed to be between 0.3 mm and 1 mm. It has a relatively high hardness and mainly serves to protect the internal optical components.

[0074] In one embodiment, based on the above embodiment, referring to Figure 4 A carrier board 7 is provided between the two optical groups, and the white light sensors 5 corresponding to the two optical groups are arranged in parallel on two sides of the carrier board 7.

[0075] In this embodiment, the carrier plate 7 can separate the two optical groups to avoid mutual interference between different optical groups. For example, if the cross section of the insertion part is circular, the carrier plate 7 can divide the internal space of the insertion part into two semicircles, and each optical group is in a different semicircular space.

[0076] In one embodiment, based on the above embodiment, referring to Figure 4 A first protective glass 8 is provided between the white light sensor 5 and the beam splitter prism 3 .

[0077] In this embodiment, the first protective glass 8 is a flat glass with an optical focal length of 0 and a thickness between 0.2 mm and 0.5 mm, and mainly serves to protect the white light sensor 5 .

[0078] certainly Figure 4 What is shown is only a preferred exemplary structure of the 3D electronic endoscope, but in other optional solutions, the first protective glass 8 may not be provided in the insertion part.

[0079] Optionally, the second protective glass 9 is a flat glass with an optical focal length of 0 and a thickness between 0.2 mm and 0.5 mm, and mainly serves to protect the fluorescent sensor 1 .

[0080] Considering the need to reserve assembly margin for back focus, the gap between the second meniscus lens 44 and the second protective glass 9 needs to be greater than 0.3 mm.

[0081] certainly Figure 4 What is shown is only a preferred exemplary structure of the 3D electronic endoscope, but in other optional solutions, the second protective glass 9 may not be provided in the insertion part.

[0082] In one embodiment, based on the above embodiment, the basic working principle of the white light imaging optical path is as follows: white light is visible light with a wavelength of 400~650nm, emitted from the edge of the object in the object space, and the white light first passes through the sapphire window 6 to enter the 3D electronic endoscope imaging optical system. The sapphire window 6 is fully transparent to white light and dual fluorescent light, and the outer surface (close to the object surface) needs to be coated with a waterproof film; the inner surface (close to the sensor) needs to be coated with an AR anti-reflection film, and the reflectivity of the 400~1000nm band is <0.8%. Then, after the light passes through the first objective lens 2 and the first internal filter 21, the white light of 400~650nm can be fully transmitted. Then, after passing through the first objective lens 2 and the first internal filter 21, the white light of 400~650nm can also be fully transmitted. Then, refer to Figure 6 , 400~650nm white light is incident on the S1 surface of the beam splitter prism 3, reflected downward on the S2 surface, and transmitted out on the S3 surface. Among them, the S1 surface and the S3 surface are highly transparent to the 400~650nm white light, and the S2 surface is highly reflective to the 400~650nm white light. Finally, the 400~650nm white light passes through the first protective glass 8 and is focused on the surface of the white light sensor 5 to realize imaging.

[0083] The basic working principle of the fluorescence imaging optical path is as follows: when the 650nm~670nm laser is irradiated on the fluorescent agent, the fluorescent agent absorbs the light energy of 650nm~670nm and excites the first fluorescence of 685~750nm; when the 775nm~795nm laser is irradiated on the fluorescent agent, the fluorescent agent absorbs the light energy of 775nm~795nm and excites the second fluorescence of 810~1000nm; the combination of the first fluorescence and the second fluorescence can realize the dual fluorescence imaging technology. The dual fluorescence light with a wavelength of 685~750nm&810~1000nm excited by these two lasers and fluorescent agents first enters the 3D electronic endoscope through the sapphire window 6. The sapphire window 6 is fully transparent to white light and dual fluorescence light. The outer surface (close to the object surface) needs to be coated with a waterproof film; the inner surface (close to the sensor) needs to be coated with an AR anti-reflection film, and the reflectivity of the 400~1000nm band is <0.8%. Then, after the light passes through the first objective lens 2 and the first filter 21 inside, the dual fluorescent light of 685~750nm & 810~1000nm can all pass through. However, the 775nm~795nm laser will be cut off by the first filter 21 and cannot pass through. Then, the dual fluorescent light of 685~750nm & 810~1000nm can pass through the S1 surface, S2 surface, and S4 surface of the beam splitter prism 3 in turn, and the light is emitted from the S4 surface and focused to form a primary image. The position of the primary image is located on the first plano-convex lens 41 of the second objective lens 4. Among them, the S1 surface, S2 surface, and S4 surface are all highly transparent to the dual fluorescent light of 685~750nm & 810~1000nm. Then, after the dual fluorescent light of 685~750nm & 810~1000nm passes through the second objective lens 4, the light will converge again. Finally, after passing through the second protective glass 9, the dual fluorescent light of 685-750nm & 810-1000nm is focused on the surface of the fluorescent sensor 1 to form a secondary image. The primary image is an inverted real image relative to the object plane, and the secondary image is an upright real image relative to the object plane.

[0084] This application discloses a 3D electronic endoscope that combines white light & fluorescence spectroscopic imaging, secondary imaging, and dual fluorescence imaging technology, which can achieve three working modes: 1. The mode in which the white light of 400~650nm and the first fluorescence of 685~750nm work simultaneously. For this working mode, the white light of the cold light source and the 650nm~670nm laser need to be turned on at the same time. The 650nm~670nm laser can excite the first fluorescence of 685~750nm. Therefore, the white light of 400~650nm and the first fluorescence of 685~750nm can be imaged simultaneously.

[0085] 2. The mode in which the white light of 400~650nm and the second fluorescence of 810~1000nm work simultaneously. For this working mode, the white light of the cold light source and the 775nm~795nm laser need to be turned on at the same time. The 775nm~795nm laser can excite the second fluorescence of 810~1000nm. Therefore, the white light of 400~650nm and the second fluorescence of 810~1000nm can be imaged simultaneously.

[0086] 3. The mode of simultaneous operation of 400~650nm white light and 685~750nm & 810~1000nm dual fluorescence. For this working mode, it is necessary to turn on the white light of the cold light source, 650nm~670nm laser, and 775nm~795nm laser at the same time.

[0087] It should be noted that the 650nm~670nm and 775nm~795nm lasers need to use stroboscopic method at the algorithm level to achieve dual fluorescence imaging. Therefore, 400~650nm white light and 685~750nm&810~1000nm dual fluorescence can be imaged simultaneously. Figure 7 As shown, the 660nm laser band is 650nm~670nm, and the 785nm laser band is 775nm~795nm.

[0088] In one embodiment, based on the above embodiment, referring to Figure 8 The 3D electronic endoscope also includes a handle portion detachably connected to the insertion portion, and an image acquisition and transmission control module (not shown in the figure) is provided in the handle portion. A communication connection is established between the image acquisition and transmission control module and the white light sensor 5 and the fluorescence sensor 1 in the insertion portion.

[0089] The image acquisition and transmission control module is used to transmit the collected white light and fluorescent images to the back-end image processor (that is, the image acquisition and transmission control module in the handle is connected to the back-end image processor through the corresponding video cable) for processing.

[0090] Among them, a communication connection can also be established between the back-end image processor and the 3D display device through a corresponding video cable, and when the image processor generates a 3D image, the video cable is used to transmit the 3D image to the 3D display device for display.

[0091] In summary, the spectral imaging method of the 3D electronic endoscope and the 3D electronic endoscope provided in the embodiments of the present application, the 3D electronic endoscope adopts a design of separating white light and fluorescence in space, and performs secondary imaging of the fluorescence, so that the secondary imaging of fluorescence can be increased without changing the white light optical path, thereby realizing the separation of the fluorescence and white light optical paths in space, so that white light and fluorescence can be collected separately and simultaneously without the need for staggered exposure in time, thereby realizing the 3D electronic endoscope to perform fluorescence imaging and color white light imaging in space simultaneously, thereby improving the sensitivity and high refresh frame rate in the fluorescence mode.

[0092] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0093] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A 3D electronic endoscope spectroscopic imaging method, characterized in that: In the insertion part of the 3D electronic endoscope, two optical groups are arranged in mirror image to each other; in each optical group, a first objective lens, a beam splitter prism and a second objective lens are arranged in sequence from the light entrance of the optical group to the fluorescence sensor; the two beam splitting directions of the beam splitter prism are perpendicular to each other, one beam splitting direction is toward the fluorescence sensor, and the other beam splitting direction is toward the white light sensor; the second objective lens is arranged in sequence from the beam splitter prism to the fluorescence sensor. The beam splitting imaging method of the 3D electronic endoscope includes: When the 3D electronic endoscope receives incident light, it converges the incident light to the beam splitter prism in each optical group based on the first objective lens in each optical group; Based on the beam splitter in each optical group, the fluorescence in the incident light is transmitted to the second objective lens in each optical group, and the white light in the incident light is reflected to the white light sensor corresponding to each optical group; and the fluorescence is focused to the fluorescence sensor corresponding to each optical group through the second objective lens in each optical group; Based on the white light sensor and fluorescence sensor corresponding to each light group, the received light is photoelectrically converted to obtain the corresponding white light image and fluorescence image; The video streams corresponding to the white light image and the fluorescent image are transmitted to the back-end image processor for processing, and the corresponding 3D images are generated and output to the 3D display device for display.

2. The spectral imaging method of a 3D electronic endoscope according to claim 1, characterized in that: The step of transmitting the video streams corresponding to the white light image and the fluorescent image to the back-end image processor for processing, generating the corresponding 3D image and outputting it to the 3D display device for display comprises: Based on high-speed SerDes, the corresponding video streams of white light images and fluorescent images are transmitted to the back-end image processor; After the image processor processes the video stream through various ISPs, it generates the corresponding 3D image and outputs it to the 3D display device for display.

3. A 3D electronic endoscope, characterized in that: In the insertion part of the 3D electronic endoscope, there are provided two optical groups which are mirror images of each other; in each optical group, a first objective lens, a dichroic prism and a second objective lens are sequentially provided from the light entrance of the optical group to the fluorescence sensor; the two dichroic directions of the dichroic prism are perpendicular to each other, one of which is toward the fluorescence sensor, and the other is toward the white light sensor; the second objective lens is sequentially provided with a first plano-convex lens, a second plano-convex lens, a first meniscus lens and a second meniscus lens in the direction from the dichroic prism to the fluorescence sensor; when the 3D electronic endoscope is used, the dichroic imaging method of the 3D electronic endoscope as described in claim 1 or 2 is implemented.

4. The 3D electronic endoscope according to claim 3, characterized in that: A first filter is also provided in the first objective lens; and a second filter is also provided between the first plano-convex lens and the second plano-convex lens.

5. The 3D electronic endoscope according to claim 4, characterized in that: The gap between the first plano-convex lens and the beam splitter prism is in the range of (0 mm, 0.1 mm]; The gap between the second filter and the first plano-convex lens is in the range of [3.5 mm, 4.5 mm]; The gap between the second filter and the second plano-convex lens is in the range of [0.2 mm, 1 mm]; The gap between the second plano-convex lens and the first meniscus lens is in the range of [0.1 mm, 0.5 mm]; The gap between the first meniscus lens and the second meniscus lens is in the range of [0.1 mm, 0.8 mm].

6. The 3D electronic endoscope according to claim 4, characterized in that: The first filter is a notch filter, which is used to transmit white light of 400-650nm, dual fluorescence of 685-750nm & 810-1000nm, and cut off laser light of 775-795nm; The second filter is a cutoff filter, which is used to transmit dual fluorescence of 685~750nm & 810~1000nm, and cut off white light of 400~650nm and laser of 650~670nm.

7. The 3D electronic endoscope according to claim 3, characterized in that: A sapphire window is provided at the light entrance of the light group.

8. The 3D electronic endoscope according to claim 3, characterized in that: A carrier board is arranged between the two optical groups, and white light sensors corresponding to the two optical groups are arranged in parallel on both sides of the carrier board.

9. The 3D electronic endoscope according to claim 3, characterized in that: A first protective glass is provided between the white light sensor and the beam splitter prism; and / or a second protective glass is provided between the fluorescence sensor and the second objective lens.

10. The 3D electronic endoscope according to any one of claims 3 to 9, characterized in that: The 3D electronic endoscope further comprises a handle portion detachably connected to the insertion portion, wherein the handle portion is provided with an image acquisition and transmission control module; The image acquisition and transmission control module is used to transmit the acquired white light and fluorescent images to the back-end image processor for processing via high-speed SerDes and video cables.

Images