Three-dimensional fluorescence electronic endoscope optical system using partitioning imaging and image calibration

Abstract

This invention provides a three-dimensional fluorescence electronic endoscope optical system employing partitioned imaging and image calibration. It includes a main tube with a left-path optical system and a right-path optical system, each comprising: an objective lens for receiving object images and performing preliminary imaging; a beam splitter for splitting the incident beam after imaging through the objective lens into a white light band and a near-infrared band; a CMOS image sensor with its photosensitive area divided into a white light imaging region and a fluorescence imaging region; an image processor for synthesizing the images from the two optical systems into three-dimensional image data; and an optical axis correction module for dynamically changing the horizontal or vertical offset of the left and right imaging optical axes by adjusting the attitude or position of the beam splitter or optical elements. This invention achieves flicker-free, high-frame-rate dual-band acquisition through CMOS partitioned synchronous imaging with the same focal plane. While maintaining a compact structure, it achieves precise alignment of the left and right optical paths and accurate registration of the dual bands, improving 3D observation comfort and imaging stability.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and specifically to a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration. Background Technology

[0002] Current real-time 3D fluorescence electronic endoscopes used in medical surgery primarily employ a time-division multiplexing method to acquire white light and fluorescence images. This approach has the following drawbacks: First, the actual image acquisition frame rate usually cannot reach 60Hz, and digital frame interpolation is needed to achieve a false high frame rate display, which can easily lead to image lag.

[0003] Secondly, due to time-division acquisition, the light source must constantly switch between white light and near-infrared laser illumination in a strobe manner. At the same time, in order to ensure the brightness of the image, the instantaneous power of the illumination source during strobe may be several times higher than the power during normal use. This is in fact equivalent to a flash light with saturation mode turned on. During surgery, the endoscope is often removed from the patient's body, which may affect the doctor's vision.

[0004] Furthermore, due to the limited internal space of the endoscope, adding more CMOS sensors would lead to problems such as increased system size and thicker insertion tube diameter. Current 3D fluorescence electronic endoscopes mostly use two independent CMOS sensors to acquire images from the left and right sides respectively, but the optical axis is fixed and cannot be adjusted. In actual surgery, the working distance of the endoscope often changes, causing the visual axis to deviate from the preset state, which may cause dizziness and fatigue in the viewer, affecting the surgical procedure. Although this optical axis deviation can be corrected by adjusting the horizontal or vertical distance between the two images on the monitor or host, this results in the surrounding areas of the displayed 3D image lacking a 3D effect. This not only affects the 3D display area but also causes discomfort for doctors and other personnel when viewing the peripheral 3D images, impacting usability. Summary of the Invention

[0005] To address the various shortcomings of existing technologies, this invention provides a three-dimensional fluorescence electronic endoscope optical system that employs partitioned imaging and image adjustment. Based on CMOS partitioned imaging and using optical axis correction, this system can achieve continuous acquisition and display of electronic 3D fluorescence and improve the 3D viewing experience at different imaging distances by adjusting the optical axis.

[0006] The present invention achieves the above objectives through the following technical solutions: A three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration includes: The main tube contains a left-path optical system and a right-path optical system symmetrically distributed along the central axis. Each of the two systems independently captures images of the same object, forming the basic optical structure for three-dimensional observation. An illumination beam guide, located inside the main lens barrel, is used to provide white light and near-infrared illumination; The left-path optical system and the right-path optical system each include: Objective lens, used to receive images of objects and perform preliminary imaging; A beam splitter, positioned behind the objective lens, splits the incident light beam after it has been imaged by the objective lens into a white light band and a near-infrared band. The CMOS image sensor has a photosensitive area divided into a white light imaging area and a fluorescence imaging area, which respectively receive the white light band image and the near-infrared band image after spectral splitting. The image processor is used to process the white light and fluorescence image data output by the CMOS image sensor and to synthesize the images from the left and right optical systems into three-dimensional image data. The optical axis correction module is used to dynamically change the horizontal or vertical offset of the left and right imaging optical axes by adjusting the attitude or position of the beam splitter or optical elements, in order to compensate for the visual axis deviation caused by the change of the endoscope working distance. This allows it to adapt to different working distances and ensure the comfort and clarity of three-dimensional observation without reducing the three-dimensional display area.

[0007] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided. The beam splitter is provided with a first beam splitter surface and a second beam splitter surface inside. When a white light beam enters the beam splitter, its path is transmitted through the second beam splitter surface, and after being reflected by the second beam splitter surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam splitter, its path is directly reflected at the first beam splitter surface, and it converges to be incident on the fluorescence imaging area of ​​the CMOS image sensor.

[0008] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided. The first beam-splitting surface is a first freeform surface, and the second beam-splitting surface is a reflective surface. When a white light beam enters the beam-splitting prism, its path is transmitted through the 45° reflective surface, and after being reflected by the reflective surface, it converges to form an image in the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam-splitting prism, its path is directly reflected at the first freeform surface, and through the curvature design and spatial layout of the first freeform surface, it converges to form an incident image in the fluorescence imaging area of ​​the CMOS image sensor.

[0009] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided. The first beam splitter is a first freeform surface, and the second beam splitter is a second freeform surface. When a white light beam enters the beam splitter, its path is transmitted through the second freeform surface, and after being reflected by the second freeform surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam splitter, its path is directly reflected at the first freeform surface, and after being converging and incident on the fluorescence imaging area of ​​the CMOS image sensor through the curvature design and spatial layout of the first freeform surface.

[0010] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image adjustment is provided. A dielectric film layer is disposed on the surface of a first freeform surface to selectively reflect and transmit the incident light beam according to the wavelength. The light beam in the white light band is transmitted through the reflective surface for imaging, while the fluorescence imaging beam in the near-infrared band is reflected by the first freeform surface. The first freeform surface itself has asphericity and, as a reflective optical element with optical power, has a converging effect on the near-infrared light beam reflected on it, shortening the physical optical path of the beam.

[0011] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided, wherein the white light imaging region and the fluorescence imaging region of the CMOS image sensor adopt a homogeneous pixel driving circuit and a global synchronous exposure control architecture. The two physically separated regions share the same set of pixel reset signals, exposure trigger signals and readout timing control signals. They synchronously complete the accumulation of photoelectric charge and parallel readout of the entire domain within the same exposure cycle. The exposure start time difference between the full pixel array in the white light imaging region and the full pixel array in the fluorescence imaging region is less than 1 μs, and the exposure time is completely consistent.

[0012] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided. The beam splitter is composed of a first prism and a second prism bonded together. Both the first and second prisms are made of H-K9L optical glass. The first surface of the first prism is a plane with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a first freeform surface with a radius of curvature of -13.922 mm and a thickness of 1.8 mm. The third surface of the first prism is a plane. The bonded surface of the second prism facing the first prism is a matching freeform surface with a radius of curvature of -13.922 mm. The second surface of the second prism is a reflective surface with a thickness of 1.8 mm. The third surface of the second prism is a plane. The first free surface is an extended polynomial surface, and its surface shape satisfies the formula:

[0013] in, For surface coordinates, For surface curvature, For the radial coordinates of the surface, For the conic coefficient of the curved surface, For polynomial quantity, For polynomial coefficients, For about The power series polynomial.

[0014] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration is provided. The beam splitter is formed by bonding a first prism and a second prism. The first surface of the first prism is a plane with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a first free-form surface with a radius of curvature of -14.455 mm and a thickness of 1.8 mm. The third surface of the first prism is a plane. The cemented surface of the second prism facing the first prism is a matching freeform surface with a radius of curvature of -14.455mm and a thickness of 3.8mm, made of H-K9L material; the second surface of the second prism is a second freeform surface with a radius of curvature of 16.055mm and a thickness of 1.8mm; the third surface of the second prism is a plane. Both the first and second freeform surfaces are extended polynomial surfaces.

[0015] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image adjustment is provided. The front end of the objective lens is also provided with a 30° rotating prism. The 30° rotating prism is an integrally formed isosceles right-angle prism group, and the incident end face and the exit end face maintain a 30° angle. The 30° rotating prism is made of the same H-K9L optical glass material as the freeform surface beam splitter prism.

[0016] According to the present invention, a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image adjustment is provided. The system is equipped with a corresponding software algorithm to achieve electronic visual axis correction, including the following steps: S1: Fix the assembled optical system to the optical bench, and set the calibration target at the required convergence distance position in front of the objective lens. The calibration target adopts a combination of cross and concentric circles. S2: Read the white light images acquired by the left and right CMOS image sensors, and overlay the two target images on the same display interface. Adjust the position of the two output image areas on the corresponding photosensitive areas of the CMOS image sensors through software so that the centers of the two target images completely overlap. S3: Based on the center overlap, continue to fine-tune the position of the two output image areas, and combine the inherent distortion characteristics of the optical system to adjust the misalignment of the edges of the two target images to be uniform and consistent, thus completing the visual axis correction of the white light 3D part. S4: Adjust the positions of the fluorescence imaging areas of the left and right channels respectively, so that the fluorescence image of each channel completely overlaps with the white light image of the corresponding channel, and complete the line-of-sight alignment of the single-channel dual-band. S5: Finally, output the white light and fluorescence images of all paths simultaneously, confirm that all images are completely superimposed, thus completing the electronic visual axis correction of the entire three-dimensional fluorescence endoscope.

[0017] Therefore, compared with the prior art, the three-dimensional fluorescence electronic endoscope optical system with partitioned imaging and image adjustment proposed in this invention has the following beneficial effects: 1. This invention, through the collaborative design of a beam splitter prism and a partitioned CMOS, enables white light and near-infrared fluorescence images to be simultaneously and focally imaged on different regions of the same CMOS through optical design, supporting high frame rate continuous acquisition, eliminating the delay and flicker risk of time-division acquisition, and avoiding the flicker and delay problems of time-division acquisition.

[0018] 2. This invention avoids the system volume expansion caused by using multiple sensors through a single CMOS dual-region imaging scheme, which is more suitable for the miniaturization requirements of endoscopes; without significantly increasing the system volume, it achieves dynamic alignment of the left and right optical paths, adapts to different working distances, and improves 3D observation comfort and imaging stability.

[0019] 3. The optical axis correction module, combined with electronic calibration software, can dynamically adapt to different working distances without reducing the 3D display area, significantly improving the comfort of 3D viewing and the accuracy of surgical operations. It corrects image shifts caused by assembly errors or optical distortions, ensuring accurate registration of dual-band images in 3D fusion.

[0020] 4. The freeform surface design of the beam splitter can be achieved through parametric optimization. The double freeform surface scheme further reduces the processing difficulty and is conducive to mass production.

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a first embodiment of the optical system of a three-dimensional fluorescent electronic endoscope that employs zoned imaging and image adjustment according to the present invention.

[0023] Figure 2 This is a schematic diagram of the left-path optical system of a three-dimensional fluorescent electronic endoscope optical system with partitioned imaging and image adjustment according to the present invention.

[0024] Figure 3 This is a schematic diagram of CMOS partition acquisition and display of a three-dimensional fluorescent electronic endoscope optical system with partitioned imaging and image adjustment according to an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the optical path simulation of a three-dimensional fluorescent electronic endoscope optical system using partitioned imaging and image adjustment according to the present invention.

[0026] Figure 5 This is a schematic diagram of the left-path optical system in Embodiment 2 of the present invention, which employs partitioned imaging and image adjustment in a three-dimensional fluorescent electronic endoscope optical system.

[0027] Figure 6 This is a schematic diagram of the optical path simulation of Embodiment 2 of the optical system of a three-dimensional fluorescent electronic endoscope that employs partitioned imaging and image adjustment according to the present invention.

[0028] Figure 7 This is a schematic diagram of the lens imaging area in Embodiment 1 of the optical system of a three-dimensional fluorescent electronic endoscope that employs zoned imaging and image adjustment according to the present invention.

[0029] Figure 8 This is a schematic diagram of the white light image after correction is completed in Embodiment 1 of the optical system of a three-dimensional fluorescent electronic endoscope using partitioned imaging and image adjustment according to the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] Example 1 of a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration See Figures 1 to 4 This embodiment provides a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration, comprising: The main tube contains a left-path optical system and a right-path optical system symmetrically distributed along the central axis. Each of the two systems independently captures images of the same object, forming the basic optical structure for three-dimensional observation. An illumination beam guide, located inside the main lens barrel, is used to provide white light and near-infrared illumination; The left-path optical system and the right-path optical system each include: Objective lens, used to receive images of objects and perform preliminary imaging; A beam splitter, positioned behind the objective lens, splits the incident light beam after it has been imaged by the objective lens into a white light band and a near-infrared band. The CMOS image sensor has a photosensitive area divided into a white light imaging area and a fluorescence imaging area, which respectively receive the white light band image and the near-infrared band image after spectral splitting. The image processor is used to process the white light and fluorescence image data output by the CMOS image sensor and to synthesize the images from the left and right optical systems into three-dimensional image data. The optical axis correction module is used to dynamically change the horizontal or vertical offset of the left and right imaging optical axes by adjusting the attitude or position of the beam splitter or optical elements, in order to compensate for the visual axis deviation caused by the change of the endoscope working distance. This allows it to adapt to different working distances and ensure the comfort and clarity of three-dimensional observation without reducing the three-dimensional display area.

[0033] The beam splitter has a first beam splitter surface and a second beam splitter surface inside. When a white light beam enters the beam splitter, its path is through the second beam splitter surface, and after being reflected by the second beam splitter surface, it is focused onto the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam splitter, its path is through direct reflection at the first beam splitter surface, and it is focused onto the fluorescence imaging area of ​​the CMOS image sensor.

[0034] The first beam-splitting surface is a first freeform surface, and the second beam-splitting surface is a reflective surface. When a white light beam enters the beam-splitting prism, its path is through the 45° reflective surface, and after being reflected by the reflective surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam-splitting prism, its path is through direct reflection at the first freeform surface, and through the curvature design and spatial layout of the first freeform surface, it converges to form an incident image on the fluorescence imaging area of ​​the CMOS image sensor.

[0035] The specific optical parameters of the freeform surface are designed to make the optical path of the near-infrared beam equal to or within a preset tolerance range of the optical path traversed by the white light beam after reflection, so as to ensure that the white light image and the fluorescence image can be clearly focused and imaged on the CMOS image sensor at the same time without the need for physical compensation mirrors or electronic focal plane calibration.

[0036] The surface of the first freeform surface is provided with a dielectric film layer to selectively reflect and transmit the incident light beam according to the wavelength. The white light beam is transmitted through the reflective surface and reflected to form an image, while the near-infrared fluorescent imaging beam is reflected by the first freeform surface. The first freeform surface itself has asphericity and, as a reflective optical element with optical power, it has a converging effect on the near-infrared beam reflected on it, shortening the physical optical path of the beam. The radius of curvature and aspheric extended polynomial parameters of the first freeform surface (e.g., the parameter range shown in Table 2 or Table 4) are optimized so that the optical path experienced by the near-infrared beam reflected by it is equal to the optical path experienced by the white light beam reflected by the 45° reflective surface after transmission, or they are matched within a preset imaging focal depth tolerance range. This allows the white light image and the fluorescent image to be simultaneously and clearly focused on different imaging areas of a single CMOS image sensor, eliminating the need for additional compensation lens groups or complex electronic focusing algorithms.

[0037] The white light imaging region and the fluorescence imaging region of the CMOS image sensor adopt the same pixel driving circuit and global synchronous exposure control architecture: Two physically separated regions share the same set of pixel reset signals, exposure trigger signals, and readout timing control signals. Within the same exposure cycle, they synchronously complete the accumulation of photoelectric charge and full-domain parallel readout. The exposure start time difference between the full pixel array in the white light imaging region and the full pixel array in the fluorescence imaging region is less than 1μs, and the exposure duration is completely consistent. Without reducing the effective pixel count of any sub-region or splitting the effective exposure duration of a single frame, the full frame rate parallel acquisition of white light images and fluorescence images is achieved, ensuring that the output frame rate of both types of images is not less than 60Hz. Furthermore, the timestamps of a single frame of white light image and the corresponding fluorescence image completely overlap, and there is no inter-frame time difference in the time-division acquisition mode.

[0038] The optical system does not require a time-division strobe illumination module. Under continuous and constant white light and near-infrared light illumination, it can synchronously output real-time white light images and fluorescence-enhanced images without time difference, completely eliminating the time misalignment problem between two frames of images in the time-division acquisition mode, and avoiding 3D fusion artifacts and observation delays caused by image asynchrony.

[0039] The beam splitter is composed of a first prism and a second prism bonded together. Both the first and second prisms are made of H-K9L optical glass. The first surface of the first prism is a flat surface with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a freeform surface with a radius of curvature of -13.922 mm and a thickness of 1.8 mm. The third surface of the first prism is a flat surface. The bonded surface of the second prism facing the first prism is a matching freeform surface with a radius of curvature of -13.922 mm. The second surface of the second prism is a reflecting surface with a thickness of 1.8 mm. The third surface of the second prism is a flat surface. The first free surface is an extended polynomial surface, and its surface shape satisfies the formula:

[0040] in, For surface coordinates, For surface curvature, For the radial coordinates of the surface, For the conic coefficient of the curved surface, For polynomial quantity, For polynomial coefficients, For about The power series polynomial; by optimizing the curvature parameters of the freeform surface and the coefficients of each polynomial, optical path matching in the near-infrared band and the white light band can be achieved, ensuring that the two bands are simultaneously and clearly imaged on the same CMOS image sensor.

[0041] In practical applications, within the main tube of an endoscope, two identical optical systems are distributed along the central axis on either side, such as... Figure 1 As shown, the two optical systems, referred to as the left optical system and the right optical system (hereinafter referred to as the right path of the left path), constitute the most basic optical system of the 3D optical system. Each of these two optical systems independently captures images of the same object and transmits the images to the backend software for processing into 3D image data, which is then sent to a 3D display for display. The human eye can observe the 3D images generated by the 3D electronic endoscope by wearing 3D glasses.

[0042] The main endoscope tube is typically made of medical-grade stainless steel, and its outer surface is polished to meet medical endoscope industry standards. Besides protecting the outer shell, it also serves to secure and reinforce internal components. The beam guide is used for illumination, ensuring uniform illumination. In white light, fluorescent, or dual-fluorescent systems, it must provide uniform illumination and minimal attenuation not only in the white light band but also in the fluorescent contrast agent's excitation band. The left and right optical systems are analogous to the two eyes of a human eye. In addition to the same optical requirements as conventional 2D endoscopes, including but not limited to resolution, distortion, chromatic aberration, and uniformity, there are also requirements specific to 3D endoscopes, including but not limited to the consistency of the two images and the time difference between the left and right channels.

[0043] It is important to note that Figure 1 This is for illustrative purposes only, mainly to show the general structure and composition of a 3D electronic endoscope. It does not limit the structure and shape to be consistent with the illustrative diagram.

[0044] Since the left and right optical systems are identical, the left optical system will be used as an example to illustrate how a fluorescence imaging optical system based on CMOS partition acquisition and display can be realized in a 3D electronic endoscope.

[0045] like Figure 2 As shown, the image of the object is first formed by the objective lens of the optical system. Then, the light beam is split by a beam splitter into a white light band and a near-infrared band. The white light band is the ordinary white light image, while the near-infrared band is the fluorescent image. The white light beam, after passing through the beam splitter, is transmitted to a 45° reflecting surface and reflected again to form an image as shown in the diagram. Figure 3 A white light image is formed within the white light imaging area of ​​the CMOS 23 shown, and after being processed into digital information by electronic devices, it is transmitted to the image processor. Meanwhile, the near-infrared beam, after passing through the beam-splitting prism, is reflected and converged at the freeform surface that performs the beam splitting function, as shown in the image. Figure 3 Image is formed on the fluorescence imaging region of the CMOS23 shown.

[0046] like Figure 2 As shown, since white light imaging has a shorter optical path than near-infrared imaging, without processing, it is impossible for the two images to be clearly imaged on different regions of the same CMOS. The role of freeform surfaces is not only to achieve spectral separation between the near-infrared and white light bands through the deposition of dielectric films. Furthermore, because it is a freeform surface, the near-infrared band will experience a converging effect on this surface, thus shortening the optical path. By selecting an appropriate surface radius and polynomial coefficients, the total optical path of the near-infrared and white light bands can be made consistent, ultimately resulting in both being imaged onto the same CMOS.

[0047] Because the system uses dual CMOS sensors, each CMOS partition receives white light and fluorescence images separately. These are isolated from each other by a beam splitter, preventing interference and eliminating the need for time-division multiplexing. Therefore, fluorescence and white light images can be continuously and clearly imaged. The fluorescence image is unaffected by the intensity of white light or the color of objects, resulting in stable images and accurate fluorescence image recognition and labeling. It avoids image delays caused by time-division multiplexing or brightness instability due to energy ramp-up from light source frequency switching.

[0048] This embodiment provides optical parameters for a beam splitter with free curves, as shown in Table 1: Table 1: Prism optical parameters of Example 1

[0049] The second surface of the first prism and the first surface 1 of the second prism are glued together, and the two prisms are glued together to form a beam splitter prism.

[0050] The freeform surface is an extended polynomial surface, and the surface shape formula is:

[0051] In this embodiment, the freeform surface uses 20 extended polynomials, and the complete data is shown in Table 2: Table 2: Freeform Surface Extended Polynomial Parameters of Example 1

[0052] In this embodiment, a 30° image-rotating prism is also provided at the front end of the objective lens. The 30° image-rotating prism is an integrally formed isosceles right-angle prism group, with the incident end face and the exit end face maintaining a 30° angle. The 30° image-rotating prism uses the same H-K9L optical glass material as the freeform surface beam splitter prism.

[0053] As can be seen, in order to deflect the optical axis, an image-rotating prism is added in front of the objective lens of an endoscope, such as the commonly used 30mm prism. The image-spinning prism will not affect subsequent imaging, so it will not be described in detail in this embodiment.

[0054] In this embodiment, the system is configured with a corresponding software algorithm to implement electronic visual axis correction, including the following steps: S1: Fix the assembled optical system to the optical bench, and set the calibration target at the required convergence distance position in front of the objective lens. The calibration target adopts a combination of cross and concentric circles. S2: Read the white light images acquired by the left and right CMOS image sensors, and overlay the two target images on the same display interface. Adjust the position of the two output image areas on the corresponding photosensitive areas of the CMOS image sensors through software so that the centers of the two target images completely overlap. S3: Based on the center overlap, continue to fine-tune the position of the two output image areas, and combine the inherent distortion characteristics of the optical system to adjust the misalignment of the edges of the two target images to be uniform and consistent, thus completing the visual axis correction of the white light 3D part. S4: Adjust the positions of the fluorescence imaging areas of the left and right channels respectively, so that the fluorescence image of each channel completely overlaps with the white light image of the corresponding channel, and complete the line-of-sight alignment of the single-channel dual-band. S5: Finally, output the white light and fluorescence images of all paths simultaneously, confirm that all images are completely superimposed, thus completing the electronic visual axis correction of the entire three-dimensional fluorescence endoscope.

[0055] like Figure 3As shown, the white light and near-infrared imaging areas do not fully utilize the entire CMOS sensor; the surrounding areas are still usable. However, for a 3D imaging system to generate 3D vision, the left and right images must overlap at a certain working distance, generally referred to as the convergence distance. The convergence distance affects the depth perception of 3D vision and human eye comfort. It is usually a fixed value set according to product requirements. However, in this system, the optical axes of the left and right optical paths are parallel, requiring a certain offset in the CMOS sensor's mounting position. Furthermore, due to unavoidable optical assembly errors, when the convergence distance needs to be within the required range, in addition to relying on mechanical positioning to achieve CMOS offset, control software must be used to adjust the image reading of different areas of the CMOS sensor according to the specific needs of each product to realize a standardized 3D vision. The specific steps are as follows: like Figure 7 As shown, the imaging range of the lens is a circle, while the actual output image is a rectangle. Therefore, the lens imaging range is designed to be larger so that the actual output image can move to a certain extent within the circular range of the lens imaging to correct optical assembly errors and ensure the convergence distance.

[0056] During the calibration phase, the optical system that has been assembled and calibrated in the previous steps is placed on an optical bench for curing. A calibration target is installed at the radial distance directly in front of the lens. There are no specific requirements for the shape of the calibration target. Generally, for convenience, a cross shape with several concentric circles is used for calibration.

[0057] At this point, the images of the two CMOS white light sources can be read and displayed on the same monitor. However, the centers of the two target images are usually not aligned. The software needs to be used to adjust the position of the output image area on the CMOS so that the two target images are completely aligned at the center.

[0058] Because the optical assembly errors of the two optical systems will not be the same, and the optical system of an endoscope generally has relatively large distortions, in order to obtain the best possible 3D visual effect, after the center is aligned, the image will be misaligned at the edges. In this case, it is necessary to adjust the position of the output image area again through software to make the size of the misalignment at the two edges as consistent as possible. Figure 8 As shown, the visual axis correction for the electronic 3D white light portion is completed.

[0059] Next, open the white light and fluorescence images on the left side. Simply adjust the fluorescence image to overlap with the white light image. Then, similarly adjust the white light and fluorescence images on the right side, ensuring the fluorescence image overlaps with the white light image. (Complete electronic 3D white light fluorescence visual axis correction) Finally, turn on the left and right white light and fluorescence images and confirm that the images have overlapped and have not changed, thus completing the electronic 3D with visual axis correction.

[0060] Example 2 of a three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image calibration like Figure 5 , Figure 6 As shown in Example 1, the beam splitter adopts a double freeform surface structure, with parameters shown in Tables 3 and 4. In this structure, the optical power of the two freeform surfaces complement each other, making the surface curvature smoother, reducing the manufacturing difficulty, and improving the imaging quality and dual-band registration accuracy.

[0061] The first beam-splitting surface is a first freeform surface, and the second beam-splitting surface is a second freeform surface. When a white light beam enters the beam-splitting prism, its path is transmitted through the second freeform surface, and after being reflected by the second freeform surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam-splitting prism, its path is directly reflected at the first freeform surface, and after being reflected by the curvature design and spatial layout of the first freeform surface, it converges to form an image on the fluorescence imaging area of ​​the CMOS image sensor.

[0062] The beam splitter is made of a first prism and a second prism bonded together. The first surface of the first prism is a plane with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a freeform surface with a radius of curvature of -14.455 mm and a thickness of 1.8 mm. The third surface of the first prism is a plane. The cemented surface of the second prism facing the first prism is a matched freeform surface with a radius of curvature of -14.455mm and a thickness of 3.8mm, made of H-K9L material; the second surface of the second prism is a second freeform surface with a radius of curvature of 16.055mm and a thickness of 1.8mm; the third surface of the second prism is a plane; wherein, the first freeform surface and the second freeform surface are both extended polynomial surfaces.

[0063] It is evident that, under specific circumstances, the 45° reflective surface can also be designed as a freeform curve. This can distribute the radius of curvature of the freeform surface 24, making its radius of curvature larger and tending towards a more flat surface, reducing higher-order aberrations. It can also make the magnification of near-infrared images and white light images closer, reducing the pressure on software matching. The two freeform surfaces each undertake the optical power correction function. By distributing the optical power, the radius of curvature of both freeform surfaces is larger than that of the single freeform surface correction scheme, resulting in a smoother surface. In Example 2, all processes are the same as in Example 1, except that the 45° reflecting surface in Example 1 is replaced by a freeform surface. However, because the optical power of the freeform surface is shared by the freeform surface, the radii of curvature of the two freeform surfaces become larger, and the surfaces become flatter. The optical parameters of the beam splitter prism in Example 2 are shown in Table 3: Table 3: Prism optical parameters of Example 2

[0064] The complete polynomial parameters of the two free-form surfaces in the same embodiment 2 are shown in Table 4: Table 4: Extended polynomial parameters of the freeform surface in Example 2

[0065] The partitioned imaging and optical calibration principle of this invention is also applicable to other optical systems that require synchronous acquisition and real-time three-dimensional display of multi-band images, such as industrial endoscopes and microscopic imaging systems.

[0066] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A three-dimensional fluorescence electronic endoscope optical system employing zoned imaging and image adjustment, characterized in that, include: The main tube contains a left-path optical system and a right-path optical system symmetrically distributed along the central axis. Each of the two systems independently captures images of the same object, forming the basic optical structure for three-dimensional observation. An illumination beam guide, located inside the main lens barrel, is used to provide white light and near-infrared illumination; The left-path optical system and the right-path optical system each include: Objective lens, used to receive images of objects and perform preliminary imaging; A beam splitter, positioned behind the objective lens, splits the incident light beam after it has been imaged by the objective lens into a white light band and a near-infrared band. The CMOS image sensor has a photosensitive area divided into a white light imaging area and a fluorescence imaging area, which respectively receive the white light band image and the near-infrared band image after spectral splitting. The image processor is used to process the white light and fluorescence image data output by the CMOS image sensor and to synthesize the images from the left and right optical systems into three-dimensional image data. The optical axis correction module is used to dynamically change the horizontal or vertical offset of the left and right imaging optical axes by adjusting the attitude or position of the beam splitter or optical elements, in order to compensate for the visual axis deviation caused by changes in the working distance of the endoscope.

2. The system according to claim 1, characterized in that: The beam splitter has a first beam splitter surface and a second beam splitter surface inside. When a white light beam enters the beam splitter, its path is through the second beam splitter surface, and after being reflected by the second beam splitter surface, it is focused onto the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam splitter, its path is through direct reflection at the first beam splitter surface, and it is focused onto the fluorescence imaging area of ​​the CMOS image sensor.

3. The system according to claim 2, characterized in that: The first beam-splitting surface is a first freeform surface, and the second beam-splitting surface is a reflective surface. When a white light beam enters the beam-splitting prism, its path is through the 45° reflective surface, and after being reflected by the reflective surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam-splitting prism, its path is through direct reflection at the first freeform surface, and through the curvature design and spatial layout of the first freeform surface, it converges to form an incident image on the fluorescence imaging area of ​​the CMOS image sensor.

4. The system according to claim 2, characterized in that: The first beam-splitting surface is a first freeform surface, and the second beam-splitting surface is a second freeform surface. When a white light beam enters the beam-splitting prism, its path is transmitted through the second freeform surface, and after being reflected by the second freeform surface, it converges to form an image on the white light imaging area of ​​the CMOS image sensor. When a near-infrared beam enters the beam-splitting prism, its path is directly reflected at the first freeform surface, and after being reflected by the curvature design and spatial layout of the first freeform surface, it converges to form an image on the fluorescence imaging area of ​​the CMOS image sensor.

5. The system according to claim 3 or 4, characterized in that: A dielectric film layer is provided on the surface of the first freeform surface to selectively reflect and transmit the incident light beam according to the wavelength. The light beam in the white light band is transmitted through the reflective surface and reflected to form an image, while the fluorescence imaging beam in the near-infrared band is reflected by the first freeform surface. The first freeform surface itself has asphericity and, as a reflective optical element with optical power, it has a converging effect on the near-infrared light beam reflected on it, shortening the physical optical path of the beam.

6. The system according to claim 1, characterized in that, The white light imaging region and the fluorescence imaging region of the CMOS image sensor adopt the same pixel driving circuit and global synchronous exposure control architecture: The two physically separated regions share the same set of pixel reset signals, exposure trigger signals and readout timing control signals. They synchronously complete the accumulation of photoelectric charge and parallel readout of the entire domain within the same exposure cycle. The exposure start time difference between the full pixel array in the white light imaging region and the full pixel array in the fluorescence imaging region is less than 1 μs, and the exposure time is completely consistent.

7. The system according to claim 3, characterized in that: The beam splitter is composed of a first prism and a second prism bonded together. Both the first and second prisms are made of H-K9L optical glass. The first surface of the first prism is a flat surface with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a freeform surface with a radius of curvature of -13.922 mm and a thickness of 1.8 mm. The third surface of the first prism is a flat surface. The bonded surface of the second prism facing the first prism is a matching freeform surface with a radius of curvature of -13.922 mm. The second surface of the second prism is a reflecting surface with a thickness of 1.8 mm. The third surface of the second prism is a flat surface. The first free surface is an extended polynomial surface, and its surface shape satisfies the formula: in, For surface coordinates, For surface curvature, For the radial coordinates of the surface, For the conic coefficient of the curved surface, For polynomial quantity, For polynomial coefficients, For about The power series polynomial.

8. The system according to claim 4, characterized in that: The beam splitter is made of a first prism and a second prism bonded together. The first surface of the first prism is a plane with a thickness of 1.8 mm and is made of H-K9L optical glass. The second surface of the first prism is a freeform surface with a radius of curvature of -14.455 mm and a thickness of 1.8 mm. The third surface of the first prism is a plane. The cemented surface of the second prism facing the first prism is a matching freeform surface with a radius of curvature of -14.455mm and a thickness of 3.8mm, made of H-K9L material; the second surface of the second prism is a second freeform surface with a radius of curvature of 16.055mm and a thickness of 1.8mm; the third surface of the second prism is a plane. Both the first and second freeform surfaces are extended polynomial surfaces.

9. The system according to any one of claims 1 to 8, characterized in that: The objective lens is also equipped with a 30° image-rotating prism at the front end. The 30° image-rotating prism is an integrally formed isosceles right-angle prism group, with the incident end face and the exit end face maintaining a 30° angle. The 30° image-rotating prism uses the same H-K9L optical glass material as the freeform surface beam splitter prism.

10. The system according to any one of claims 1 to 8, characterized in that, The system is equipped with corresponding software algorithms to implement electronic axis correction, including the following steps: S1: Fix the assembled optical system to the optical bench, and set the calibration target at the required convergence distance position in front of the objective lens. The calibration target adopts a combination of cross and concentric circles. S2: Read the white light images acquired by the left and right CMOS image sensors, and overlay the two target images on the same display interface. Adjust the position of the two output image areas on the corresponding photosensitive areas of the CMOS image sensors through software so that the centers of the two target images completely overlap. S3: Based on the center overlap, continue to fine-tune the position of the two output image areas, and combine the inherent distortion characteristics of the optical system to adjust the misalignment of the edges of the two target images to be uniform and consistent, thus completing the visual axis correction of the white light 3D part. S4: Adjust the positions of the fluorescence imaging areas of the left and right channels respectively, so that the fluorescence image of each channel completely overlaps with the white light image of the corresponding channel, and complete the line-of-sight alignment of the single-channel dual-band. S5: Finally, output the white light and fluorescence images of all paths simultaneously, confirm that all images are completely superimposed, thus completing the electronic visual axis correction of the entire three-dimensional fluorescence endoscope.

Images