High-flexibility integrated three-dimensional endoscope based on multi-view miniaturized imaging equipment

Abstract

The invention discloses a high-flexibility integrated three-dimensional endoscope based on multi-view miniaturized imaging equipment, belongs to the field of optical system design and manufacturing, and solves the problem that a traditional light field endoscope cannot realize digestive tract detection due to a rigid large-size probe during three-dimensional imaging. The miniaturized imaging equipment is processed and integrated through precise elements, and comprises a lens, four LED light sources, a CMOS, a metal shell and a data cable. According to the invention, the flexibility and the application range of the system are greatly improved through miniaturized design and manufacturing of the rigid probe of the light field endoscope. The working adaptability of the light field endoscope in the digestive tract is achieved, the size and the mass of the system are greatly reduced, the system is expected to be widely applied to remote and hard areas such as villages, and the village medical quality is improved.

Description

Technical Field

[0001] The invention belongs to the field of optical system design and manufacturing, and relates to a highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device. Background Art

[0002] The development of biomedical imaging technologies and equipment has significantly advanced medical science, from diagnosis to treatment. Optical endoscopic imaging, in particular, offers minimally invasive and high-resolution imaging, enabling doctors to visualize hard-to-reach areas within the patient's body. Optical endoscopy has become a crucial tool for disease diagnosis and treatment, effectively reducing infection risk, recovery time, and treatment costs. However, image sensors can only project information within the field of view onto a two-dimensional imaging surface, resulting in a loss of some optical field information, such as axial depth. Current clinical cases, such as those involving dense tumor structures and vascular capsules, require precise three-dimensional information to aid treatment and diagnosis, which often relies on the physician's experience. Therefore, 3D imaging can effectively reduce errors during minimally invasive surgery and increase diagnostic accuracy.

[0003] Light field imaging technology can record the spatial and angular information of a sample, thereby obtaining three-dimensional or tomographic information of the scene. Depth quantification and focal refocusing can be obtained in a single-frame exposure. This scan-free technology is, in principle, very conducive to manufacturing a more compact and cost-effective paradigm. However, existing light field endoscopes often use microlens arrays or gradient refractive index lens arrays combined with a long-distance relay system. This results in the probe becoming a large and rigid structure, making the device only suitable for shallow or invasive observations and unsuitable for the diagnosis and treatment of gastrointestinal diseases. Summary of the Invention

[0004] In view of this, the present invention provides a highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device. The miniaturized imaging device manufactured by precision machining is used to miniaturize the rigid part of the light field endoscope probe, greatly improving its flexibility, thereby meeting the application requirements in the digestive tract environment.

[0005] It should be noted that the highly flexible integrated three-dimensional endoscope described in the present invention is manufactured by processing a miniaturized imaging device with a sufficiently small size and integrating it into the form of a four-eye array, thereby greatly reducing the volume of the rigid part of the system and increasing the overall flexibility, so that it can realize single-frame three-dimensional observation and reconstruction of the inside of the digestive tract.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device. The system is a miniaturized multi-eye flexible light field imaging system implemented by integrating four miniaturized imaging devices.

[0008] The highly flexible integrated three-dimensional endoscope consists of four calibrated miniaturized imaging devices integrated into a probe housing and wrapped with a flexible coating material;

[0009] The components required for the miniaturized imaging device include: an LED light source, an integrated sleeve, a lens, a CMOS, and a data cable.

[0010] Optionally, the LED light source, lens, and CMOS are all integrated in a metal integrated sleeve with a length of 3mm to 7mm and a diameter of 1.5mm to 2.5mm, preferably a length of 5mm and a diameter of 1.8mm; and a data cable is provided at the tail of the metal integrated sleeve.

[0011] Optionally, the outside of the lens is surrounded by a plurality of (preferably four) LED light sources, and the LED light sources are 0201 model or white light LED lamp beads with a size of less than 0.65mm×0.35mm.

[0012] Furthermore, the LED light source is a 0201 model white light LED lamp bead, and four LED light sources are surrounded outside the lens to achieve illumination of the sample.

[0013] Optionally, the diameter of the lens is 0.8 mm to 1.2 mm, the focal length is 0.3 mm to 0.5 mm, and the F number is 4 to 6.

[0014] Furthermore, the diameter of the lens is 1 mm, the focal length is 0.418 mm, and the F number is 5.

[0015] Optionally, the pixel size of the CMOS is 1.5×1.5 μm to 2.0×2.0 μm, and the pixel resolution is 300×300 to 600×600.

[0016] Furthermore, the pixel size of the CMOS is 1.75×1.75 μm, and the pixel resolution is 400×400.

[0017] Furthermore, the power supply and information transmission of the miniaturized imaging device are achieved through the data cable. The data cable may include an integrated structure of a power line and a signal line, and is preferably a multi-core shielded cable or a flexible cable.

[0018] Optionally, the probe housing is a cylinder made of metal material (preferably aluminum) with a diameter of 4mm to 6mm and a height of 4mm to 6mm. It is provided with multiple (preferably four) through holes with a diameter of 1.5mm to 2.5mm inside, which are arranged symmetrically in a square, and the spacing between adjacent through holes is 2-5mm.

[0019] Furthermore, the probe housing is a cylinder made of aluminum with a diameter of 5 mm and a height of 4.5 mm. There are four through holes with a diameter of 2.2 mm inside, which are arranged symmetrically in a square shape, and the distance between adjacent through holes is 2 mm.

[0020] Optionally, the flexible covering material is a flexible polymer material, preferably polyamide or polyurethane, which is used to cover and protect the integrated flexible light field system. After covering, the diameter of the entire system is 6.5 mm to 8.0 mm, preferably 7.5 mm.

[0021] Optionally, the lens in the miniaturized imaging device exhibits severe barrel distortion, which affects imaging and 3D reconstruction. Therefore, each sub-device needs to be calibrated before use. This calibration method combines the checkerboard and Zhang Zhengyou calibration methods to obtain the system's intrinsic parameters and distortion coefficients, which are then incorporated into the distortion equation to correct image distortion.

[0022] Optionally, the system uses four sub-devices to shoot the target and correct the distortion. In order to reconstruct the three-dimensional information, it is necessary to use the Fourier light field deconvolution method. The quantitative depth information of the sample can be obtained through Fourier light field deconvolution.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention discloses a highly flexible, integrated 3D endoscope based on a multi-lens miniaturized imaging device. It boasts high flexibility and lightweight, enabling quantitative 3D imaging with a single-frame exposure. Unlike traditional medical endoscopes and light-field endoscopes, this invention, with its highly flexible, quantitative 3D imaging capabilities, enables tomographic observation and diagnosis of lesions within the digestive tract.

[0025] In addition, the system's light weight also makes it suitable for mass production and distribution to remote villages and other places with poor medical conditions, thereby improving local medical standards. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0027] Figure 1 This is a schematic structural diagram of a miniaturized imaging device based on a multi-eye miniaturized imaging device of the present invention, which is a highly flexible integrated three-dimensional endoscope.

[0028] Figure 2 This is a schematic structural diagram of the highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to the present invention.

[0029] In the figure: 1 LED light source, 2 metal integrated sleeve, 3 lens, 4 CMOS, 5 data cable, 6 probe housing, 7 flexible covering material. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of an actual implementation are described in this specification. However, it should be understood that in the process of developing any such actual implementation, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is a routine task for those skilled in the art who benefit from the disclosure of the present invention.

[0031] It is also necessary to explain here that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show the device structure and / or processing steps closely related to the solution according to the present invention, while other details that are not closely related to the present invention are omitted.

[0032] The present invention discloses a highly flexible integrated three-dimensional endoscope based on a multi-lens miniaturized imaging device. The device belongs to the field of optical system design and manufacturing and solves the problem that traditional light-field endoscopes cannot perform digestive tract inspections during three-dimensional imaging due to the rigid, large-sized probe. The miniaturized imaging device, through precision component processing and integration, includes a lens, four LED light sources, a CMOS, a metal housing, and a data cable. The rigid portion of a single miniaturized imaging device has a diameter of 1.8 mm and a length of 5 mm, enabling high-quality imaging. Four calibrated miniaturized imaging devices are arranged and integrated in a square shape, and single-frame three-dimensional information tomography of the sample can be achieved through light field theory and deconvolution algorithms. The rigid probe portion of the system has a diameter and length of 5 mm and is coated with a flexible material, allowing it to adapt to the curved environment of the digestive tract and easily reach the lesion for imaging. The present invention greatly improves the flexibility and application range of the system through the miniaturized design and manufacture of the rigid probe of the light-field endoscope. This invention not only solves the working adaptability of light field endoscopes in the digestive tract, but also greatly reduces the size and weight of the system. It is expected to be widely used in remote and difficult areas such as rural areas to improve the quality of rural medical care.

[0033] Specifically, as attached Figure 1 As shown, the embodiment of the present invention provides a miniaturized imaging device. Figure 2 As shown, an embodiment of the present invention provides a highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device.

[0034] A highly flexible integrated three-dimensional endoscope based on multi-eye miniaturized imaging devices. The system is a miniaturized multi-eye flexible light field imaging system integrated by four miniaturized imaging devices.

[0035] The components required for the miniaturized imaging device are: LED light source 1, metal integrated sleeve 2, lens 3, CMOS 4, and data cable 5;

[0036] The highly flexible integrated three-dimensional endoscope is composed of four calibrated miniaturized imaging devices integrated into a probe housing 6 and wrapped with a flexible covering material 7 .

[0037] More specifically, the LED light source 1 is a 0201 model white light LED lamp bead; four LED light sources 1 are surrounded outside the lens 3 to achieve illumination of the sample.

[0038] More specifically, the diameter of the lens 3 is 1 mm, the focal length is 0.418 mm, and the F number is 5.

[0039] More specifically, the pixel size of the CMOS 4 is 1.75×1.75 μm, and the pixel resolution is 400×400.

[0040] More specifically, the LED light source 1 , the lens 3 , and the CMOS 4 are all integrated into a metal integrated sleeve 2 with a length of 5 mm and a diameter of 1.8 mm.

[0041] More specifically, the power supply and information transmission of the miniaturized imaging device are achieved by the data cable 5 at the rear end thereof.

[0042] More specifically, the probe housing 6 is a cylinder made of aluminum with a diameter of 5 mm and a height of 4.5 mm. There are four through holes with a diameter of 2.2 mm inside, which are symmetrically arranged in a square shape, and the distance between adjacent through holes is 2 mm.

[0043] More specifically, the flexible coating material 7 is polyamide, which wraps the integrated flexible light field system for protection and increases the robustness during use. After coating, the diameter of the entire system is 7.5 mm.

[0044] More specifically, the lens 3 in the miniaturized imaging device suffers from severe barrel distortion, which affects imaging and 3D reconstruction. Therefore, each sub-device must be calibrated before use. This calibration method combines the checkerboard and Zhang Zhengyou calibration methods to obtain the system's intrinsic parameters and distortion coefficients, which are then incorporated into the distortion equation to correct image distortion.

[0045] More specifically, the calibration method includes the following steps:

[0046] Use a standard checkerboard pattern with a size of 9×6 grids and a single grid side length of 25mm (can be adjusted according to the application);

[0047] Place the checkerboard image at different plane positions and angles in sequence, and use the lens 3 of each sub-device in the miniaturized imaging device to capture images from multiple angles. It is recommended to capture no less than 20 images.

[0048] Based on Zhang Zhengyou's calibration method, the coordinates of the corner points are extracted using tools such as OpenCV, and the lens intrinsic parameters (focal length, principal point position) and distortion coefficients (radial distortion, tangential distortion) are solved using the least squares method.

[0049] Substitute the obtained internal parameters and distortion coefficients into the distortion correction equation to calculate and output the distortion-corrected image;

[0050] The above calibration process is performed independently on all sub-device lenses to eliminate the impact of barrel distortion on subsequent 3D reconstruction.

[0051] More specifically, the system uses four sub-devices to photograph the target and correct the distortion. In order to reconstruct three-dimensional information, the Fourier light field deconvolution method is needed. The quantitative depth information of the sample can be obtained through Fourier light field deconvolution.

[0052] More specifically, the three-dimensional information reconstruction process includes the following steps:

[0053] Arrange multiple sub-devices (preferably four) around the target area and simultaneously capture the same target sample from different perspectives;

[0054] The captured images are calibrated and distortion corrected to ensure geometric consistency between images;

[0055] Based on the intrinsic and extrinsic parameters obtained by calibration, the light field image matrix is ​​reconstructed, that is, a set of projection images of the light field at different angles is obtained;

[0056] Perform Fourier transform on the light field image matrix to obtain its frequency domain representation;

[0057] Deconvolution of frequency domain data using pre-built optical transfer functions (OTFs) combined with deep inference models or prior structural information.

[0058] Perform inverse Fourier transform on the deconvolution frequency domain result to restore the three-dimensional image with depth dimension;

[0059] Extract and quantify the depth information of the sample in space and generate visual three-dimensional structural data.

[0060] The Fourier light field deconvolution method has the functions of eliminating the influence of point spread function (PSF) and enhancing the depth resolution of imaging, and is suitable for obtaining three-dimensional information of complex microstructure samples.

[0061] Although the embodiments disclosed in the present invention are as described above, the contents thereof are merely embodiments adopted to facilitate understanding of the technical solutions of the present invention and are not intended to limit the present invention. Any person skilled in the art of the present invention may make any modifications and changes in the form and details of the implementation without departing from the core technical solutions disclosed in the present invention. However, the scope of protection defined by the present invention shall still be subject to the scope defined in the appended claims.

Claims

1. A highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device, characterized in that: It is a miniaturized multi-eye flexible light field imaging system realized by integrating four miniaturized imaging devices; The highly flexible integrated three-dimensional endoscope is composed of four calibrated miniaturized imaging devices integrated into a probe housing (6) and wrapped with a flexible covering material (7); The components required for the miniaturized imaging device include: an LED light source (1), a metal integrated sleeve (2), a lens (3), a CMOS (4), and a data cable (5).

2. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1, characterized in that: The LED light source (1), the lens (3), and the CMOS (4) are all integrated in a metal integrated sleeve (2) with a length of 3 mm to 7 mm and a diameter of 1.5 mm to 2.5 mm; and a data cable (5) is provided at the rear of the metal integrated sleeve (2).

3. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1 or 2, characterized in that: The outside of the lens (3) is surrounded by a plurality of (preferably four) LED light sources (1), and the LED light sources (1) are 0201 model or white light LED lamp beads with a size of less than 0.65mm×0.35mm.

4. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 3 is characterized in that: The diameter of the lens (3) is 0.8 mm to 1.2 mm, the focal length is 0.3 mm to 0.5 mm, and the F number is 4 to 6.

5. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1 or 2, characterized in that: The pixel size of the CMOS (4) is 1.5×1.5 μm to 2.0×2.0 μm, and the pixel resolution is 300×300 to 600×600.

6. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1 or 2, characterized in that: The power supply and information transmission of the miniaturized imaging device are achieved through the data cable (5), which may include an integrated structure of a power line and a signal line, and is preferably a multi-core shielded cable or a flexible cable.

7. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1, characterized in that: The probe housing (6) is a cylinder made of metal material (preferably aluminum), with a diameter of 4mm to 6mm and a height of 4mm to 6mm. It is provided with a plurality of (preferably four) through holes with a diameter of 1.5mm to 2.5mm, arranged symmetrically in a square shape, and a spacing between adjacent through holes of 2-5mm.

8. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1, characterized in that: The flexible coating material (7) is a flexible polymer material, preferably polyamide or polyurethane, and is used to coat and protect the integrated flexible light field system. After coating, the diameter of the entire system is 6.5 mm to 8.0 mm.

9. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1, characterized in that: The lens (3) in the miniaturized imaging device needs to be calibrated for each sub-device before use. The calibration method is to obtain the internal parameters and distortion coefficients of the system by combining the checkerboard method and the Zhang Zhengyou calibration method, and then bring them into the distortion equation to correct the image distortion.

10. The highly flexible integrated three-dimensional endoscope based on a multi-eye miniaturized imaging device according to claim 1, characterized in that: The system uses four sub-devices to shoot the target and correct the distortion. In order to reconstruct the three-dimensional information, it is necessary to use the Fourier light field deconvolution method. The quantitative depth information of the sample can be obtained through Fourier light field deconvolution.

Images