Asymmetric binocular endoscope and three-dimensional reconstruction method therefor

Abstract

The present application relates to the field of endoscopes. Disclosed are an asymmetric binocular endoscope and a three-dimensional reconstruction method therefor, which can ensure an ultra-high definition and also solve the problem of a spatial arrangement being insufficient when a binocular camera module is used in an endoscope. The method comprises: a first camera module performing white-light imaging, and a second camera module performing fluorescence imaging, wherein the diameter of the first camera module is greater than the diameter of the second camera module, and the resolution of the first camera module is greater than the resolution of the second camera module; processing an image collected by means of the first camera module, such that the resolution of an image obtained by processing the image collected by means of the first camera module is consistent with the resolution of an image collected by means of the second camera module; on the basis of the image obtained by processing the image collected by means of the first camera module and the image collected by means of the second camera module, performing registration and three-dimensional reconstruction to obtain a three-dimensional image; overlaying the three-dimensional image with, as texture, the image collected by means of the first camera module; and Superimposing, onto the texture, the image collected by means of the second camera module, so as to obtain a fluorescence three-dimensional image.

Description

Asymmetric binocular endoscope and three-dimensional reconstruction method thereof Technical Field

[0001] The present application relates to the field of endoscopes, and in particular to an asymmetric binocular endoscope and a three-dimensional reconstruction method thereof. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present application as recited in the claims. No admission is made that the description herein is prior art as disclosed by virtue of its inclusion in this section.

[0003] An endoscope is a diagnostic instrument that integrates traditional optics, ergonomics, precision mechanics, modern electronics, mathematics, and software. It features an image sensor, optical lens, light source, and mechanical mechanism. It can be inserted into the stomach through the mouth or other natural orifices. Endoscopes are extremely valuable to doctors because they can reveal lesions that X-rays cannot. For example, an endoscope allows doctors to visualize ulcers or tumors in the stomach, enabling them to develop optimal treatment plans.

[0004] Traditional endoscopes only have a single camera module. Currently, a binocular endoscope with two camera modules is provided on the market to solve the problem of low image quality in the white light imaging mode of traditional endoscopes. However, the current common binocular endoscopes use two camera modules with exactly the same diameter and resolution. If both camera modules are high-definition or ultra-high-definition, there is a problem of insufficient space arrangement in the endoscope with a very narrow diameter. Therefore, a certain degree of clarity needs to be sacrificed to allow the two camera modules to be accommodated in the endoscope, which also leads to the problem of low imaging quality. In addition, fluorescence synthesis or enhancement does not require the same resolution as white light imaging, which also leads to the problem of waste of camera module resources. Summary of the Invention

[0005] The purpose of this application is to provide an asymmetric binocular endoscope and a three-dimensional reconstruction method thereof, which can solve the problem of insufficient spatial arrangement of binocular camera modules on the endoscope while ensuring clarity.

[0006] The present application discloses a three-dimensional reconstruction method of an asymmetric binocular endoscope, comprising:

[0007] The first camera module 1 performs white light imaging, and the second camera module 2 performs fluorescence imaging, wherein the diameter of the first camera module 1 is larger than the diameter of the second camera module 2, and the resolution of the first camera module 1 is larger than the resolution of the second camera module 2;

[0008] Processing the image captured by the first camera module 1 so that the resolution of the processed image captured by the first camera module 1 is consistent with the resolution of the image captured by the second camera module 2;

[0009] Based on the image obtained by processing the image captured by the first camera module 1 and the image captured by the second camera module 2, the three-dimensional reconstruction is performed to obtain a three-dimensional image;

[0010] Overlaying the image captured by the first camera module 1 as a texture on the three-dimensional image;

[0011] The image captured by the second camera module 2 is superimposed on the texture to obtain a fluorescent three-dimensional image.

[0012] In a preferred embodiment, the processing of the image captured by the first camera module 1 further includes:

[0013] The image captured by the first camera module 1 is downsampled so that the resolution of the image captured by the first camera module 1 after downsampling is consistent with the resolution of the image captured by the second camera module 2.

[0014] In a preferred embodiment, the processing of the image captured by the first camera module 1 further includes:

[0015] The image captured by the second camera module 2 is interpolated so that the resolution of the interpolated image captured by the second camera module 2 is consistent with the resolution of the image captured by the first camera module 1.

[0016] In a preferred example, the resolution of the first camera module 1 is 4K, and the resolution of the second camera module 2 is 2K.

[0017] The present application also discloses an asymmetric binocular endoscope, comprising:

[0018] A first camera module 1, wherein the first camera module 1 is configured to perform white light imaging;

[0019] a second camera module 2, wherein the second camera module 2 is configured to perform fluorescence imaging;

[0020] The diameter of the first camera module 1 is larger than the diameter of the second camera module 2;

[0021] The resolution of the first camera module 1 is greater than the resolution of the second camera module 2;

[0022] The image captured by the first camera module 1 is configured to have a resolution consistent with the resolution of the image captured by the second camera module 2 so as to perform three-dimensional reconstruction to obtain a three-dimensional image.

[0023] In a preferred example, the resolution of the first camera module 1 is 4K, and the resolution of the second camera module 2 is 2K.

[0024] In a preferred example, the image captured by the first camera module 1 is configured to be downsampled so that the resolution is consistent with the resolution of the image captured by the second camera module 2.

[0025] In a preferred example, the image captured by the second camera module 2 is configured to be interpolated so that the resolution is consistent with the resolution of the image captured by the first camera module 1.

[0026] In a preferred example, it also includes a first light source module, a second light source module, a lens and a light guide. The first camera module 1, the second camera module 2 and the light guide are located in the same endoscope tube. The first light source module is configured as a fluorescent light source with an emission wavelength of 760nm-900nm, the second light source module is configured as a white light source with an emission wavelength of 400nm-700nm, the lens is configured to refract the fluorescent light source and the white light source, and the light guide is configured to transmit the refracted light from the fluorescent light source and the white light source to the illumination lens part of the endoscope.

[0027] The present application also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps in the method described above are implemented.

[0028] In the embodiment of the present application, a first camera module with a larger diameter and higher resolution and a second camera module with a smaller diameter and lower resolution are used. The first camera module is used for white light imaging, and the second camera module is used for fluorescence imaging. Since fluorescence synthesis or enhancement does not require the same resolution as white light imaging, the higher resolution of the first camera module is processed to make it consistent with the resolution of the second camera module, and then three-dimensional reconstruction is performed to obtain a three-dimensional image of the lesion. Subsequently, the image collected by the higher-definition first camera module is superimposed on the three-dimensional image as a texture. The doctor can not only observe the three-dimensional image of the lesion, but also observe clearer textures, that is, a three-dimensional texture image of the lesion with higher clarity is obtained. In the case where there is almost no extra space to be allocated in the narrow endoscope channel, the present application can simultaneously accommodate two camera modules with high-definition or ultra-high-definition clarity in a narrow diameter endoscope channel, so that the doctor can clearly observe the three-dimensional graphics of the lesion.

[0029] Furthermore, by superimposing the fluorescent image obtained by the second camera module through fluorescence imaging on the three-dimensional image with high-definition texture, a high-definition three-dimensional texture image and a fluorescent image can be obtained simultaneously within the same range of the lesion image to be observed, which can help doctors better observe tumors and other lesions.

[0030] The various technical features disclosed in the above summary of the invention, the various technical features disclosed in the various embodiments and examples below, and the various technical features disclosed in the accompanying drawings can be freely combined with each other to form various new technical solutions. These technical solutions should be deemed to have been recorded in this specification unless such combination of technical features is technically infeasible. For example, in one example, features A+B+C are disclosed, and in another example, features A+B+D+E are disclosed. Features C and D are equivalent technical means that perform the same function. Technically, only one of them needs to be used, and it is impossible to use them simultaneously. Feature E can be technically combined with feature C. In this case, the solution A+B+C+D should not be deemed to have been recorded because it is technically infeasible, while the solution A+B+C+E should be deemed to have been recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a flow chart of a three-dimensional reconstruction method of an asymmetric binocular endoscope according to one embodiment of the present application.

[0032] FIG2 is a schematic structural diagram of an asymmetric binocular endoscope according to one embodiment of the present application;

[0033] Description of reference numerals:

[0034] 1-first camera module; 2-second camera module. DETAILED DESCRIPTION

[0035] In the following description, many technical details are provided to help readers better understand this application. However, those skilled in the art will understand that even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented.

[0036] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0037] The first embodiment of the present application relates to a three-dimensional reconstruction method of an asymmetric binocular endoscope, the flow chart of which is shown in FIG1 . The method comprises:

[0038] In step 101 , the first camera module 1 performs white light imaging, and the second camera module 2 performs fluorescence imaging, wherein the diameter of the first camera module 1 is larger than the diameter of the second camera module 2 , and the resolution of the first camera module 1 is larger than the resolution of the second camera module 2 .

[0039] In step 102 , the image captured by the first camera module 1 is processed so that the resolution of the processed image captured by the first camera module 1 is consistent with the resolution of the image captured by the second camera module 2 .

[0040] In step 103, based on the image obtained by processing the image captured by the first camera module 1 and the image captured by the second camera module 2, they are aligned and reconstructed to obtain a three-dimensional image.

[0041] In step 104 , the image captured by the first camera module 1 is overlaid onto the three-dimensional image as a texture.

[0042] In step 105, the image captured by the second camera module 2 is superimposed on the texture to obtain a fluorescent three-dimensional image.

[0043] In an optional embodiment, step 102 may further include: downsampling the image captured by the first camera module 1 so that the resolution of the image captured by the first camera module 1 after downsampling is consistent with the resolution of the image captured by the second camera module 2.

[0044] In an optional embodiment, step 102 may further include: performing interpolation processing on the image captured by the second camera module 2 so that the resolution of the interpolated image captured by the second camera module 2 is consistent with the resolution of the image captured by the first camera module 1.

[0045] In an optional embodiment, the resolution of the first camera module 1 may be 4K, and the resolution of the second camera module 2 may be 2K.

[0046] In order to better understand the technical solution of the present application, several specific examples are provided below for illustration. The details listed in the examples are mainly for ease of understanding and are not intended to limit the scope of protection of the present application.

[0047] Example 1

[0048] The first camera module 1 performs white light imaging, and the second camera module 2 performs fluorescence imaging, wherein the diameter of the first camera module 1 is larger than the diameter of the second camera module 2 , and the resolution of the first camera module 1 is larger than the resolution of the second camera module 2 .

[0049] The image captured by the first camera module 1 is downsampled so that the resolution of the downsampled image captured by the first camera module 1 is consistent with the resolution of the image captured by the second camera module 2.

[0050] Based on the image obtained by downsampling the image captured by the first camera module 1 and the image captured by the second camera module 2, three-dimensional reconstruction is performed to obtain a three-dimensional image;

[0051] Overlaying the image captured by the first camera module 1 on the three-dimensional image as a texture;

[0052] The image captured by the second camera module 2 is superimposed on the texture to obtain a fluorescent three-dimensional image.

[0053] Example 2

[0054] The first camera module 1 performs white light imaging, and the second camera module 2 performs fluorescence imaging, wherein the diameter of the first camera module 1 is larger than the diameter of the second camera module 2 , and the resolution of the first camera module 1 is larger than the resolution of the second camera module 2 .

[0055] The image captured by the second camera module 2 is interpolated so that the resolution of the interpolated image captured by the second camera module 2 is consistent with the resolution of the image captured by the first camera module 1.

[0056] Based on the image acquired by the first camera module 1 and the image acquired by the second camera module 2, the images are interpolated and registered to perform three-dimensional reconstruction to obtain a three-dimensional image;

[0057] Overlaying the image captured by the first camera module 1 on the three-dimensional image as a texture;

[0058] The image captured by the second camera module 2 is superimposed on the texture to obtain a fluorescent three-dimensional image.

[0059] The second embodiment of the present application relates to an asymmetric binocular endoscope, a structural diagram of which is shown in FIG2 . The asymmetric binocular endoscope includes a first camera module 1 and a second camera module 2 .

[0060] The first camera module 1 is configured to perform white light imaging;

[0061] The second camera module 2 is configured to perform fluorescence imaging;

[0062] The diameter of the first camera module 1 is larger than the diameter of the second camera module 2;

[0063] The resolution of the first camera module 1 is greater than the resolution of the second camera module 2;

[0064] The image captured by the first camera module 1 is configured to have a resolution consistent with the resolution of the image captured by the second camera module 2 so as to perform three-dimensional reconstruction to obtain a three-dimensional image.

[0065] In an optional embodiment, the resolution of the first camera module 1 is 4K.

[0066] In an optional embodiment, the resolution of the second camera module 2 is 2K.

[0067] In an optional embodiment, the image captured by the first camera module 1 is configured to be downsampled so that the resolution is consistent with the resolution of the image captured by the second camera module 2.

[0068] In an optional embodiment, the image captured by the second camera module 2 is configured to be interpolated so that the resolution is consistent with the resolution of the image captured by the first camera module 1.

[0069] In an optional embodiment, it may also include a first light source module, a second light source module, a lens and a light guide, the first camera module 1, the second camera module 2 and the light guide are located in the same endoscope tube, the first light source module is configured as a fluorescent light source with an output wavelength of 760nm-900nm, the second light source module is configured as a white light source with an output wavelength of 400nm-700nm, the lens is configured to refract the fluorescent light source and the white light source, and the light guide is configured to transmit the refracted light from the fluorescent light source and the white light source to the illumination lens part of the endoscope.

[0070] In order to better understand the technical solution of the present application, a specific example is provided below for illustration. The details listed in the example are mainly for ease of understanding and are not intended to limit the scope of protection of the present application.

[0071] The asymmetric binocular endoscope includes two camera modules: a first camera module 1 and a second camera module 2. The diameter and resolution of the first camera module 1 are both larger than those of the second camera module 2. The first camera module 1 has a higher resolution, such as 4K, while the second camera module 2 has a lower resolution, such as 2K. Both camera modules are located within a single endoscope channel.

[0072] The first camera module 1 is used for white light imaging, and the second camera module 2 is used for fluorescence imaging. Preferably, the image captured by the first camera module 1 is downsampled to a resolution consistent with that of the image captured by the second camera module 2, thereby enabling 3D reconstruction to produce a 3D image. Alternatively, the image captured by the second camera module 2 can be interpolated to a resolution consistent with that of the first camera module 1, thereby enabling 3D reconstruction to produce a 3D image.

[0073] Technical indicators:

[0074] Optics

[0075] Camera element: white light 2*5K cmos, fluorescent 2*2K cmos, 60 frames

[0076] Depth of field: 3-200mm

[0077] Field of view: 100°

[0078] Viewing angle: 0 / 30° Spectral range: White light 400-700nm; Fluorescence 760-900nm

[0079] Fluorescence: NIR (Near Infrared), ICG (Indigo Green)

[0080] illumination

[0081] Multispectral: White light 400-760nm LED; NBI (Narrow Band Imaging) (415m+530nm) LED, color rendering index ≥90; output light intensity blue 600lm

[0082] Fluorescence excitation: LED, 785nm+805nm

[0083] White light brightness: automatic / manual.

[0084] image

[0085] Display: white light, fluorescence, white light + fluorescence fusion, 6-color fluorescence, color fluorescence.

[0086] Chromatography: white light + NBI, white light

[0087] Image processing: 12-zone metering, defogging, metal reflection processing, dynamic contrast enhancement, adaptive sharpening adjustment, etc.

[0088] Image output: 3840*2160X; 12G-SDI*4

[0089] The first embodiment is a method embodiment corresponding to the present embodiment. The technical details in the first embodiment can be applied to the present embodiment, and the technical details in the present embodiment can also be applied to the first embodiment.

[0090] Accordingly, the embodiments of the present application also provide a computer-readable storage medium having computer-executable instructions stored therein, which implement the various method embodiments of the present application when executed by a processor. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0091] It should be noted that, in this application, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further restriction, an element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article, or device comprising the element. In this application, if it is mentioned that an action is performed according to a certain element, it means that the action is performed at least according to that element, including two situations: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "multiple," and "multiple" include 2, 2 times, 2 kinds, and more than 2, more than 2 times, and more than 2 kinds.

[0092] The serial numbers used in describing the steps of a method do not themselves limit the order of these steps. For example, a step with a larger serial number does not necessarily have to be executed after a step with a smaller serial number. The step with a larger serial number can be executed first and then the step with a smaller serial number, or they can be executed in parallel, as long as this execution order is reasonable to those skilled in the art. For another example, having multiple steps with consecutive serial numbers (e.g., step 101, step 102, step 103, etc.) does not limit other steps that can be executed in between. For example, there can be other steps between step 101 and step 102.

[0093] This specification includes combinations of the various embodiments described herein. Individual references to embodiments (e.g., "one embodiment" or "some embodiments" or "preferred embodiments") are not mutually exclusive unless indicated as such or clear to one skilled in the art. It should be noted that the word "or" is used in this specification in a non-exclusive sense unless the context clearly indicates or requires otherwise.

[0094] All documents mentioned in this specification are considered to be included in their entirety in the disclosure of this application so that they can be used as a basis for modification when necessary. In addition, it should be understood that the above description is only a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification should be included in the scope of protection of one or more embodiments of this specification.

[0095] In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A three-dimensional reconstruction method of an asymmetric binocular endoscope, characterized in that: include: The first camera module (1) performs white light imaging, and the second camera module (2) performs fluorescence imaging, wherein the diameter of the first camera module (1) is greater than the diameter of the second camera module (2), and the resolution of the first camera module (1) is greater than the resolution of the second camera module (2); Processing the image captured by the first camera module (1) so that the resolution of the processed image captured by the first camera module (1) is consistent with the resolution of the image captured by the second camera module (2); Based on the image obtained by processing the image collected by the first camera module (1) and the image collected by the second camera module (2), registering and performing three-dimensional reconstruction to obtain a three-dimensional image; Overlaying the image captured by the first camera module (1) as a texture on the three-dimensional image; The image captured by the second camera module (2) is superimposed on the texture to obtain a fluorescent three-dimensional image.

2. The three-dimensional reconstruction method of an asymmetric binocular endoscope according to claim 1, characterized in that: The processing of the image captured by the first camera module (1) further comprises: The image captured by the first camera module (1) is downsampled so that the resolution of the image captured by the first camera module (1) after the downsampling processing is consistent with the resolution of the image captured by the second camera module (2).

3. The three-dimensional reconstruction method of an asymmetric binocular endoscope according to claim 1, characterized in that: The processing of the image captured by the first camera module (1) further comprises: Interpolation processing is performed on the image captured by the second camera module (2) so that the resolution of the interpolated image captured by the second camera module (2) is consistent with the resolution of the image captured by the first camera module (1).

4. The three-dimensional reconstruction method of an asymmetric binocular endoscope according to claim 1, characterized in that: The resolution of the first camera module (1) is 4K, and the resolution of the second camera module (2) is 2K.

5. An asymmetric binocular endoscope, characterized in that: include: A first camera module (1), wherein the first camera module (1) is configured to perform white light imaging; A second camera module (2), wherein the second camera module (2) is configured to perform fluorescence imaging; The diameter of the first camera module (1) is greater than the diameter of the second camera module (2); The resolution of the first camera module (1) is greater than the resolution of the second camera module (2); The image captured by the first camera module (1) is configured to have a resolution consistent with the resolution of the image captured by the second camera module (2) so as to perform three-dimensional reconstruction to obtain a three-dimensional image.

6. The asymmetric binocular endoscope according to claim 5, characterized in that: The resolution of the first camera module (1) is 4K, and the resolution of the second camera module (2) is 2K.

7. The asymmetric binocular endoscope according to claim 5, characterized in that: The image captured by the first camera module (1) is configured to be downsampled so that the resolution is consistent with the resolution of the image captured by the second camera module (2).

8. The asymmetric binocular endoscope according to claim 5, characterized in that: The image captured by the second camera module (2) is configured to be interpolated so that the resolution is consistent with the resolution of the image captured by the first camera module (1).

9. The asymmetric binocular endoscope according to claim 5, characterized in that: The invention also comprises a first light source module, a second light source module, a lens and a light guide, wherein the first camera module (1), the second camera module (2) and the light guide are located in the same endoscope tube, the first light source module is configured as a fluorescent light source with an emission wavelength of 760nm-900nm, the second light source module is configured as a white light source with an emission wavelength of 400nm-700nm, the lens is configured to refract the fluorescent light source and the white light source, and the light guide is configured to transmit the refracted light from the fluorescent light source and the white light source to the illumination lens portion of the endoscope.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps in the method according to any one of claims 1 to 4 are implemented.

Images