Multi-view endoscope imaging system

Abstract

The invention relates to the technical field of medical apparatus and instruments, and discloses a multi-view endoscope imaging system, which comprises a multispectral light source; the multi-view imaging front end comprises a front end column body, and an imaging system and an illumination system which are arranged on the front end column body, the imaging system comprises a main camera module located on the end face of the front-end column body and auxiliary camera modules located on the side face of the front-end column body and distributed at equal intervals. The lighting system comprises a main lighting module located on the end face of the front-end column body and auxiliary lighting modules located on the side face of the front-end column body and matched with the auxiliary camera modules. The main illumination module is in light path connection with the multispectral light source; the image processing system is in signal connection with the main camera module and the auxiliary camera module; according to the method, the lesion can be observed from different angles in combination with a multi-view endoscope, more comprehensive lesion features are provided in combination with a deep learning model, and the accuracy and reliability of polyp detection can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a multi-viewing angle endoscope imaging system. Background Art

[0002] With the continuous development of medical technology, early diagnosis and precise treatment have become important goals in the field of digestive tract diseases. Polyps, especially gastrointestinal polyps, are common lesions. Early detection and accurate identification are crucial to preventing malignant transformation and timely treatment. Although traditional endoscopic examination is the main means, it still has certain limitations in polyp detection. For example, some tiny polyps or polyps with atypical morphology may be missed. At the same time, in recent years, narrow-band optical imaging technology has gradually been applied to the field of endoscopy. Through the irradiation of light of a specific wavelength, it can highlight the fine structure and vascular morphology of the mucosal surface, which helps to improve the ability to identify polyps. However, relying solely on the doctor's naked eye observation and experience judgment is still subjective and unstable.

[0003] How to design an endoscope that can help medical workers identify polyps or lesions more quickly and efficiently under limited outer diameter conditions, while reducing the chance of missed polyps in real-time diagnosis, is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a multi-view endoscope imaging system.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions: A multi-view endoscope imaging system, comprising: Multispectral light sources, which can emit white light as well as light in a specific frequency range; A multi-view imaging front end, comprising a front end column, and an imaging system and a lighting system arranged on the front end column; the imaging system comprises a main camera module located on the end face of the front end column and a plurality of equally spaced auxiliary camera modules located on the side face of the front end column; the lighting system comprises a main lighting module located on the end face of the front end column and an auxiliary lighting module located on the side face of the front end column and coordinated with each auxiliary camera module; the main lighting module is optically connected to a multi-spectral light source; Image processing system, used to receive and process image signals from the main camera module and the auxiliary camera module, and to assist in the detection of polyps and suspected lesions by combining artificial intelligence algorithms; The field of view of the main camera module is 140°, and the field of view of the auxiliary camera module is 120°; if the imaging plane of the auxiliary camera module is tilted toward the center of the end face of the front end column, and forms an inclination angle of not less than 82° with the cross section of the front end column, the imaging system can achieve an overlapping observation range of a field of view of not less than 280° in a cavity with a diameter of not less than 26 mm; if the imaging plane of the auxiliary camera module is perpendicular to the cross section of the front end column, the imaging system can achieve an overlapping observation range of a field of view of not less than 300° in a cavity with a diameter of not less than 26 mm.

[0006] The present invention realizes multi-view imaging through the main camera module and the auxiliary camera module, and also combines the deep learning model of multiple scenes to assist medical workers in detecting the locations of polyps and suspected lesions, thereby improving the work efficiency of medical workers.

[0007] Furthermore, the auxiliary lighting module is connected to an image processing system; the image processing system determines the illumination light intensity currently required by the auxiliary camera module based on the received image signal, and controls the output light intensity of the corresponding auxiliary lighting module based on the determined illumination light intensity currently required by the auxiliary camera module.

[0008] Furthermore, it also includes a water vapor system; the water vapor system includes a water vapor channel arranged in the front end column, a main water vapor nozzle arranged on the end face of the front end column and connected to the water vapor channel, and an auxiliary water vapor outlet arranged at each auxiliary camera module and connected to the water vapor channel; the auxiliary water vapor outlet is arranged toward the auxiliary camera module and the auxiliary lighting module.

[0009] Furthermore, it also includes a four-way rotating handle and a bending snake bone; the front end column is installed on the bending snake bone, and the bending direction of the bending snake bone is controlled by the four-way rotating handle, thereby controlling the direction of the front end column.

[0010] Furthermore, it also includes a light guide part; the main lighting module includes an aspherical light homogenizing lens and an illumination optical fiber, and the illumination optical fiber is connected to the multi-spectral light source through the light guide part.

[0011] Compared with the prior art, the beneficial technical effects of the present invention are: Under limited outer diameter conditions, the characteristic patterns of polyps and suspected lesions are learned through existing artificial intelligence algorithms to achieve automatic detection and identification of the locations of polyps and suspected lesions. Combined with multi-view endoscopes, lesions can be observed from different angles, providing more comprehensive lesion characteristics, which can significantly improve the accuracy and reliability of polyp detection and further enhance the work efficiency of medical workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram of a multi-view endoscope imaging system in an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a multi-view imaging front end in an embodiment of the present invention; Figure 3 is a connection diagram of a lighting system in an embodiment of the present invention; Figure 4 This is a block diagram of multi-view adaptive equalization lighting control in an embodiment of the present invention; Figure 5 It is a partially cutaway left-side structural schematic diagram of a water gas system in an embodiment of the present invention; Figure 6 Schematic diagram of the overlapping side view fields of a multi-view imaging front end in an embodiment of the present invention; Figure 7 Schematic diagram of the overlapping fields of view of the auxiliary camera modules in an embodiment of the present invention; Figure 8 Schematic diagram of the display mode of the multi-view endoscope imaging system in an embodiment of the present invention.

[0013] In the figure: 1. Multi-view imaging front end; 2. Four-way rotating handle; 3. Light guide; 4. Multi-channel anti-shielding aviation plug; 5. Image processing system; 6. Multi-spectral light source; 7. Display; 8. Cavity; 10. Main camera module; 11. Main lighting module; 12. Main water and gas nozzle; 13. Auxiliary water supply channel; 14. Instrument channel; 15. Auxiliary water and gas outlet; 16. Auxiliary lighting module; 17. Auxiliary camera module; 18. Auxiliary assembly base; 19. Bending snake bone; 110. Aspheric homogenizing lens; 111. Illumination optical fiber; 120. Auxiliary water and gas outlet inside the side; 121. Water and gas channel processing port; 122. Water and gas valve Control the water vapor input port; 20. Field of view coverage of the main camera module; 21. Overlapping range of the field of view of the main camera module and the auxiliary camera module; 22. Blind spot of the field of view of the main camera module and the auxiliary camera module; 23. Field of view coverage of the auxiliary camera module; 30. Overlapping imaging area of ​​adjacent auxiliary camera modules; 31. Blind spot of auxiliary camera module imaging; 32. Auxiliary camera module imaging area; 41. Image display area of ​​the main camera module; 42. Imaging display area of ​​the first auxiliary camera module; 43. Imaging display area of ​​the second auxiliary camera module; 44. Imaging display area of ​​the third auxiliary camera module; 45. Imaging display area of ​​the fourth auxiliary camera module. DETAILED DESCRIPTION

[0014] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0015] like Figure 1 As shown, the multi-view endoscope imaging system of the present invention includes a multi-view imaging front end 1, a four-direction rotating handle 2, a light guide 3, a multi-channel anti-shielding aviation plug 4, an image processing system 5, a multi-spectral light source 6 and a display 7.

[0016] The multi-spectral light source 6 can realize white light imaging and narrow-band light imaging in a special mode.

[0017] The multi-view imaging front end 1 includes an imaging system, a lighting system and a water and gas system, specifically including a front end column, a main camera module 10, a main lighting module 11, a main water and gas nozzle 12, an auxiliary water supply channel 13, an instrument channel 14, an auxiliary water and gas outlet 15, an auxiliary lighting module 16, an auxiliary camera module 17, an auxiliary assembly base 18, and a bending snake bone 19.

[0018] In order to achieve 360° observation inside the natural cavity of the human body, the auxiliary camera modules are distributed at equal distances on the sides of the multi-view imaging front end. According to the different assembly orientations of the auxiliary camera modules in the positive direction, the imaging front end can obtain different imaging field of view coupling states.

[0019] Figure 2 The schematic diagram of the structure of the multi-view imaging front end 1 is shown in FIG. The field of view of the main camera module 10 is 140° and the resolution is , which can reach 2.25 million pixels. The main lighting module 11 uses an aspherical light-homogenizing lens coupled with an illumination fiber to achieve uniform illumination in the imaging field of the main camera module. The main water-gas nozzle 12 enables the surface cleaning of the main camera module and the main lighting module when the endoscope is working in the body. The auxiliary water delivery channel 13 is used to meet the water delivery and cleaning needs in the body cavity.

[0020] In this embodiment, the inner diameter of the instrument channel 14 is 2.8 mm.

[0021] The auxiliary water vapor outlet 15 and the corresponding water vapor channel are hidden structures, and the surface welding structural parts are not shown, so as to realize the surface cleaning requirements of the auxiliary camera module and the auxiliary lighting module.

[0022] The auxiliary lighting module 16 uses a small-sized light-emitting diode (LED) with a large divergence angle, which is arranged on both sides of the auxiliary camera module to achieve uniform illumination in the imaging field of the auxiliary camera module.

[0023] Four auxiliary camera modules 17 of the same specification are equidistantly spaced on the side of the front end column. In this embodiment, the auxiliary camera module has a viewing angle of 120° and a resolution of , which can reach about 520,000 pixels. The auxiliary assembly base 18 is used to ensure the structural assembly stability of the auxiliary camera module and the auxiliary lighting module and to ensure the position distribution of the auxiliary camera module and the auxiliary lighting module, and to ensure the lighting uniformity of the auxiliary lighting module within the imaging range of the auxiliary camera module. The bending snake bone 19 is connected to the front end column to realize large-angle bending and observation in four directions of the imaging front end.

[0024] The multi-view imaging front end has complete functions and reasonable assembly. There are no sharp structures such as protrusions and edges on the outer surface. The outer diameter does not exceed 13.8mm and the length does not exceed 14mm, which meets the needs of intestinal usage scenarios.

[0025] The imaging quality of the multi-view endoscope imaging system is greatly affected by the lighting quality. In order to ensure the lighting quality of the imaging front end of the multi-view endoscope when imaging in vivo, the imaging lighting system is designed in combination with the outer diameter size limit of the imaging front end. The lighting system includes a main lighting module 11 that cooperates with the main camera module 10 for imaging and an auxiliary lighting module 16 that cooperates with the auxiliary camera module 17 for imaging. Figure 3 As shown, the main lighting module 11 is composed of an aspheric light-homogenizing lens 110 and an illumination fiber 111. The aspheric lens 110 is designed for secondary optics with reference to a light-emitting diode (LED). The nanoimprint manufacturing technology is combined to ensure the processing accuracy of the aspheric lens, realize the surface processing of the complex lens surface of the polynomial aspheric lens system, and ensure the uniformity of the light field within the 140° imaging field of view of the main camera module 10. At the same time, the design of a single aspheric lens greatly reduces the assembly difficulty and assembly size limitations of the original multi-spherical lens. The illumination fiber 111 uses a specific aperture glass fiber, combined with the length of the multi-view endoscope body, to process and assemble the illumination fiber. After the illumination fiber 111 is assembled in the body, it is connected to the multi-spectral light source 6 through the light guide 3. The light guide 3 and the multi-spectral light source 6 are connected through a special structural connector. The light source that meets the specific structural interface can be connected and coupled with the light guide 3 to realize the output of any spectral combination illumination light at the front end of multi-view imaging. The auxiliary lighting module 16 uses a small-sized, large-divergence-angle light-emitting diode (LED), which is assembled together with the auxiliary camera module 17 in the auxiliary assembly base 18, and then connected to the image processing system 5 after being welded to the multi-channel anti-shielding aviation plug 4 through a cable. In this embodiment, the functions of the image processing system include receiving, processing, and outputting image signals of the imaging system and independent intensity control of the light-emitting diode (LED) of the auxiliary lighting module.

[0026] In actual use by medical workers, the multi-view endoscope is often in a non-centered position. For example, when the multi-view imaging front end 1 is in a non-coaxial eccentric position in the intestine, one side is closer to the intestinal wall, and the auxiliary imaging module 17 will cause uneven or overbright illumination light field distribution due to the complex structure of the intestinal wall folds, which will lead to uneven image brightness or overexposure; the auxiliary imaging module 17 far away from the intestinal wall will have too low illumination intensity due to its relatively long distance from the intestinal wall, resulting in insufficient image brightness. The multi-view imaging system establishes a feedback mechanism based on the image brightness distribution state of each auxiliary imaging module 17 (the image brightness distribution state is obtained based on the received image signal), such as Figure 4As shown, the image processing system 5 adaptively adjusts the illumination output light intensity of the corresponding position of the auxiliary imaging module, thereby improving the real-time imaging quality of the imaging module and realizing multi-view adaptive balanced lighting design.

[0027] Compared with ordinary gastrointestinal endoscopes, the multi-view endoscopic imaging system has more imaging modules and can observe more image information. However, the complex cavity environment of the digestive tract often causes foreign matter to adhere to the surface of the imaging module, resulting in obstruction of the image and inability to observe normally. The foreign matter on the surface of the imaging module needs to be cleaned through a water-gas system. The main water-gas nozzle 12 or the auxiliary water-gas outlet 15 is a key component of the multi-view imaging front end for normal imaging in the body. Figure 5 It is a cross-sectional left view of the internal structure of the water and gas system of the multi-view imaging front end. The main water and gas nozzle 12 is connected to the internal water and gas channel, and the side internal auxiliary water and gas outlet 120 is connected to the internal water and gas channel. The water and gas channel processing port 121 is sealed with a plug or a screw combined with glue. The side internal auxiliary water and gas outlet 120 and the water and gas channel processing port 121 are provided with metal structural parts that play a sealing role, so that the water or gas output from the side internal auxiliary water and gas outlet 120 is output to the auxiliary water and gas outlet 15. The internal water and gas channel is fluid simulated by software to achieve a one-to-five equal output of the flow. The water or gas input port 122 is controlled by the water and gas valve at the handle position to input a specific flow of water or gas. The water or gas is diverted through the water and gas channel to achieve an equal flow output of water or gas from one main water and gas nozzle 12 and four auxiliary water and gas outlets 15, so as to achieve the purpose of cleaning the dirt on the surface of the imaging module and ensure the display and acquisition of normal images when the imaging front end is working in the body. The one-to-five water-gas channel design of this embodiment has a simple structure, low processing difficulty, and low assembly difficulty, and solves the practical use problem of cleaning the surface dirt of the multi-view imaging module at the imaging front end.

[0028] The side view field of the multi-view imaging front end in the cavity 8 is shown in FIG. Figure 6 , the main camera module 10 has a field of view of 140°, and the auxiliary camera module 17 has a field of view of 120°. The main camera module field of view coverage range 20, the auxiliary camera module field of view coverage range 23, the main camera module and the auxiliary camera module field of view overlap range 21, the main camera module and the auxiliary camera module field of view blind area 22 refer to Figure 6. In this embodiment, in order to better couple the auxiliary camera module 17 with the main camera module in field of view, and in order to better coordinate the auxiliary camera module with the auxiliary water vapor outlet, the auxiliary camera module assembly surface (the imaging surface of the auxiliary camera module is parallel to the assembly surface) forms an inclination angle of not less than 82° with the cross section of the front end column, so that the imaging system composed of the main camera module and the auxiliary camera module can achieve an overlapping observation range of not less than 280° in a cavity with a diameter of not less than 26 mm. If the auxiliary camera module assembly surface is perpendicular to the cross section of the front end column, the imaging system can form a maximum overlapping observation range of not less than 300° in a cavity with a diameter of not less than 26 mm. The closer the distance between the main camera module and the auxiliary camera module, the better the effect. However, due to structural size limitations, in this embodiment, the main camera module and the auxiliary camera module use the shortest distance allowed by the structure. Specifically, the distance between the center of the auxiliary camera module and the center of the main camera module in the axial direction of the front end column is 11.4 mm. Figure 6 As shown, this embodiment achieves an overlapping observation range with a field of view angle of 284.41°. Figure 6 The 25.38 in the figure represents the minimum cavity diameter required for the observation fields to overlap, in units of mm.

[0029] In this embodiment, four auxiliary camera modules 17 with a 120° viewing angle are arranged in a ring at equal intervals on the side of the front column, forming a Figure 7 The auxiliary camera module fields of view are superimposed, and the adjacent auxiliary camera module imaging overlap area 30, the auxiliary camera module imaging blind area 31, and the auxiliary camera module imaging area 32 are shown in FIG. Figure 7 The imaging field of the auxiliary camera module is a square area, and the 120° field angle is the diagonal field angle of the imaging field square area. The setting scheme of the auxiliary camera module in this embodiment can meet the requirement of forming a 360° imaging field of view within a circle with a diameter of not less than 46 mm.

[0030] The present invention realizes multi-view imaging while combining artificial intelligence algorithms to assist medical workers in detecting polyps and suspected lesions. When the multi-view endoscope of the present invention is working in the body, medical workers can turn on the intelligent assistance system at an appropriate time to identify the locations of polyps and suspected lesions. When the system is working, the main camera module and the four auxiliary camera modules simultaneously image. The main camera module image display area 41, the first auxiliary camera module imaging display area 42, the second auxiliary camera module imaging display area 43, the third auxiliary camera module imaging display area 44, and the fourth auxiliary camera module imaging display area 45, see Figure 8. The images of the five camera modules displayed simultaneously are adaptively adjusted through exposure time, gain, white balance, color space, and histogram equalization to achieve consistent display effects and facilitate observation by medical staff. When the intelligent assistance system suspects that suspected lesions or polyps appear in the intestine, the outer contour of the image display area at the corresponding position will be highlighted to remind medical workers of possible missed lesions, and the parts where the system highly suspects lesions will be marked in real time, helping medical workers to find more potentially dangerous lesions and achieve early prevention, early detection and early treatment of colorectal cancer. The artificial intelligence algorithm used in the present invention is an existing algorithm. In a preferred embodiment, the MSCAN architecture is used for training, a large number of colonoscopy images are collected, and the polyps therein are finely labeled. During the training process, the parameters of the network model are continuously adjusted so that the network model can learn the characteristics of the polyps, such as shape, color, texture, etc. At the same time, data from other modalities, such as narrow band imaging (NBI) or chromoendoscopy images, are also combined to input these different modal images into the network model, which can learn the characteristic performance of polyps in different modalities. By fusing multimodal information, the detection accuracy of the network model has been significantly improved. For some polyps with irregular morphology and low contrast with surrounding tissues, multimodal data provides the network model with richer features, which helps to detect and segment more accurately.

[0031] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0032] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A multi-view endoscope imaging system, characterized in that: include: Multi-spectral light source; A multi-view imaging front end, comprising a front end column, and an imaging system and a lighting system arranged on the front end column; the imaging system comprises a main camera module located on the end surface of the front end column and a plurality of equally spaced auxiliary camera modules located on the side surface of the front end column; The lighting system includes a main lighting module located on the end surface of the front end column and an auxiliary lighting module located on the side surface of the front end column and cooperating with each auxiliary camera module; The main lighting module is optically connected to the multi-spectrum light source; Image processing system, used to receive and process image signals from the main camera module and the auxiliary camera module, and to assist in the detection of polyps and suspected lesions by combining artificial intelligence algorithms; The field of view of the main camera module is 140°, and the field of view of the auxiliary camera module is 120°; if the imaging plane of the auxiliary camera module is tilted toward the center of the end face of the front end column, and forms an inclination angle of not less than 82° with the cross section of the front end column, the imaging system can achieve an overlapping observation range of a field of view of not less than 280° in a cavity with a diameter of not less than 26 mm; if the imaging plane of the auxiliary camera module is perpendicular to the cross section of the front end column, the imaging system can achieve an overlapping observation range of a field of view of not less than 300° in a cavity with a diameter of not less than 26 mm.

2. The multi-view endoscope imaging system according to claim 1, characterized in that: The auxiliary lighting module is connected to the image processing system; the image processing system determines the illumination light intensity currently required by the auxiliary camera module based on the received image signal, and controls the output light intensity of the corresponding auxiliary lighting module based on the determined illumination light intensity currently required by the auxiliary camera module.

3. The multi-view endoscope imaging system according to claim 1, characterized in that: It also includes a water vapor system; the water vapor system includes a water vapor channel arranged in the front end column, a main water vapor nozzle arranged on the end face of the front end column and connected to the water vapor channel, and an auxiliary water vapor outlet arranged at each auxiliary camera module and connected to the water vapor channel; the auxiliary water vapor outlet is arranged toward the auxiliary camera module and the auxiliary lighting module.

4. The multi-view endoscope imaging system according to claim 1, characterized in that: It also includes a four-way rotating handle and a bending snake bone; the front end column is installed on the bending snake bone, and the bending direction of the bending snake bone is controlled by the four-way rotating handle, thereby controlling the direction of the front end column.

5. The multi-view endoscope imaging system according to claim 1, characterized in that: It also includes a light guide part; the main lighting module includes an aspherical light homogenizing lens and an illumination optical fiber, and the illumination optical fiber is connected to the multi-spectral light source through the light guide part.

Images