A visualization system with real-time imaging function

Abstract

The present invention discloses a visualization system with real-time imaging function, which specifically includes a receiving end, a light source, an imaging system and a display device, wherein the light source includes a red light source and a fluorescent excitation light source, and the imaging system includes a red light image sensor and a fluorescent image sensor, and the signal is processed to output a real-time visualization signal to the display device. Through this technical solution, only using a specific band of red light to illuminate the surgical scene can provide the surgeon with sufficient background information and obtain a clear and complete real-time image. The overall structure of the visualization system is streamlined, the control method is convenient, the imaging processing process is simple and efficient, and the manufacturing and maintenance costs are low while having a good user experience, and it has extremely high applicability and scalability.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to a visualization system with real-time imaging function. Background Art

[0002] In the existing technology, fluorescence navigation imaging is a new surgical image acquisition technology that injects specific near-infrared fluorescent contrast agents before surgery, uses two light sources, white light and excitation light, to illuminate the imaging area, and simultaneously obtains a clear color image of the imaging area and a fluorescent image reflecting the lesion information. Regarding how to generate white light and analyze background information, the existing technology can usually be implemented using a single-chip RGB system or a three-chip RGB system. For a single-chip RGB system, since the single-chip system cannot capture and process white light signals and fluorescent signals at the same time, a time multiplexing setting needs to be added. In order to achieve a better imaging effect, the light source management system is often set up very complexly to achieve a high synchronization frequency (such as 50Hz or 60Hz) to alternately turn on / off the white light source and the fluorescent excitation light source; and for a three-chip RGB system, three sensors are required to detect the reflection signals of the respective RGB channels. In this case, since the red sensor is also used to detect near-infrared signals, in order to avoid frequent turning on and off of the light source, the red light is always turned off in this type of architecture, and only green and blue lights are used to provide an imaging environment, and the images obtained by the green and blue sensors are used to create a background image. In this structure, since the red light band information cannot be collected, some tissue information cannot be well displayed in the background image, such as being unable to capture certain types of anatomical structures in the human body, or providing too much background information to the surgeon, and the background is indistinguishable from the target boundary or key information, which can easily lead to confusion in diagnosis and treatment or even excessive resection. Therefore, an optimized visualization system is needed to avoid the above-mentioned potential imaging problems while greatly simplifying the image processing and light source excitation process to facilitate daily use and related maintenance, thereby expanding the scope of application of this new visualization system. Summary of the invention

[0003] In view of the above problems existing in the prior art, a visualization system with real-time imaging function is now provided, and the specific technical solution is as follows:

[0004] A visualization system with real-time imaging function, specifically comprising:

[0005] A receiving end, including a camera interface and a light guide interface;

[0006] A light source, connected to the light guide interface through a light guide, for continuously providing an imaging light source for the receiving end, the light source comprising a first light source and a second light source, the first light source emitting fluorescent excitation light, and the second light source emitting red light;

[0007] An imaging system, connected to the camera interface via a light guide connector, for acquiring fluorescence imaging signals and red light imaging signals respectively according to the reflected light of the imaging light source continuously received by the receiving end, and outputting a real-time observation image;

[0008] A display device is electrically connected to the imaging system and is used to display real-time observation images.

[0009] Preferably, in the visualization system, the imaging system comprises a camera group and a camera controller;

[0010] The camera group is used to continuously collect the reflected light of the imaging light source and filter it, and output the corresponding fluorescent image and red light image;

[0011] The camera controller is connected to the camera group via a camera cable, and outputs corresponding fluorescence imaging signals and red light imaging signals according to the fluorescence image and the red light image.

[0012] Preferably, in the visualization system, the camera controller is also electrically connected to the light source;

[0013] The camera controller generates a corresponding light source instruction according to an external user instruction, and the light source turns on or off the first light source and / or the second light source according to the light source instruction.

[0014] Preferably, in the visualization system, the camera group includes a fluorescent image sensor and a red light image sensor;

[0015] The fluorescence image sensor includes a fluorescence filter, which is used to filter all other reflected light except the fluorescence band corresponding to the fluorescence excitation light in the reflected light of the imaging light source, and the reflected light of the imaging light source generates a fluorescence image in the fluorescence image sensor through the fluorescence filter;

[0016] The red light image sensor includes a red light filter, which is used to filter all other reflected light in the imaging light source reflected light except the band corresponding to red light. The imaging light source reflected light generates a red light image in the red light image sensor through the red light filter.

[0017] Preferably, in the visualization system, the fluorescent image sensor and the red light image sensor are both arranged inside the same device and are in a spatially staggered arrangement.

[0018] Preferably, in the visualization system, the camera controller further comprises an image enhancement unit;

[0019] The image enhancement unit uses nonlinear histogram equalization to perform image enhancement on the fluorescence image and the red light image, and outputs the fluorescence image signal and the red light image signal according to the image enhancement result.

[0020] Preferably, in the visualization system, the image enhancement unit includes a spatial filtering mechanism, which has the function of reducing low spatial frequency signals in the red light image signal and increasing high spatial frequency signals in the red light image signal. In addition, the image enhancement unit also includes an inversion function, which can map a higher signal level at the input end to a lower signal level at the output end, and vice versa.

[0021] Preferably, in the visualization system, the camera controller further comprises a light source controller;

[0022] The light source controller is used to determine whether the amount of light reflected by the imaging light source collected by the camera group is sufficient according to a preset standard, and to control the light source to provide fill light when the amount of light is insufficient.

[0023] Preferably, in the visualization system, the imaging device uses RGB color mode to present the real-time observation image, wherein:

[0024] The RGB red channel is determined by the red light image signal;

[0025] The RGB green channel is determined by the superposition of the red light image signal and the fluorescent image signal;

[0026] The RGB blue channel is determined by the red light image signal.

[0027] Preferably, in the visualization system, the red light emitted by the second light source has a wavelength band of [600nm, 700nm].

[0028] Preferably, in this visualization system, the fluorescence excitation light emitted by the first light source has a wavelength band of [780nm, 820nm], which is used to excite the ICG dye.

[0029] Preferably, in the visualization system, the fluorescence excitation light emitted by the first light source has a wavelength band of [480 nm, 520 nm], which is used to excite the fluorescent dye.

[0030] Preferably, in the visualization system, the receiving end is an endoscope.

[0031] Preferably, when the visualization system is applied to open surgery, the receiving end is an optical coupling system for open surgery.

[0032] The above technical solution has the following advantages or beneficial effects:

[0033] Through this technical solution, only a specific band of red light is used to illuminate the surgical scene at the receiving end to provide the surgeon with sufficient background information and obtain clear and complete real-time images. The overall structure of the visualization system is streamlined, the control method is convenient, the imaging processing process is simple and efficient, and it has a good user experience while having low manufacturing and maintenance costs. It has extremely high applicability and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural schematic diagram of a visualization system with real-time imaging function according to the present invention;

[0035] Figure 2 The present invention discloses a filtering mode in a single-chip RGB infrared imaging system in a visualization system with a real-time imaging function. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0039] In view of the above problems existing in the prior art, a visualization system with real-time imaging function is now provided, and the specific technical solution is as follows:

[0040] A visualization system with real-time imaging capabilities, such as Figure 1 As shown, specifically including:

[0041] A receiving end 1, comprising a camera interface 11 and a light guide interface 12;

[0042] A light source 2, connected to the light guide interface 12 via a light guide 20, for continuously providing an imaging light source for the receiving end 1, the light source 2 comprises a first light source 21 and a second light source 22, the first light source 21 emits fluorescent excitation light, and the second light source 22 emits red light;

[0043] An imaging system 3, connected to the camera interface 11 via a light guide connector 30, for acquiring fluorescence imaging signals and red light imaging signals respectively according to the reflected light of the imaging light source continuously received by the receiving end 1 and outputting a real-time observation image;

[0044] A display device 4 is electrically connected to the imaging system 3 and is used to display real-time observation images.

[0045] As a preferred embodiment, in the visualization system, the receiving end is an endoscope.

[0046] As a preferred embodiment, when this type of visualization system is applied to open surgery, the receiving end is an optical coupling system for open surgery.

[0047] In a preferred embodiment of the present invention, the receiving end in the visualization system can be flexibly configured to become an endoscope with a camera interface and a light guide interface or an optical coupling system for open surgery according to the actual needs of the surgery, and can be flexibly adjusted according to different surgical requirements, thereby enhancing the scope of application of the visualization system.

[0048] As a preferred embodiment, in this visualization system, the imaging system 3 includes a camera group 31 and a camera controller 32;

[0049] The camera group 31 is used to continuously collect the reflected light of the imaging light source and filter it, and output the corresponding fluorescent image and red light image;

[0050] The camera group 31 continuously collects and filters light from the light guide connector 30 through an image sensor array configured to detect fluorescent imaging signals and red imaging signals, and outputs corresponding fluorescent images and red images;

[0051] The camera controller 32 is connected to the camera group 31 via a camera cable, and outputs corresponding fluorescence imaging signals and red light imaging signals according to the fluorescence image and the red light image.

[0052] As a preferred embodiment, in this visualization system, the camera controller 32 is also electrically connected to the light source 2;

[0053] The camera controller 32 generates a corresponding light source instruction according to an external user instruction, and the light source 2 turns on or off the first light source 21 and / or the second light source 22 and / or the third light source 23 according to the light source instruction.

[0054] In another preferred embodiment of the present invention, the light source 2 further includes a third light source 23 emitting white light.

[0055] As a preferred embodiment, in the visualization system, the camera group 31 includes a fluorescent image sensor 312 and a red light image sensor 311 .

[0056] The fluorescence image sensor 312 includes a plurality of sub-image sensors 3121, 3122, and 3123, which may be optionally configured to receive light of different wavelengths, wherein the sub-image sensor 3121 receives blue light, the sub-image sensor 3122 receives green light, and the sub-image sensor 3123 receives near-infrared light;

[0057] The red image sensor 311 is configured to receive red light.

[0058] In another preferred embodiment of the present invention, the camera group 31 in the visualization system includes a fluorescent image sensor and a red light image sensor, wherein the fluorescent image sensor is used to collect the reflected fluorescence generated by the fluorescent excitation light through a filter lens and generate a fluorescent image. Since only the area where the contrast agent is effective will produce fluorescence under the influence of the fluorescent excitation light, the aforementioned fluorescent image represents the area of ​​interest to the doctor; similarly, the red light image sensor is used to collect the red light reflected light through a filter lens and generate a red light image to represent the background area. Since during surgical operations, human tissue can provide obvious reflection within the red light spectrum range, using only red light as the background light source can provide good background information for the surgical environment.

[0059] As a preferred embodiment, in the visualization system, the fluorescent image sensor and the red light image sensor are both arranged inside the same device and are in a spatially staggered arrangement.

[0060] As a preferred embodiment, in this visualization system, the camera controller 32 further includes an image enhancement unit 321;

[0061] The image enhancement unit 321 performs image enhancement on the fluorescence image and the red light image by using nonlinear histogram equalization, and outputs the fluorescence image signal and the red light image signal according to the image enhancement result.

[0062] In another preferred embodiment of the present invention, the obtained fluorescence image and red light image are subjected to image enhancement by nonlinear mapping, which can enhance the image presentation effect and improve the user experience. Furthermore, the image enhancement unit includes a spatial filtering mechanism, which has the ability to reduce low spatial frequency signals in the red signal and increase high spatial frequency signals in the red signal; in addition, the image enhancement unit also includes an inversion function, which can map a higher signal level at the input end to a lower signal level at the output end, and vice versa.

[0063] As a preferred embodiment, in this visualization system, the camera controller 32 further includes a light source controller 322;

[0064] The light source controller 322 is used to determine whether the amount of light reflected by the imaging light source collected by the camera group is sufficient according to a preset standard, and to control the light source to provide fill light when the amount of light is insufficient.

[0065] In another preferred embodiment of the present invention, the camera controller 32 also has the function of automatic fill light, which can determine whether there is a situation of too little light input based on parameters such as brightness, clarity, contrast, etc. of the returned image, and control the light source 2 to perform a secondary fill light operation when the situation of too little light input occurs.

[0066] As a preferred embodiment, in this visualization system, the imaging device 4 uses RGB color mode to present the real-time observation image, wherein:

[0067] The RGB red channel is determined by the red light image signal;

[0068] The RGB green channel is determined by the superposition of the red light image signal and the fluorescent image signal;

[0069] The RGB blue channel is determined by the red light image signal.

[0070] In another preferred embodiment of the present invention, when the observed image needs to be presented in real time, the RGB red channel and the blue channel are both directly presented in a unified manner using the red light image signal, while the green channel is presented by simply superimposing the red light image signal and the fluorescent image signal, and in the final presentation process, the background areas corresponding to the red channel and the blue channel are grayed out to better highlight the green area for surgeons to perform more precise surgical operations.

[0071] As a preferred embodiment, in this visualization system, the red light emitted by the second light source 22 has a wavelength band of [600nm, 700nm].

[0072] In another preferred embodiment of the present invention, the visualization system proposed by the present invention can also be used in a four-chip RGB system to achieve better imaging effects, wherein the four-chip RGB system includes a red light image sensor 311, a green light image sensor 3121, a blue light image sensor 3122 and a near-infrared light image sensor 3123, and the light sources used at this time are a fluorescent excitation light source 21, a red light source 22 and a white light source 23, and the wavelength of the red light source 22 is set to be mainly concentrated in the range of [600nm, 700nm]. Since the wavelength of the commonly used white light source is mainly concentrated in the range of [400nm, 700nm] and decays rapidly after 600nm, the red light wavelength of the red light source 22 is required to optically complement the white light source 23. The advantage of this is that human tissue or blood has very important information for doctors in the red light spectrum, such as blood vessel location, etc., which can provide a good background environment for surgeons when performing surgical procedures such as resection.

[0073] In another preferred embodiment of the present invention, the visualization system provided by the present invention is also suitable for use in a single-chip RGB infrared imaging system to obtain better imaging effects: Specifically, when applied to a single-chip RGB infrared imaging system, the visualization system includes a CMOS image sensor having the following features: Figure 2 The exemplary filtering pattern shown has a light source similar to that previously described.

[0074] As a preferred embodiment, in this visualization system, the fluorescence excitation light emitted by the first light source 21 has a wavelength band of [800 nm, 820 nm].

[0075] In another preferred embodiment of the present invention, the wavelength band of the fluorescence excitation light emitted by the first light source 21 is limited to [800nm, 820nm]. In order to better collect the fluorescence excited by the fluorescence excitation light, the wavelength band of the fluorescence received by the corresponding fluorescence image sensor is limited to [820nm, 840nm] to capture the fluorescence image.

[0076] As a preferred embodiment, in the visualization system, the fluorescence excitation light emitted by the first light source 21 has a wavelength band of [480 nm, 500 nm] to excite a specific contrast agent, such as fluorescein.

[0077] In another preferred embodiment of the present invention, the wavelength band of the fluorescence excitation light emitted by the first light source 21 is limited to [480nm, 500nm]. In order to better collect the fluorescence excited by the fluorescence excitation light, the wavelength band of the fluorescence received by the corresponding fluorescence image sensor is limited to [500nm, 520nm] to capture the fluorescence image.

[0078] A specific embodiment is now provided to further illustrate and explain the technical solution:

[0079] In a specific embodiment of the present invention, the camera controller and the display device are integrated into a personal computer with a display screen, and the personal computer is connected to the camera group and the light source. Before the operation, the surgeon sends a start command to the light source through the relevant program in the personal computer. The red light source and the fluorescent excitation light source are turned on at the same time. The surgical area is irradiated through the coupling of the light guide through the receiving end. At this time, the surgical object area will be excited to emit a fluorescent signal, while other background areas will reflect red light. The camera group collects the reflected light and performs corresponding filtering through the filter, and outputs the fluorescent image and the red light image to the camera controller. The camera controller will judge whether the light input is too low according to a pre-set light input comparison table and the brightness, clarity, contrast and other parameters of the returned image. If it exists, the light source is controlled to perform secondary fill light. If not, the image is enhanced based on nonlinear mapping, and then the processed image is displayed in real time on the display screen. The displayed image is first processed through the RGB color mode. The red channel and the blue channel are directly determined by the red light image signal and are the same, and together constitute the background. The green channel is determined by the superposition of the red light image signal and the fluorescent image signal, and constitutes the surgical target. Then, the background part is grayed to remove the color, so as to obtain a real-time image with better contrast, which helps the surgeon to perform the operation smoothly.

[0080] In summary, through this technical solution, only red light of a specific band is used to illuminate the surgical scene to provide surgeons with sufficient background information and obtain clear and complete real-time images. The overall structure of the visualization system is streamlined, the control method is convenient, the imaging processing process is simple and efficient, and it has a good user experience while having low manufacturing and maintenance costs. It has extremely high applicability and scalability.

[0081] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A visualization system with real-time imaging function, characterized in that: Specifically include: A receiving end, including a camera interface and a light guide interface; a light source connected to the light guide interface via a light guide, and used to continuously provide an imaging light source for the receiving end, wherein the light source comprises a first light source and a second light source, wherein the first light source emits fluorescent excitation light, and the second light source emits red light; An imaging system, connected to the camera interface via a light guide connector, for acquiring fluorescence imaging signals and red light imaging signals respectively according to the reflected light of the imaging light source continuously received by the receiving end, and outputting a real-time observation image; The imaging system includes a camera group, which is used to continuously collect the reflected light of the imaging light source and filter it, and output corresponding fluorescent images and red light images; The camera assembly includes a fluorescent image sensor and a red light image sensor; The fluorescence image sensor comprises a first sub-image sensor, a second sub-image sensor and a third sub-image sensor, which are optionally configured to receive light of different wavelengths, wherein the first sub-image sensor is used to receive blue light, the second sub-image sensor is used to receive green light, and the third sub-image sensor is used to receive near-infrared light; The red light image sensor is configured to receive red light; The red light image sensor receives the red light through a filter lens; A display device is electrically connected to the imaging system and is used to display the real-time observation image.

2. The visualization system according to claim 1, characterized in that The imaging system includes a camera controller; The camera controller is connected to the camera group via a camera cable, and outputs the corresponding fluorescence imaging signal and the red light imaging signal according to the fluorescence image and the red light image.

3. The visualization system according to claim 2, characterized in that The camera controller is also electrically connected to the light source, and the light source further includes a third light source, and the third light source emits white light; The camera controller generates a corresponding light source instruction according to an external user instruction, and the light source turns on or off the first light source and / or the second light source and / or the third light source according to the light source instruction.

4. The visualization system according to claim 1, characterized in that The fluorescent image sensor and the red light image sensor are both arranged inside the same device and are in a spatially staggered arrangement.

5. The visualization system according to claim 2, characterized in that: The camera controller also includes an image enhancement unit; The image enhancement unit performs image enhancement on the fluorescence image and the red light image by using nonlinear histogram equalization, and outputs the fluorescence imaging signal and the red light imaging signal according to the image enhancement result.

6. The visualization system according to claim 5, characterized in that The image enhancement unit includes a spatial filtering mechanism, which has the function of reducing low spatial frequency signals in the red light imaging signal and increasing high spatial frequency signals in the red light imaging signal.

7. The visualization system according to claim 3, characterized in that The camera controller also includes a light source controller; The light source controller is used to determine whether the amount of light reflected by the imaging light source collected by the camera group is sufficient according to a preset standard, and to control the light source to perform fill light when the amount of light is insufficient.

8. The visualization system according to claim 1 or 5, characterized in that: The imaging system uses RGB color mode to present the real-time observation image, wherein: The RGB red channel is determined by the red light imaging signal; The RGB green channel is determined by the superposition of the red light imaging signal and the fluorescence imaging signal; The RGB blue channel is determined by the red light imaging signal.

9. The visualization system according to claim 1, characterized in that: The red light wavelength band emitted by the second light source is [600nm, 700nm].

10. The visualization system according to claim 1, characterized in that The fluorescence excitation light wavelength band emitted by the first light source is [780nm, 820nm], which is used to excite ICG dye.

11. The visualization system according to claim 1, characterized in that: The fluorescence excitation light emitted by the first light source has a wavelength band of [480nm, 520nm], which is used to excite fluorescent dyes.

12. The visualization system according to claim 1, characterized in that The receiving end is an endoscope.

13. The visualization system according to claim 1, characterized in that When the visualization system is applied to open surgery, the receiving end is an optical coupling system for open surgery.

Images