Exterior mirror system and method for fluorescence imaging

Abstract

The invention relates to an exterior mirror system and method for fluorescence imaging, and relates to the technical field of image processing, the system can detect fluorescence under a white light operation condition in a tumor excision process, and imaging in a dark or dim operating room is not needed; white light in the fluorescent light is separated, so that a clear fluorescent light image is presented; in addition, fluorescence images can be displayed in a 3D mode, and development is more visual. According to the invention, the technical problems of dark fluorescence image, poor definition and complicated operation in the developing process in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an exoscopic system and method for fluorescence imaging. Background Art

[0002] In neurosurgery, the fluorescence imaging of malignant gliomas uses 5-aminolevulinic acid (5-ALA). 5-ALA is a metabolite of the hemoglobin biosynthesis pathway in the body. After being taken up by malignant glioma cells, it is converted into a photosensitizer, protoporphyrin. The photosensitivity of protoporphyrin is stimulated under the fluorescence mode of the intraoperative microscope to produce a pink fluorescence effect, which replaces the past microscopic identification of tumor boundaries and is widely used in surgical operations for malignant gliomas.

[0003] In addition to 5-ALA fluorescence imaging technology, sodium fluorescein (yellow fluorescence) auxiliary technology under the microscope can also help depict the morphology of the tumor, determine the location and range of the tumor tissue, and guide doctors to complete the resection of the tumor tissue.

[0004] The currently used 5-ALA fluorescence imaging technology and sodium fluorescein technology are limited by the fact that the excitation light and fluorescence emission light are within the white light band range and overlap with the white light band, so it is impossible to separate the white light image and the fluorescence image by filtering, and it is also impossible to collect and display the white light image and the fluorescence image at the same time. That is, the current fluorescence imaging exoscopic system can only view the fluorescence image alone, resulting in dark fluorescence images (low brightness) and poor clarity;

[0005] During surgery, the ambient light is turned off to reduce interference from ambient lighting in the operating room and white light sources, which is cumbersome and prolongs the surgery time.

[0006] Fluorescence images appear as two-dimensional (2D) images, lacking a three-dimensional sense when observed;

[0007] In addition, white light images and fluorescent images need to be switched during the operation. Resection, suturing, and observation need to be performed under white light images. When the tumor boundary needs to be determined, the fluorescent image is switched to. When the existing technology uses this fluorescent function, the operation is cumbersome and even requires the replacement of equipment or modules.

[0008] Based on this, there are problems in fluorescence imaging technology, such as dark fluorescence images, poor clarity, and complex development process operations. Summary of the invention

[0009] The embodiments of the present application provide an exoscopic system and method for fluorescence imaging to solve the technical problems in the prior art of dark fluorescence images, poor clarity, and complex development process operations.

[0010] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0011] In a first aspect, an embodiment of the present application provides an exoscopic system for fluorescence imaging, the exoscopic system comprising:

[0012] a light source generator configured to irradiate white light and excitation light toward a target area;

[0013] A filter switcher is disposed on the imaging optical path and is configured to allow light with a wavelength within the white light wavelength range to pass through after switching to the first white light filter, and to allow light with a wavelength within the fluorescence wavelength range to pass through after switching to the fluorescence filter;

[0014] A first lens group and a second lens group, wherein the first lens group is configured to collect light in the optical path where the filter switch is located and focus it into the first image sensor, and the second lens group is configured to collect light in the optical path where the second white light filter is located and focus it into the second image sensor;

[0015] The first image sensor is configured to collect first white light data of the optical path where the filter switch is located when the filter switch is switched to the first white light filter; collect fluorescence environmental noise when the filter switch is switched to the fluorescence filter and the excitation light is turned off; and collect fluorescence raw data when the filter switch is switched to the fluorescence filter and the excitation light is turned on;

[0016] A second image sensor is configured to collect second white light data of the light path where the second white light filter is located;

[0017] The image processor is configured to generate a 3D white light image based on the first white light data and the second white light data in white light mode; to remove the fluorescence environment noise from the fluorescence raw data to obtain effective fluorescence data, and to generate a 2D fluorescence image based on the effective fluorescence data in fluorescence mode; and to fuse the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image.

[0018] In combination with the first aspect, in one possible design, the white light is used to illuminate the target area, and the excitation light is used to induce fluorescent substances in the tumor tissue of the target area to emit fluorescence, and the fluorescent substances include protoporphyrin induced by 5-aminolevulinic acid applied to tumor tissue cells, or fluorescent sodium applied to tumor tissue cells.

[0019] In combination with the first aspect, in a possible design, when the fluorescent substance is protoporphyrin, the peak value of the excitation light is in the range of 380nm to 420nm, and the peak value of the fluorescence emission light of the protoporphyrin is in the range of 600nm to 700nm.

[0020] In combination with the first aspect, in a possible design, when the fluorescent substance is the fluorescein, the peak value of the excitation light is in the range of 460nm to 490nm, and the peak value of the fluorescence emission light of the fluorescein is in the range of 510nm to 530nm.

[0021] The wavelength band of the white light is between 400nm and 700nm, and the excitation light and the fluorescent emission light are within the wavelength band of the white light.

[0022] In combination with the first aspect, in a possible design, the filter switch is further configured to be controlled by a motor and to switch the first white light filter or the fluorescent filter to the optical path through gear transmission.

[0023] In combination with the first aspect, in a possible design, removing the fluorescence environment noise from the fluorescence raw data to obtain effective fluorescence data includes:

[0024] A pixel difference operation is performed on the fluorescence raw data and the fluorescence environmental noise to obtain effective fluorescence data.

[0025] In combination with the first aspect, in a possible design, fusing the 2D fluorescent image with the 2D white light image to generate a 3D fluorescent image includes:

[0026] After the 2D fluorescence image and the 2D white light image are stereoscopically corrected, the odd-numbered rows of pixels of the stereoscopically corrected 2D fluorescence image and the even-numbered rows of pixels of the stereoscopically corrected 2D white light image are alternately output row by row to generate a 3D fluorescence image.

[0027] In combination with the first aspect, in a possible design, the image processor is further configured to superimpose a pseudo color on the pixel value in the effective fluorescence data, wherein the color intensity of the pseudo color is positively correlated with the size of the pixel value in the effective fluorescence data.

[0028] In combination with the first aspect, in a possible design, the exterior mirror system further includes a display screen configured to be used with 3D polarized glasses, the display screen displaying a 3D fluorescent image;

[0029] The left lens of the 3D polarized glasses assists the left eye in receiving the 2D fluorescent image, and the right lens of the 3D polarized glasses assists the right eye in receiving the 2D white light image, wherein the 2D white light image and the 2D fluorescent image are received simultaneously, and the 2D white light image is used to provide clarity, brightness, and tissue color information, and the 2D fluorescent image is used to provide tumor tissue fluorescence information.

[0030] The brightness of the 2D white light image is greater than the brightness of the 2D fluorescent image, and the clarity of the 2D white light image is greater than the clarity of the 2D fluorescent image.

[0031] In a second aspect, an embodiment of the present application provides a fluorescence imaging method, comprising:

[0032] In the white light mode, white light is irradiated to the target area through the light source generator;

[0033] Controlling the filter switcher located on the imaging optical path to switch to the first white light filter, collecting the light in the optical path where the filter switcher is located through the first lens group, focusing it into the first image sensor, and collecting first white light data through the first image sensor; at the same time, collecting the light in the optical path where the second white light filter is located through the second lens group, focusing it into the second image sensor, and collecting second white light data through the second image sensor;

[0034] Converting the first white light data and the second white light data into a 3D white light image through an image processor;

[0035] Controlling the filter switcher to switch to the fluorescent filter, and collecting fluorescent environmental noise through the first image sensor;

[0036] After the tumor tissue cells in the target area contain fluorescent substances, in the fluorescence mode, the excitation light is irradiated to the target area by the light source generator;

[0037] Collecting raw fluorescence data through the first image sensor;

[0038] Eliminating the fluorescence environmental noise from the fluorescence raw data by the image processor to obtain effective fluorescence data, and generating a 2D fluorescence image based on the effective fluorescence data;

[0039] The 2D fluorescence image is fused with the 2D white light image by the image processor to generate a 3D fluorescence image.

[0040] In a third aspect, an embodiment of the present application provides a terminal, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method of the second aspect and possible design methods thereof.

[0041] In a fourth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the method of the second aspect and its possible design manner when running.

[0042] Compared with the prior art, an exoscopic system and method for fluorescence imaging provided in an embodiment of the present application, in the exoscopic system: a light source generator is configured to irradiate white light and excitation light to a target area; a filter switch is arranged on an imaging optical path, and is configured to allow light with a wavelength within the white light wavelength range to pass through after switching to a first white light filter, and to allow light with a wavelength within the fluorescence wavelength range to pass through after switching to a fluorescence filter; a first lens group and a second lens group, the first lens group being configured to collect light in the optical path where the filter switch is located, and focus it on a first image sensor, and the second lens group being configured to collect light in the optical path where the second white light filter is located, and focus it on a second image sensor; the first image sensor is configured to be switched to the first white light filter when the filter switch is switched to the first white light filter. The first white light data of the optical path where the filter switch is located is collected; when the filter switch is switched to the fluorescent filter and the excitation light is turned off, the fluorescent environmental noise is collected; and when the filter switch is switched to the fluorescent filter and the excitation light is turned on, the fluorescent raw data is collected; the second image sensor is configured to collect the second white light data of the optical path where the second white light filter is located; the image processor is configured to generate a 3D white light image based on the first white light data and the second white light data in the white light mode; in the fluorescence mode, the fluorescent environmental noise is removed from the fluorescence raw data to obtain effective fluorescence data, and a 2D fluorescence image is generated based on the effective fluorescence data; and the 2D fluorescence image is fused with the 2D white light image to generate a 3D fluorescence image. The present application improves the clarity of visual effects through 3D imaging, and realizes the simultaneous output of 3D fluorescence images and 3D white light images through differentiated filter parameter settings for the left and right eyes, filter switches, and excitation light switching, so that the images viewed by doctors have both the brightness and clarity of white light images and tumor fluorescence information, with better visual effects. In addition, when displaying fluorescent images, there is no need to turn off the operating room lights and equipment lighting sources, which not only provides surgical brightness but also simplifies the operation.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1A The excitation light and emission light spectra of a 5-ALA fluorescence provided in an embodiment of the present application are shown;

[0046] Figure 1BThe excitation light and emission light spectra of a fluorescent sodium element provided in an embodiment of the present application are shown;

[0047] Figure 2 A schematic structural diagram of a fluorescence imaging exoscopic system provided in an embodiment of the present application is shown;

[0048] Figure 3 A schematic diagram of a light source system provided by an embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of a filter switch provided in an embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of a denoising method provided in an embodiment of the present application is shown;

[0051] Figure 6 A flow chart of a fluorescence imaging method provided in an embodiment of the present application is shown;

[0052] Figure 7 A schematic structural diagram of an exterior mirror system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] Complete resection of gliomas often enables patients to obtain a better prognosis and a longer progression-free survival period, and the effects of postoperative radiotherapy and chemotherapy are also significantly enhanced. However, malignant gliomas are highly invasive, and they are often unclearly demarcated from the surrounding normal brain tissue, making it difficult to "see" the whole picture under a microscope, which causes great trouble to surgeons, because incomplete resection will result in a higher recurrence rate. The application of 5-ALA fluorescence technology has broken this limitation. Specifically, malignant gliomas usually destroy the normal blood-brain barrier, and tumor tissues will take up a large amount of 5-ALA and convert it into protoporphyrin with a fluorescent effect. The strong pink tumor itself and the light pink tumor boundary can be observed through the microscope fluorescence mode. In this way, surgeons can distinguish normal tissue from tumor tissue, and the resection rate of malignant tumors is greatly increased. The damage to normal tissues is greatly reduced, and the resection of malignant gliomas becomes faster, safer, and more effective.

[0056] Figure 1A The excitation light and emission light spectra of a 5-ALA fluorescence provided in the embodiment of the present application are shown as follows: Figure 1AAs shown, the solid line represents the excitation light and the dotted line represents the emission light. It can be seen that when the peak light intensity of the excitation light is in the range of 380nm to 420nm and the peak light intensity of the emission light is in the range of 600nm to 700nm, it means that the 5-ALA fluorescence imaging effect is good in this band. Since the white light band is between 400nm and 700nm, the fluorescence emission light overlaps with the white light band, so it is impossible to separate the white light and fluorescence by filtering, and it is impossible to collect and display the white light image and the fluorescence image at the same time. In addition, the white light plays a role in increasing the brightness. When the fluorescence image is viewed alone, it will cause the fluorescence image to be dark and have poor clarity.

[0057] In addition to 5-ALA fluorescence imaging technology, sodium fluorescein (yellow fluorescence) assisted technology under a microscope can also help depict the morphology of the tumor and determine the location and range of the tumor tissue. Especially for tumors with unclear boundaries such as gliomas, or involving important brain functional areas, sodium fluorescein can help to remove the tumor to the greatest extent, safely and reliably.

[0058] Figure 1B The excitation light and emission light spectra of a sodium fluorescein provided in an embodiment of the present application are shown in FIG. Figure 1B As shown, the peak light intensity of the excitation light is in the range of 460nm to 490nm, and the peak light intensity of the fluorescence emission light of fluorescein is in the range of 510nm to 530nm. The white light band is in the range of 400nm to 700nm, and the fluorescence emission light of fluorescein overlaps with the white light band, so it is impossible to separate the white light and fluorescence by filtering, and it is also impossible to collect and display the white light image and the fluorescence image at the same time. In addition, white light plays a role in increasing the brightness. When the fluorescence image is viewed alone without the assistance of the white light image, the fluorescence image will be dark and have poor clarity.

[0059] In order to improve the contrast of the fluorescent area of ​​the image, the existing technology uses filters to cut off a part of the white light band, such as cutting off the white light of about 400nm to 550nm when using 5-ALA fluorescence imaging. In addition, in order to reduce the influence of the spectrum in nature on the results, the ambient light is usually turned off during surgery, which improves the contrast of the fluorescent area, but also sacrifices the brightness of the image, which brings great inconvenience to observation. At the same time, turning on and off the ambient light during surgery is cumbersome and prolongs the operation time.

[0060] The fluorescent image obtained in the prior art is presented as a 2D image, which lacks a three-dimensional sense when observed.

[0061] In addition, it is necessary to switch between white light images and fluorescent images during the operation, wherein excision, suturing, and observation need to be performed under white light images, and when it is necessary to determine the tumor boundary, it is necessary to switch to fluorescent images. It can be seen that when the existing technology uses the fluorescent function, the operation is cumbersome and the equipment or module needs to be replaced.

[0062] As can be seen from the above, the fluorescence images displayed by the current 5-ALA fluorescence technology or sodium fluorescein technology have low brightness, can only be viewed alone, and are 2D images, resulting in poor visual effects; during the operation, it is necessary to switch back and forth between white light images and fluorescence images, turn off the lights in the operating room, and even replace equipment or modules, which is cumbersome.

[0063] In view of this, an embodiment of the present application provides a fluorescence imaging exoscopic system that can detect fluorescence under white light operating conditions (i.e., without the need for a dark or dim operating room) during tumor resection; and can separate the white light in the fluorescence to present a clear fluorescence image; in addition, the fluorescence image can also be displayed in 3D, making the imaging more intuitive.

[0064] It should be noted that although the present application specifically discusses fluorescence imaging for the purpose of tumor removal, the teachings of the present application can be extended to other types of surgical procedures, used to detect other types of tissues, and to detect other types of fluorophores, wherein the technology provided by the present application can have better visual effects for imaging fluorophores whose reflected light wavelengths are within the white light range.

[0065] Figure 2 FIG. 4 shows a schematic structural diagram of a fluorescent imaging exoscopic system provided in an embodiment of the present application. Figure 2As shown, the exterior mirror system includes: a light source generator 21, a filter switcher 22, two groups of lenses, a first image sensor 23, a second image sensor 24, and an image processor 25. Among them, the light source generator 21 is configured to irradiate white light and excitation light to the target area; the filter switcher 22 is arranged on the imaging optical path, and is configured to allow light with a wavelength within the white light wavelength range to pass through after switching to the first white light filter, and to allow light with a wavelength within the fluorescence wavelength range to pass through after switching to the fluorescence filter; the first lens group 26 is configured to collect the light in the optical path where the filter switcher 22 is located, and focus it on the first image sensor 23, and the second lens group 27 is configured to collect the light in the optical path where the second white light filter 28 is located, and focus it on the second image sensor 24; the first image sensor 23 is configured to collect the light in the optical path where the filter switcher 22 is located when the filter switcher 22 is switched to the first white light filter. The first white light data of the optical path of the second white light filter 28 is collected; when the filter switch 22 is switched to the fluorescent filter and the excitation light is turned off, the fluorescent environmental noise is collected; and when the filter switch 22 is switched to the fluorescent filter and the excitation light is turned on, the fluorescent raw data is collected; the second image sensor 24 is configured to collect the second white light data of the optical path where the second white light filter 28 is located; the image processor 25 is configured to generate a 3D white light image based on the first white light data and the second white light data in the white light mode; in the fluorescence mode, the fluorescent environmental noise is removed from the fluorescent raw data to obtain effective fluorescent data, and a 2D fluorescent image is generated based on the effective fluorescent data; and the 2D fluorescent image is fused with the 2D white light image to generate a 3D fluorescent image.

[0066] In this embodiment, the light source generator 21 can emit two kinds of light, namely white light and excitation light. The white light is used to illuminate the target area, and the excitation light induces the fluorescent substance in the tumor tissue of the target area to emit fluorescence, and the fluorescent substance includes protoporphyrin induced by 5-ALA applied to tumor tissue cells, or fluorescent sodium applied to tumor tissue cells. The light source generator 21 controls the two light sources through the illumination light path combination module, and outputs the two light sources through one interface, and the specific reference is made to the following description.

[0067] Figure 3 A schematic diagram of a light source system provided in an embodiment of the present application is shown. Figure 3As shown, the light source system (i.e., the light source generator 21) includes a communication control interface, a core control module (ARM), a white light LED driving circuit, an excitation light LED driving circuit, a white light LED, an excitation light LED, an illumination optical path combination module, and a light output interface. The light source system receives control instructions through the communication control interface to realize the function of switching the light source circuit. Specifically, during routine surgical operations, a control instruction to turn on the white light LED driving circuit is sent to the light source system, and the light source system emits white light; when tumor fluorescence imaging is required for observation, a control instruction to turn on the excitation light LED driving circuit is sent to the light source system, and the light source system emits excitation light. The excitation light uses LED as a light source, and the peak of the excitation light can be selected in the interval that can excite the emission light, and this application does not limit this.

[0068] As an example, the light source system can be connected to the touch control screen through the communication control interface, so that the medical professional can switch the light source by operating the touch control screen. In addition, the above control instructions can also be issued by voice, gesture, remote control, etc., and this application does not limit this.

[0069] In some embodiments, the light source emitted by the light source generator 21 is a point light source, and its light spot is circular or elliptical, and the point light source is emitted from the lens group and illuminates the observed part. In this way, reflection and glare can be reduced, and the lighting is concentrated, so that the captured image is brighter and the visual effect is better.

[0070] Figure 4 A schematic diagram of a filter switch provided in an embodiment of the present application is shown. Figure 4 As shown, the filter switch 22 includes a first white light filter 41, a fluorescent filter 42, a gear and transmission device 43 and a motor 44. The filter switch 22 controls the transmission of the gear and transmission device 43 through the motor 44, thereby switching the first white light filter 41 or the fluorescent filter 42 to the optical path. Specifically, the first white light filter 41 and the fluorescent filter 42 are connected, and after receiving the command, the filter switch 22 sends a control signal to drive the motor to rotate, and the motor drives the gear and transmission device 43 to transmit, thereby simultaneously moving the first white light filter 41 and the fluorescent filter 42 to complete the filter switching. In some embodiments, the filter switch 22 only exists in one optical path, such as the optical path corresponding to the left eye, or the optical path corresponding to the right eye, wherein the optical path corresponding to the left eye refers to the image generated by the optical path that the left eye can observe, and the optical path corresponding to the right eye refers to the image generated by the optical path that the right eye can observe. It can be understood that if the filter switch 22 is in one optical path, then the second white light filter 28 is in another optical path. For example, if the filter switch 22 is in the optical path corresponding to the left eye, then the second white light filter 28 is in the optical path corresponding to the right eye, and vice versa. The second white light filter 28 has the same light transmission parameters as the first white light filter in the filter switch 22.

[0071] As an example, the first white light filter allows light with a wavelength within the wavelength range of white light to pass through, wherein the first white light filter can use a high pass of 400nm to 700nm. If the high pass of 400nm is used, the first white light filter allows light with a wavelength above 400nm to pass through, while light shorter than 400nm is cut off. The wavelength of light that can pass through the first white light filter of 500nm and 700nm can refer to the introduction of the high pass of 400nm, and no redundant explanation is given here. The fluorescent filter allows light with a wavelength within the fluorescent wavelength range to pass through. When the fluorescent substance is protoporphyrin, it can use a high pass of 550nm to 750nm. When the fluorescent substance is natrium fluorescens, it can use a high pass of 500nm to 550nm. The above examples of filter parameters are only exemplary and are not intended to limit the parameters. The actual filter parameters can float on this basis, subject to the realization of the functions of this application.

[0072] The first image sensor 23 is also configured to be on the optical path where the filter switch 22 is located, just like the first lens group 26, so as to collect the light focused by the first lens group 26. The filter switch 22 will switch between the first white light filter and the fluorescent filter modes. The light source generator 21 irradiating white light to the target area is referred to as the white light mode, and the light source generator 21 irradiating white light and excitation light to the target area is referred to as the fluorescent mode. The data collected by the first image sensor 23 will be described. In the white light mode, the filter switch 22 switches to the first white light filter, and the first image sensor 23 collects the first white light data to present a 2D white light image; in the white light mode, the filter switch 22 switches to the fluorescent filter, and the first image sensor 23 collects the fluorescent environment noise; in the fluorescent mode, the filter switch 22 switches to the fluorescent filter, and the first image sensor 23 collects the fluorescent raw data. In the fluorescent mode, the fluorescent environment noise is used to denoise the fluorescent raw data to generate effective fluorescent data to present a 2D fluorescent image. The fluorescence raw data represents the reflected light (or fluorescence) emitted by the protoporphyrin in the target area when it is excited.

[0073] In some of the embodiments, the white light mode and the fluorescence mode can be displayed on the control touch screen. When the white light mode is pressed, the control touch screen detects the trigger operation of the white light mode control, and then generates a control instruction to control the light source generator 21 to emit white light, and controls the filter switch 22 to switch to the first white light filter; when the fluorescence mode is pressed, the control touch screen detects the trigger operation of the fluorescence mode control, and then generates a control instruction to control the light source generator 21 to emit excitation light and white light, and controls the filter switch 22 to switch to the fluorescence filter; the operation can be simplified through this linkage control.

[0074] The second image sensor 24 is also configured on the optical path of the second white light filter 28, just like the second lens group 27, so as to collect the light focused by the second lens group 27 and obtain the second white light data to present a 2D white light image. In the white light mode, the 2D white light image generated by the first white light data can be fused with the 2D white light image generated by the second white light data to generate a 3D white light image. The 3D white light image has depth information, which helps doctors confirm the location of tissues. In the fluorescence mode, the 2D white light image generated by the second white light data can also be fused with the 2D fluorescence image to generate a 3D fluorescence image. The 3D fluorescence image provides depth information of the tumor site, and the visual effect is clear.

[0075] It is worth mentioning that after wearing 3D polarized glasses, the doctor's left eye receives the fluorescent image and the right eye receives the white light image. The human brain's own stereoscopic fusion function is used to process and fuse these two images with parallax, thereby generating a sense of stereo and depth. Since the brightness of the 2D white light image is greater than the brightness of the 2D fluorescent image, and the clarity of the 2D white light image is greater than the clarity of the 2D fluorescent image, the human eye receives the image at the same time, and can have the brightness and clarity of the right eye white light image and the left eye tumor fluorescence information at the same time. There is no relative time interval between the acquisition of the white light image and the fluorescent image, and there is no need to reduce the original sampling frame rate of the fluorescent image / white light image, that is, while ensuring that the fluorescent image and the white light image are sampled, processed, transmitted, and displayed at the same time, the brightness and clarity of the tumor fluorescence image are improved. And because it is not necessary to turn off the ambient light when acquiring the fluorescent image, but to remove the influence of white light by denoising, the operation is convenient.

[0076] Regarding the method of generating 3D images, the 3D image can be obtained by alternately outputting the odd and even lines of the two images line by line. Specifically, the image sensors of the left and right eyes are used for image acquisition, the analog signals are converted into digital signals, the cables are used for digital signal transmission, the image processor 25 is used for image signal processing, and the 3D video format line by line is output after stereo correction, that is, the odd lines of the left eye image and the even lines of the right eye image are extracted and combined to form a 3D image. According to this implementation case, if a 4K 3D video image is output, the left eye takes lines 1, 3, 5, 7, ..., 2159, and the right eye takes lines 2, 4, 6, 8, ..., 2160; the left eye takes the fluorescent image, and the right eye takes the white light image. If the output image format is 1080P, that is, 1920*1080, the pixels of each line are 1920, then the left eye takes lines 1, 3, 5, 7, ..., 1919, and the right eye takes lines 2, 4, 6, 8, ..., 1920.

[0077] The fluorescence denoising method provided in the embodiment of the present application is further introduced below. Figure 5A schematic diagram of a denoising method provided in an embodiment of the present application is shown, Figure 5 As shown, the pixel value of each pixel represented by the fluorescence environmental noise (one is derived from the fluorescence observation band energy contained in the operating room environmental lighting and other equipment in the operating room; the other is derived from the fluorescence observation band energy contained in the equipment white light illumination source band) is represented by a1, a2, ..., nn, the pixel value of each pixel represented by the fluorescence raw data is represented by a1', a2', ..., nn', and the pixel value of each pixel of the fluorescence effective data is represented by a1", a2", ..., nn", wherein a1" = a1'-a1, a" = a2'-a2, and so on. Pixel difference operation is performed on the fluorescence raw data and the fluorescence environmental noise to obtain the effective fluorescence data.

[0078] In some of the embodiments, in order to further improve the contrast of the fluorescence image, the image processor 25 is further configured to superimpose a pseudo color on the pixel value in the effective fluorescence data, wherein the color intensity of the pseudo color is positively correlated with the size of the pixel value in the effective fluorescence data.

[0079] Specifically, pseudo-color is superimposed on the fluorescence response area. For example but not limitation, green, purple, etc. can be superimposed, or other colors that meet the above requirements can be superimposed. The effective fluorescence data is calibrated and corresponds to the superimposed pseudo-color intensity to form a fluorescence intensity and pseudo-color intensity comparison table. Different color intensities are superimposed according to the value of the effective fluorescence data. The larger the fluorescence effective data, the stronger the color intensity, and the smaller the color, the weaker the color.

[0080] By adding a color different from that of brain and spinal tissue, the color difference between the tumor area and normal tissue can be increased to facilitate observation of tissue details.

[0081] The above is a description of an exoscopic system for fluorescence imaging provided in an embodiment of the present application. The exoscopic system improves the clarity of visual effects through 3D imaging, and realizes the simultaneous output of fluorescence images and white light images through the setting of differentiated filter parameters for the left and right eyes, the filter switch 22, and the switching of the excitation light, so that the image viewed by the doctor has both the brightness and clarity of the white light image and the tumor fluorescence information, with better visual effects. When displaying the fluorescence image, there is no need to turn off the operating room lights and equipment lighting sources, which not only provides surgical brightness but also simplifies the operation. In addition, through the scheme of superimposing colors, the clarity and brightness of the white light picture are guaranteed, and the fluorescence contrast of the tumor lesion area is improved, so that the tumor and its boundaries can be better identified. In addition, by switching the mode of the control touch screen, the light source generator 21 and the filter switch 22 can be driven to work synchronously, and the operation steps can be reduced by linkage control to achieve automatic switching.

[0082] In conjunction with the exterior mirror system introduced in the above embodiment, the embodiment of the present application also provides a fluorescence imaging method, which is applied to be executed in the exterior mirror system. The method is described below.

[0083] Figure 6 A flow chart of a fluorescence imaging method provided in an embodiment of the present application is shown. Figure 6 As shown, the method includes steps S601 to S608.

[0084] Step S601: In white light mode, white light is irradiated toward a target area through a light source generator.

[0085] Step S602, control the filter switch located on the imaging optical path to switch to the first white light filter, collect the light in the optical path where the filter switch is located through the first lens group, and focus it on the first image sensor, and collect first white light data through the first image sensor; at the same time, collect the light in the optical path where the second white light filter is located through the second lens group, and focus it on the second image sensor, and collect second white light data through the second image sensor.

[0086] Step S603: converting the first white light data and the second white light data into a 3D white light image through an image processor.

[0087] Step S604: Control the filter switcher to switch to the fluorescence filter, and collect the fluorescence environment noise through the first image sensor.

[0088] Step S605: After the tumor tissue cells in the target area contain fluorescent substances, the target area is irradiated with excitation light by a light source generator in a fluorescence mode.

[0089] Step S606: Collecting raw fluorescence data through the first image sensor.

[0090] Step S607: removing the fluorescence environment noise from the fluorescence raw data by an image processor to obtain effective fluorescence data, and generating a 2D fluorescence image based on the effective fluorescence data.

[0091] Step S608: The 2D fluorescence image and the 2D white light image are fused by an image processor to generate a 3D fluorescence image.

[0092] For the description of steps S601 to S608 , reference may be made to the above description of the various components of the exterior mirror system, which will not be elaborated here.

[0093] Figure 7 A schematic diagram of the structure of an exterior mirror system provided in an embodiment of the present application is shown. Figure 7 As shown, the exterior mirror system may include:

[0094] Camera 71: It contains an optical system and an image acquisition module. The optical system adopts a dual-light path variable magnification optical system solution to provide three-dimensional imaging.

[0095] The image acquisition module uses a CMOS sensor for optical imaging and image signal conversion, which is equivalent to the first image sensor 23 and the second image sensor 24 mentioned above.

[0096] The image processor receives the digital signal collected by the CMOS sensor and transmitted to the back end via the cable, performs image processing based on the digital signal, and outputs the video to the 3D monitor through the port for display. The 3D monitor provides a clear and enlarged image.

[0097] Lighting system: LED is used as the light source, and the light beam and lighting optical system are used to provide lighting within the field of view; the light source is located inside the trolley body, and the lighting optical system is located inside the camera.

[0098] Support system: It consists of a movable trolley body 72, a camera arm 73, and a monitor bracket 74; the trolley has multiple locking casters 75 (such as four locking casters) that can be moved and locked. The trolley body 72 is equipped with a balancing device to ensure that the equipment and the arm bracket remain stable when moving. The camera arm 73 is supported by a 6-degree-of-freedom mechanical arm and can move in three dimensions to meet the doctor's needs for operating the instrument during surgery; at the same time, it can carry the monitor 76 to move in multiple directions such as front, back, up, down, left, and right to meet the needs of different observation angles during surgery.

[0099] Electrical installations( Figure 7 (not shown): located inside the support system, providing the electrical energy required for the operation of various parts of the equipment.

[0100] Touch control screen 77: used for the operation control of the entire device, including white balance, light source adjustment, video recording, display mode, photo taking, video recording, arm control, focus, zoom, image adjustment, dual-mirror combination, user management, patient management, file management, settings, etc., wherein the lighting system and the filter switch 22 can be automatically and synchronously controlled by setting the white light mode and the fluorescent mode.

[0101] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0102] In addition, in combination with the method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the 5-aminolevulinic acid fluorescence imaging exoscopic systems in the above embodiments is implemented.

[0103] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the processor in the above method embodiment.

[0104] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0105] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0106] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0107] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A fluorescence imaging exoscopic system, characterized in that: include: a light source generator configured to irradiate white light and excitation light toward a target area; A filter switcher is disposed on the imaging optical path and is configured to allow light with a wavelength within the white light wavelength range to pass through after switching to the first white light filter, and to allow light with a wavelength within the fluorescence wavelength range to pass through after switching to the fluorescence filter; A first lens group and a second lens group, wherein the first lens group is configured to collect light in the optical path where the filter switch is located and focus it into the first image sensor, and the second lens group is configured to collect light in the optical path where the second white light filter is located and focus it into the second image sensor; A first image sensor is configured to collect first white light data of the optical path where the filter switch is located when the filter switch is switched to the first white light filter; When the filter switch is switched to the fluorescence filter and the excitation light is turned off, collecting fluorescence environmental noise; and collecting fluorescence raw data when the filter switch is switched to the fluorescence filter and the excitation light is turned on; A second image sensor is configured to collect second white light data of the light path where the second white light filter is located; an image processor configured to generate a 3D white light image based on the first white light data and the second white light data in a white light mode; In the fluorescence mode, the fluorescence environment noise is removed from the fluorescence raw data to obtain effective fluorescence data, and a 2D fluorescence image is generated based on the effective fluorescence data; and the 2D fluorescence image is fused with the 2D white light image to generate a 3D fluorescence image.

2. The fluorescence imaging exoscopic system according to claim 1, characterized in that: The white light is used to illuminate the target area, and the excitation light is used to induce fluorescent substances in the tumor tissue of the target area to emit fluorescence. The fluorescent substances include protoporphyrin induced by 5-aminolevulinic acid applied to tumor tissue cells, or natriuretic acid applied to tumor tissue cells.

3. The fluorescence imaging exoscopic system according to claim 2, characterized in that: When the fluorescent substance is protoporphyrin, the peak value of the excitation light is within a range of 380 nm to 420 nm, and the peak value of the fluorescence emission light of protoporphyrin is within a range of 600 nm to 700 nm.

4. The fluorescence imaging exoscopic system according to claim 2, characterized in that: In the case where the fluorescent substance is the fluorescein, the peak value of the excitation light is in the range of 460nm to 490nm, and the peak value of the fluorescence emission light of the fluorescein is in the range of 510nm to 530nm.

5. The fluorescence imaging exoscopic system according to claim 1, characterized in that: The step of removing the fluorescence environment noise from the fluorescence raw data to obtain effective fluorescence data includes: A pixel difference operation is performed on the fluorescence raw data and the fluorescence environmental noise to obtain effective fluorescence data.

6. The fluorescence imaging exoscopic system according to claim 1, characterized in that: The step of fusing the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image includes: After the 2D fluorescence image and the 2D white light image are stereoscopically corrected, the odd-numbered rows of pixels of the stereoscopically corrected 2D fluorescence image and the even-numbered rows of pixels of the stereoscopically corrected 2D white light image are alternately output row by row to generate a 3D fluorescence image.

7. The fluorescence imaging exoscopic system according to claim 1, characterized in that: The image processor is further configured to superimpose a pseudo color on the pixel value in the effective fluorescence data, wherein the color intensity of the pseudo color is positively correlated with the size of the pixel value in the effective fluorescence data.

8. The fluorescence imaging exoscopic system according to claim 1, characterized in that: Also includes: A display screen configured to be used with 3D polarized glasses, the display screen displaying 3D fluorescent images; The left lens of the 3D polarized glasses assists the left eye in receiving the 2D fluorescent image, and the right lens of the 3D polarized glasses assists the right eye in receiving the 2D white light image, wherein the 2D white light image and the 2D fluorescent image are received simultaneously, and the 2D white light image is used to provide clarity, brightness, and tissue color information, and the 2D fluorescent image is used to provide tumor tissue fluorescence information.

9. A fluorescence imaging method, characterized in that: The method comprises: In the white light mode, white light is irradiated to the target area through the light source generator; Controlling a filter switcher located on an imaging optical path to switch to a first white light filter, collecting light in the optical path where the filter switcher is located through a first lens group, focusing the light on a first image sensor, and collecting first white light data through the first image sensor; at the same time, collecting light in the optical path where a second white light filter is located through a second lens group, focusing the light on a second image sensor, and collecting second white light data through the second image sensor; Converting the first white light data and the second white light data into a 3D white light image through an image processor; Controlling the filter switch to switch to the fluorescent filter, and collecting fluorescent environmental noise through the first image sensor; After the tumor tissue cells in the target area contain fluorescent substances, in the fluorescence mode, the excitation light is irradiated to the target area by the light source generator; Collecting raw fluorescence data through the first image sensor; Eliminating the fluorescence environmental noise from the fluorescence raw data by the image processor to obtain effective fluorescence data, and generating a 2D fluorescence image based on the effective fluorescence data; The 2D fluorescence image is fused with the 2D white light image by the image processor to generate a 3D fluorescence image.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the fluorescence imaging method of claim 9 when executed.

Images