An exterior mirror system and method of fluorescence imaging

Abstract

The application relates to an external mirror system and method for fluorescence imaging, and relates to the technical field of image processing.The system can detect fluorescence under white light operation conditions in the process of tumor resection, and does not need to image in a dark or dim operating room; white light in fluorescence is separated, and a clear fluorescence image is presented; in addition, the fluorescence image can be displayed in 3D, and the display is more intuitive. Through the application, the technical problems of poor darkness, poor definition and complex development process operation in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an external viewing mirror system and method for fluorescence imaging. Background Technology

[0002] In neurosurgery, 5-aminolevulinic acid (5-ALA) is used for fluorescence imaging of malignant gliomas. 5-ALA is a metabolite of hemoglobin biosynthesis in the body. After being taken up by malignant glioma cells, it is converted into a photosensitizer—protoporphyrin. The photosensitizing effect of protoporphyrin is excited in intraoperative microscope fluorescence mode, producing a pink fluorescence effect. This has replaced the previous method of identifying tumor boundaries under a microscope and is now widely used in the surgical treatment of malignant gliomas.

[0003] In addition to 5-ALA fluorescence imaging, fluorescein sodium (yellow fluorescence) assisted imaging under a microscope can also help depict the morphology of tumors, determine the location and extent of tumor tissue, and guide doctors to complete the removal of tumor tissue.

[0004] Currently used 5-ALA fluorescence imaging technology and sodium fluorescein technology are limited by the fact that the excitation light and fluorescence emission light are in the white light wavelength range, which overlaps with the white light wavelength range. Therefore, it is impossible to separate the white light image and the fluorescence image by filtering, nor can it simultaneously acquire and display the white light image and the fluorescence image. In other words, current fluorescence imaging external viewing systems can only view the fluorescence image separately, resulting in a dark (low brightness) and poor clarity fluorescence image.

[0005] During the surgery, ambient light is turned off to reduce interference from operating room ambient light and white light sources, but the operation is cumbersome and prolongs the operation time.

[0006] Fluorescence images are presented as two-dimensional (2D) images, lacking a sense of depth when observed;

[0007] In addition, the operation requires switching between white light images and fluorescence images. Resection, suturing and observation need to be performed under white light images, and when it is necessary to determine the tumor boundary, it is necessary to switch to fluorescence images. The operation of using the fluorescence function in the current technology is cumbersome and may even require the replacement of equipment or modules.

[0008] As a result, fluorescence imaging technology suffers from problems such as dark fluorescence images, poor clarity, and complex development processes. Summary of the Invention

[0009] This application provides an external viewing system and method for fluorescence imaging to solve the technical problems of dark fluorescence images, poor clarity, and complex development process in the prior art.

[0010] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0011] In a first aspect, embodiments of this application provide a fluorescence imaging external viewing mirror system, the external viewing mirror system comprising:

[0012] The light source generator is configured to illuminate the target area with white light and excitation light;

[0013] A filter switcher, located in the imaging optical path, is configured to allow light with wavelengths within the white light wavelength range to pass through after switching to the first white light filter, and to allow light with wavelengths within the fluorescence wavelength range to pass through after switching to the fluorescence filter.

[0014] A first lens group and a second lens group, wherein the first lens group is configured to collect light from the optical path of the filter switch and focus it into a first image sensor, and the second lens group is configured to collect light from the optical path of the second white light filter and focus it into a second image sensor.

[0015] A first image sensor is configured to: acquire first white light data in the optical path where the filter switch is located when the filter switch is switched to the first white light filter; acquire fluorescence ambient noise when the filter switch is switched to the fluorescence filter and the excitation light is off; and acquire raw fluorescence data when the filter switch is switched to the fluorescence filter and the excitation light is on.

[0016] The second image sensor is configured to acquire second white light data in the optical path where the second white light filter is located;

[0017] An image processor is configured to generate a 3D white light image based on the first white light data and the second white light data in white light mode; to remove the fluorescence environmental noise from the original fluorescence data to obtain effective fluorescence data in fluorescence mode, and to generate a 2D fluorescence image based on the effective fluorescence data; and to fuse the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image.

[0018] In conjunction with the first aspect, in one possible design, the white light is used to illuminate the target area, and the excitation light is used to induce fluorescence emission from a fluorescent substance in the tumor tissue of the target area, the fluorescent substance comprising protoporphyrin induced by 5-aminolevulinic acid applied to tumor tissue cells, or fluorescein applied to tumor tissue cells.

[0019] In conjunction with the first aspect, in one possible design, when the fluorescent substance is the protoporphyrin, the peak value of the excitation light is in the range of 380 nm to 420 nm, and the peak value of the fluorescence emission light of the protoporphyrin is in the range of 600 nm to 700 nm.

[0020] In conjunction with the first aspect, in one possible design, when the fluorescent material is the sodium fluorescein, the peak value of the excitation light is in the range of 460 nm to 490 nm, and the peak value of the fluorescence emission light of the sodium fluorescein is in the range of 510 nm to 530 nm.

[0021] The wavelength of the white light is between 400nm and 700nm, and the excitation light and the fluorescence emission light are within the wavelength range of the white light.

[0022] In conjunction with the first aspect, in one possible design, the filter switcher is also configured to accept motor control and switch the first white light filter or the fluorescent filter onto the optical path via gear transmission.

[0023] In conjunction with the first aspect, in one possible design approach, removing the fluorescence environmental noise from the raw fluorescence data to obtain valid fluorescence data includes:

[0024] The original fluorescence data and the fluorescence ambient noise are subjected to pixel difference calculation to obtain effective fluorescence data.

[0025] In conjunction with the first aspect, in one possible design approach, fusing the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image includes:

[0026] After stereo correction of the 2D fluorescence image and the 2D white light image, the odd-numbered rows of pixels in the stereo-corrected 2D fluorescence image and the even-numbered rows of pixels in the stereo-corrected 2D white light image are output alternately line by line to generate a 3D fluorescence image.

[0027] In conjunction with the first aspect, in one possible design, the image processor is further configured to overlay a pseudo-color onto the pixel values ​​in the effective fluorescence data, wherein the color intensity of the pseudo-color is positively correlated with the magnitude of the pixel values ​​in the effective fluorescence data.

[0028] In conjunction with the first aspect, in one possible design, the external viewing system also includes a display screen configured for use with 3D polarized glasses, the display screen displaying a 3D fluorescent image;

[0029] The left lens of the 3D polarized glasses assists the left eye in receiving 2D fluorescence images, and the right lens of the 3D polarized glasses assists the right eye in receiving 2D white light images. The 2D white light images and 2D fluorescence images are received simultaneously. The 2D white light images are used to provide information on clarity, brightness, and tissue color, while the 2D fluorescence images are used to provide fluorescence information of tumor tissue.

[0030] The brightness of the 2D white light image is greater than that of the 2D fluorescent image, and the sharpness of the 2D white light image is greater than that of the 2D fluorescent image.

[0031] Secondly, embodiments of this application provide a fluorescence imaging method, including:

[0032] In white light mode, white light is shone onto the target area through a light source generator;

[0033] The filter switcher located on the imaging optical path is controlled to switch to the first white light filter. The light in the optical path where the filter switcher is located is collected by the first lens group and focused into the first image sensor. The first white light data is acquired through the first image sensor. At the same time, the light in the optical path where the second white light filter is located is collected by the second lens group and focused into the second image sensor. The second white light data is acquired through the second image sensor.

[0034] The first white light data and the second white light data are converted into a 3D white light image using an image processor.

[0035] The filter switcher is controlled to switch to the fluorescent filter, and the fluorescent ambient noise is collected by the first image sensor;

[0036] After the tumor tissue cells in the target area contain fluorescent substances, in fluorescence mode, excitation light is irradiated onto the target area through the light source generator;

[0037] Raw fluorescence data is acquired through the first image sensor;

[0038] The image processor removes the fluorescence environmental noise from the raw fluorescence data to obtain effective fluorescence data, and generates a 2D fluorescence image based on the effective fluorescence data.

[0039] The image processor fuses the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image.

[0040] Thirdly, embodiments of this application provide a terminal including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method of the second aspect and its possible design.

[0041] Fourthly, embodiments of this application provide a storage medium storing a computer program, wherein the computer program is configured to execute the second aspect and its possible design methods at runtime.

[0042] Compared with the prior art, the present application provides an external viewing mirror system and method for fluorescence imaging. The external viewing mirror system includes: a light source generator configured to irradiate a target area with white light and excitation light; a filter switcher disposed in the imaging optical path, configured to allow light with wavelengths within the white light wavelength range to pass through after switching to a first white light filter, and to allow light with wavelengths within the fluorescence wavelength range to pass through after switching to a fluorescence filter; a first lens group and a second lens group, the first lens group being configured to collect light from the optical path where the filter switcher is located and focus it into a first image sensor, and the second lens group being configured to collect light from the optical path where a second white light filter is located and focus it into a second image sensor; the first image sensor being configured to, when the filter switcher is switched to the first white light filter... The system acquires first white light data in the optical path where the filter switcher is located; acquires fluorescence ambient noise when the filter switcher is switched to the fluorescence filter and the excitation light is off; and acquires raw fluorescence data when the filter switcher is switched to the fluorescence filter and the excitation light is on. A second image sensor is configured to acquire second white light data in the optical path where the second white light filter is located. An image processor is configured to generate a 3D white light image based on the first white light data and the second white light data in white light mode; remove the fluorescence ambient noise from the raw fluorescence data in fluorescence mode to obtain effective fluorescence data, and generate a 2D fluorescence image based on the effective fluorescence data; and fuse the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image. This application improves the clarity of visual effects through 3D imaging. By setting different filter parameters for the left and right eyes, a filter switcher, and switching the excitation light on and off, it achieves the simultaneous output of 3D fluorescence images and 3D white light images. This allows doctors to see images that have both the brightness and clarity of white light images and tumor fluorescence information, resulting in better visual effects. Furthermore, it eliminates the need to turn off the operating room lights and equipment lighting sources when displaying fluorescence images, thus providing surgical brightness and simplifying the operation.

[0043] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1A The following is a spectrum of excitation and emission light of 5-ALA fluorescence provided in an embodiment of this application;

[0046] Figure 1BThe following is a spectrum of excitation and emission light of a fluorescent sodium fluorescein provided in an embodiment of this application;

[0047] Figure 2 This illustration shows a schematic diagram of the structure of an external viewing mirror system for fluorescence imaging provided in an embodiment of this application;

[0048] Figure 3 A schematic diagram of a light source system provided in an embodiment of this application is shown;

[0049] Figure 4 A schematic diagram of a filter switcher provided in an embodiment of this application is shown;

[0050] Figure 5 A schematic diagram of a noise reduction method provided in an embodiment of this application is shown;

[0051] Figure 6 A flowchart of a fluorescence imaging method provided in an embodiment of this application is shown;

[0052] Figure 7 A schematic diagram of the structure of an external viewing mirror system provided in an embodiment of this application is shown. Detailed Implementation

[0053] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0055] Complete resection of gliomas often leads to better prognosis and longer progression-free survival, with significantly enhanced effects from postoperative radiotherapy and chemotherapy. However, malignant gliomas are highly invasive, often with indistinct boundaries from surrounding normal brain tissue, making them difficult to visualize under a microscope. This poses a significant challenge for surgeons, as incomplete removal results in a higher recurrence rate. The application of 5-ALA fluorescence technology overcomes this limitation. Specifically, malignant gliomas typically disrupt the normal blood-brain barrier, causing the tumor tissue to take up large amounts of 5-ALA and convert it into fluorescent protoporphyrins. Under a microscope, the strongly pink tumor itself and its pale pink borders can be observed, allowing surgeons to effectively distinguish between normal and tumor tissue. This significantly increases the resection rate of malignant tumors while minimizing damage to normal tissue, making malignant glioma resection surgery faster, safer, and more effective.

[0056] Figure 1A The following is a spectrum of excitation and emission light of 5-ALA fluorescence provided in an embodiment of this application, as shown. Figure 1AAs shown, the solid line represents the excitation light, and the dashed line represents the emission light. It can be seen that the peak intensity of the excitation light is in the range of 380nm to 420nm, and the peak intensity of the emission light is in the range of 600nm to 700nm, indicating good 5-ALA fluorescence imaging performance in this wavelength range. Since the white light wavelength is in the range of 400nm to 700nm, the fluorescence emission wavelength overlaps with the white light wavelength, making it impossible to separate the white light and fluorescence by filtering. Therefore, it is impossible to simultaneously acquire and display white light and fluorescence images. Furthermore, the white light increases brightness, resulting in a dark and poorly sharp fluorescence image when viewed alone.

[0057] In addition to 5-ALA fluorescence imaging, microscopic fluorescein sodium (yellow fluorescence) assisted technology can also help depict the morphology of tumors and determine the location and extent of tumor tissue. Especially for tumors with unclear boundaries, such as gliomas, or those involving important brain functional areas, fluorescein sodium can help to complete tumor resection to the greatest extent, safely and reliably.

[0058] Figure 1B The following is an example of the excitation and emission spectra of sodium fluorescein provided in this application. Figure 1B As shown, the peak intensity of the excitation light is in the range of 460nm to 490nm, while the peak intensity of the fluorescence emission light from sodium fluorescein is in the range of 510nm to 530nm. The white light wavelength is in the range of 400nm to 700nm. Since the fluorescence emission light from sodium fluorescein overlaps with the white light wavelength, it is impossible to separate the white light and fluorescence by filtering, thus making it impossible to simultaneously acquire and display white light and fluorescence images. Furthermore, since white light increases brightness, viewing the fluorescence image alone without the aid of a white light image results in a dark and unclear image.

[0059] To improve the contrast of fluorescent regions in images, existing techniques use filters to cut off a portion of white light wavelengths. For example, in 5-ALA fluorescence imaging, white light in the range of approximately 400nm to 550nm is cut off. Furthermore, to reduce the influence of ambient light on the results, ambient light is typically turned off during surgery. This improves the contrast of fluorescent regions but sacrifices image brightness, causing significant inconvenience for observation. Additionally, switching ambient light on and off during surgery is cumbersome and prolongs the procedure.

[0060] Fluorescence images obtained in the prior art are presented as 2D images, which lack a sense of three-dimensionality when observed.

[0061] Furthermore, the procedure requires switching between white light and fluorescence images. Resection, suturing, and observation must be performed under white light, while switching to fluorescence images is necessary to determine the tumor boundary. It is evident that the current technology is cumbersome to use with fluorescence function, requiring the replacement of equipment or modules.

[0062] As can be seen from the above, the fluorescent images displayed by the current 5-ALA fluorescence technology or sodium fluorescein technology have low brightness, can only be viewed alone, and are 2D images, resulting in poor visual effects. During the operation, it is necessary to switch back and forth between white light images and fluorescent images, and the operating room lights need to be turned off, and even equipment or modules need to be replaced, which is cumbersome.

[0063] In view of this, embodiments of this application provide an exoscopic system for fluorescence imaging, which can detect fluorescence under white light operating conditions (i.e., without the need for a dark or dimly lit operating room) during tumor resection; and can separate the white light in the fluorescence to present a clear fluorescence image; in addition, it can also display the fluorescence image in 3D, making the imaging more intuitive.

[0064] It should be noted that although this application specifically discusses fluorescence imaging for the purpose of tumor resection, the teachings of this application can be extended to other types of surgery, to detect other types of tissues, and to detect other types of fluorophores. In particular, the technology provided in this application provides better visual imaging of fluorophores with reflected light wavelengths in the white light range.

[0065] Figure 2 A schematic diagram of the structure of an external viewing mirror system for fluorescence imaging provided in an embodiment of this application is shown, as follows: Figure 2As shown, the external viewing mirror system includes: a light source generator 21, a filter switcher 22, two lens groups, a first image sensor 23, a second image sensor 24, and an image processor 25. The light source generator 21 is configured to illuminate the target area with white light and excitation light; the filter switcher 22 is disposed in the imaging optical path and is configured to allow light with wavelengths within the white light wavelength range to pass through after switching to the first white light filter, and to allow light with wavelengths within the fluorescence wavelength range to pass through after switching to the fluorescence filter; the first lens group 26 is configured to collect the light in the optical path of the filter switcher 22 and focus it into the first image sensor 23; the second lens group 27 is configured to collect the light in the optical path of the second white light filter 28 and focus it into the second image sensor 24; the first image sensor 23 is configured to acquire the light in the optical path of the filter switcher when the filter switcher 22 is switched to the first white light filter. The system acquires first white light data of the optical path containing the second white light filter 28; acquires fluorescence ambient noise when the filter switcher 22 is switched to the fluorescence filter and the excitation light is off; and acquires raw fluorescence data when the filter switcher 22 is switched to the fluorescence filter and the excitation light is on; a second image sensor 24 is configured to acquire second white light data of the optical path containing the second white light filter 28; an image processor 25 is configured to generate a 3D white light image based on the first white light data and the second white light data in white light mode; remove fluorescence ambient noise from the raw fluorescence data to obtain effective fluorescence data in fluorescence mode, and generate a 2D fluorescence image based on the effective fluorescence data; and fuse the 2D fluorescence image with the 2D white light image to generate a 3D fluorescence image.

[0066] In this embodiment, the light source generator 21 can emit two types of light: white light and excitation light. The white light is used to illuminate the target area, and the excitation light induces fluorescence emission from fluorescent substances in the tumor tissue of the target area. The fluorescent substances include protoporphyrin induced by 5-ALA applied to tumor tissue cells, or fluorescein applied to tumor tissue cells. The light source generator 21 controls the two light sources through an illumination optical path combination module, and outputs the two light sources through a single interface, as described below.

[0067] Figure 3 A schematic diagram of a light source system provided in an embodiment of this application is shown, such as... Figure 3As shown, the light source system (i.e., light source generator 21) includes a communication control interface, a core control module (ARM), a white LED driving circuit, an excitation LED driving circuit, a white LED, an excitation LED, an illumination optical path combination module, and a light output interface. The light source system receives control commands through the communication control interface to switch the light source circuit on and off. Specifically, during routine surgical procedures, a control command is sent to the light source system to activate the white LED driving circuit, at which point the light source system emits white light. When tumor fluorescence imaging is required for observation, a control command is sent to the light source system to activate the excitation LED driving circuit, at which point the light source system emits excitation light. The excitation light uses an LED as the light emitter, and the peak of the excitation light can be selected within the range capable of emitting emission light; this application does not impose any limitations on this.

[0068] As an example, the light source system can be connected to a touch control screen via a communication control interface, allowing medical professionals to switch light sources by operating the touch control screen. Furthermore, the aforementioned control commands can also be issued via voice, gestures, remote control, etc., and this application does not impose any limitations on this.

[0069] In some embodiments, the light source emitted by the light source generator 21 is a point light source with a circular or elliptical light spot. The point light source shines from the lens group onto the observed area. This reduces reflection and glare, concentrates illumination, and makes the captured image brighter and provides a better visual effect.

[0070] Figure 4 A schematic diagram of a filter switcher provided in an embodiment of this application is shown, such as... Figure 4 As shown, the filter switcher 22 includes a first white light filter 41, a fluorescent filter 42, a gear and transmission device 43, and a motor 44. The filter switcher 22 controls the gear and transmission device 43 via the motor 44, thereby switching either the first white light filter 41 or the fluorescent filter 42 onto the optical path. Specifically, the first white light filter 41 and the fluorescent filter 42 are connected. After receiving a command, the filter switcher 22 sends a control signal to drive the motor to rotate. The motor drives the gear and transmission device 43 to move simultaneously, thereby completing the filter switching. In some embodiments, the filter switcher 22 exists only in one optical path, such as in the optical path corresponding to the left eye or the optical path corresponding to the right eye. The optical path corresponding to the left eye means that the left eye can observe the image generated by this optical path, and the optical path corresponding to the right eye means that the right eye can observe the image generated by this optical path. It is understandable that if filter switcher 22 is in one optical path, then the second white light filter 28 is in the other optical path. For example, if filter switcher 22 is in the optical path corresponding to the left eye, then the second white light filter 28 is in the optical path corresponding to the right eye, and vice versa. The second white light filter 28 has the same light transmission parameters as the first white light filter in filter switcher 22.

[0071] As an example, the first white light filter allows light with wavelengths within the white light wavelength range to pass through. This first white light filter can be a high-pass filter with wavelengths between 400nm and 700nm. If a 400nm high-pass filter is used, the first white light filter allows light with wavelengths above 400nm to pass through, while blocking light with wavelengths shorter than 400nm. For the wavelengths of light that can pass through a 500nm or 700nm first white light filter, please refer to the description of using a 400nm high-pass filter; it will not be repeated here. The fluorescent filter allows light with wavelengths within the fluorescence wavelength range to pass through. When the fluorescent material is protoporphyrin, a high-pass filter with wavelengths between 550nm and 750nm can be used; when the fluorescent material is sodium fluorescein, a high-pass filter with wavelengths between 500nm and 550nm can be used. The above examples of filter parameters are merely illustrative and do not constitute limitations. Actual filter parameters can be adjusted based on these examples to achieve the functionality of this application.

[0072] The first image sensor 23, like the first lens group 26, is configured in the optical path of the filter switcher 22 to collect the light focused by the first lens group 26. The filter switcher 22 switches between two modes: a first white light filter and a fluorescence filter. The illumination of the target area with white light by the light source generator 21 is referred to as the white light mode, and the illumination of the target area with white light and excitation light by the light source generator 21 is referred to as the fluorescence mode. The data collected by the first image sensor 23 will be explained below. In white light mode, the filter switcher 22 switches to the first white light filter, and the first image sensor 23 collects the first white light data to present a 2D white light image. In white light mode, the filter switcher 22 switches to the fluorescence filter, and the first image sensor 23 collects the fluorescence ambient noise. In fluorescence mode, the filter switcher 22 switches to the fluorescence filter, and the first image sensor 23 collects the raw fluorescence data. In fluorescence mode, the raw fluorescence data is denoised using the fluorescence ambient noise to generate effective fluorescence data to present a 2D fluorescence image. Among them, the raw fluorescence data characterizes the reflected light (or fluorescence) emitted by the protoporphyrin in the target region when it is excited.

[0073] In some embodiments, the touchscreen can display a white light mode and a fluorescence mode. When the white light mode is pressed, the touchscreen detects the trigger operation of the white light mode control and generates a control command to control the light source generator 21 to emit white light and control the filter switcher 22 to switch to the first white light filter. When the fluorescence mode is pressed, the touchscreen detects the trigger operation of the fluorescence mode control and generates a control command to control the light source generator 21 to emit excitation light and white light, and control the filter switcher 22 to switch to the fluorescence filter. This linkage control simplifies the operation.

[0074] The second image sensor 24, like the second lens group 27, is positioned in the optical path of the second white light filter 28 to collect the light focused by the second lens group 27, obtaining second white light data to present a 2D white light image. In white light mode, the 2D white light image generated from the first white light data can be fused with the 2D white light image generated from the second white light data to generate a 3D white light image. The 3D white light image has depth information, helping doctors to confirm the location of tissues. In fluorescence mode, the 2D white light image generated from the second white light data can also be fused with a 2D fluorescence image to generate a 3D fluorescence image. The 3D fluorescence image provides depth information of the tumor site, with a clear visual effect.

[0075] It's worth noting that after wearing 3D polarized glasses, the doctor's left eye receives the fluorescence image, and the right eye receives the white light image. Utilizing the brain's inherent stereoscopic fusion function, these two images, which have parallax, are processed and fused to create a sense of depth and stereoscopic effect. Because the brightness and clarity of the 2D white light image are greater than those of the 2D fluorescence image, the simultaneous reception of images by the human eye allows for both the brightness and clarity of the right-eye white light image and the tumor fluorescence information from the left eye. Furthermore, there is no relative time interval between the acquisition of the white light and fluorescence images, and there is no need to reduce the original sampling frame rate of the fluorescence / white light images. This means that while ensuring that the fluorescence and white light images are sampled, processed, transmitted, and displayed simultaneously, the brightness and clarity of the tumor fluorescence imaging image are improved. Moreover, since it is not necessary to turn off the ambient light when acquiring the fluorescence image, but rather to remove the influence of white light through noise reduction, the operation is convenient.

[0076] Regarding the method of generating 3D images, one approach is to output the odd and even rows of two images alternately, line by line, to obtain the 3D image. Specifically, the images are acquired by the left and right eye image sensors, the analog signals are converted into digital signals, transmitted via cable, and processed by the image processor 25. After stereo correction, the images are output line by line in a 3D video format. That is, the odd-numbered rows of the left-eye image and the even-numbered rows of the right-eye image are extracted and combined to form the 3D image. In this implementation example, if a 4K 3D video image is output, the left eye takes rows 1, 3, 5, 7, ..., 2159, and the right eye takes rows 2, 4, 6, 8, ..., 2160; the left eye takes a fluorescent image, and the right eye takes a white light image. If the output image format is 1080P, which is 1920*1080, then each row has 1920 pixels. Therefore, the left eye takes rows 1, 3, 5, 7, ..., 1919, and the right eye takes rows 2, 4, 6, 8, ..., 1920.

[0077] The fluorescence denoising method provided in the embodiments of this application will be further described below. Figure 5The diagram illustrates a noise reduction method provided in an embodiment of this application, as shown below. Figure 5 As shown, the pixel values ​​of each pixel represented by the fluorescence environmental noise (one source is the fluorescence observation band energy contained in the operating room ambient light and other equipment in the operating room; the other source is the fluorescence observation band energy contained in the white light source band of the equipment) are denoted as a1, a2, ..., nn. The pixel values ​​of each pixel represented by the original fluorescence data are denoted as a1', a2', ..., nn'. The pixel values ​​of each pixel in the effective fluorescence data are denoted as a1”, a2”, ..., nn”, where a1” = a1' - a1, a” = a2' - a2, and so on. The effective fluorescence data is obtained by performing pixel difference calculation on the original fluorescence data and the fluorescence environmental noise.

[0078] In some embodiments, to further improve the contrast of the fluorescence image, the image processor 25 is also configured to overlay a false color on the pixel values ​​in the effective fluorescence data, wherein the color intensity of the false color is positively correlated with the magnitude of the pixel values ​​in the effective fluorescence data.

[0079] Specifically, a pseudo-color is superimposed on the fluorescence response region. For example, but not limited to, green, purple, or other colors can be superimposed. The effective fluorescence data is calibrated and correlated with the intensity of the superimposed pseudo-color to form a fluorescence intensity vs. pseudo-color intensity comparison table. Different color intensities are superimposed according to the value of the effective fluorescence data. The larger the effective fluorescence data, the stronger the color intensity, and the smaller the value, the weaker the color intensity.

[0080] By adding a color different from that of the brain and spinal cord tissues, the color difference between the tumor area and normal tissue can be increased, allowing for the observation of tissue details.

[0081] The above describes an external endoscope system for fluorescence imaging provided in this application embodiment. This system improves visual clarity through 3D imaging. By setting differentiated filter parameters for the left and right eyes, using the filter switcher 22, and controlling the excitation light, it simultaneously outputs fluorescence and white light images. This allows the doctor to view images that possess both the brightness and clarity of white light images and tumor fluorescence information, resulting in better visual effects. Furthermore, it eliminates the need to turn off operating room lights and equipment illumination sources when displaying fluorescence images, providing surgical brightness and simplifying operation. In addition, by using a color overlay scheme, the clarity and brightness of the white light image are ensured, while the fluorescence contrast of the tumor lesion area is improved, allowing for better identification of the tumor and its boundaries. Moreover, by controlling the mode switching on the touchscreen, the light source generator 21 and filter switcher 22 can be simultaneously activated, reducing operational steps and achieving automatic switching through linked control.

[0082] In conjunction with the external viewing mirror system described in the above embodiments, this application also provides a fluorescence imaging method, which is applied to the external viewing mirror system and will be described below.

[0083] Figure 6 A flowchart of a fluorescence imaging method provided in an embodiment of this application is shown, as follows: Figure 6 As shown, the method includes steps S601 to S608.

[0084] Step S601: In white light mode, illuminate the target area with white light through the light source generator.

[0085] Step S602: Control the filter switcher located on the imaging optical path to switch to the first white light filter, collect the light in the optical path where the filter switcher is located through the first lens group, focus it into the first image sensor, and collect the first white light data through the first image sensor; at the same time, collect the light in the optical path where the second white light filter is located through the second lens group, focus it into the second image sensor, and collect the second white light data through the second image sensor.

[0086] Step S603: Convert the first white light data and the second white light data into a 3D white light image using an image processor.

[0087] Step S604: Control the filter switcher to switch to the fluorescence filter, and collect the fluorescence ambient noise through the first image sensor.

[0088] Step S605: After the tumor tissue cells in the target area contain fluorescent substances, in fluorescence mode, excitation light is irradiated onto the target area through a light source generator.

[0089] Step S606: Acquire raw fluorescence data through the first image sensor.

[0090] Step S607: Remove fluorescence environmental noise from the original fluorescence data using an image processor to obtain effective fluorescence data, and generate a 2D fluorescence image based on the effective fluorescence data.

[0091] Step S608: The 2D fluorescence image and the 2D white light image are fused by an image processor to generate a 3D fluorescence image.

[0092] For an explanation of steps S601 to S608, please refer to the above description of the various components of the external viewing mirror system; they will not be repeated here.

[0093] Figure 7 A schematic diagram of the structure of an external viewing mirror system provided in an embodiment of this application is shown, as follows: Figure 7 As shown, the external viewing system may include:

[0094] Camera 71: It contains an optical system and an image acquisition module. The optical system adopts a dual-path variable magnification optical system scheme to provide three-dimensional imaging.

[0095] The image acquisition module uses a CMOS sensor for optical imaging and image signal conversion, which is equivalent to the first image sensor 23 and the second image sensor 24 mentioned above.

[0096] The image processor receives digital signals acquired by the CMOS sensor and transmitted to the back end via cable. It performs image processing based on the digital signals and outputs the video to the 3D monitor for display through the port. The 3D monitor provides a clear and magnified image.

[0097] Lighting system: LEDs are used as the light source, and the lighting is provided within the field of view through the beam guide and lighting optics system; the light source is located inside the trolley body, and the lighting optics system is located inside the camera.

[0098] The support system consists of a movable trolley body 72, a camera arm 73, and a monitor bracket 74. The trolley has multiple locking casters 75 (e.g., four locking casters) for movement and locking. The trolley body 72 is equipped with an internal balancing device to ensure stability during equipment and arm movement. The camera arm 73 is supported by a 6-degree-of-freedom robotic arm, allowing for three-dimensional spatial movement to meet the surgeon's operational needs. It also supports the monitor 76, enabling multi-directional movement (forward, backward, up, down, left, and right) to meet different observation angles during surgery.

[0099] Electrical installations ( Figure 7 (Not shown in the image): Located inside the support system, it provides the electrical energy required for the operation of each part of the equipment.

[0100] Touch control screen 77: Used for operation control of the entire device, including white balance, light source adjustment, video recording, display mode, photo taking, video recording, arm control, focus, zoom, image adjustment, dual-lens combination, user management, patient management, file management, settings, etc. Among them, the lighting system and filter switcher 22 can be automatically and synchronously controlled by setting white light mode and fluorescent mode.

[0101] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0102] Furthermore, in conjunction with the methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the 5-aminolevulinic acid fluorescence imaging exoscope systems described in the above embodiments.

[0103] This application also provides a computer program product that, when run on a computer, causes the computer to perform various functions or steps executed by the processor in the above method embodiments.

[0104] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0105] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0106] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0107] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An exterior mirror system for fluorescence imaging, characterized in that The application comprises: a light source generator configured to irradiate a target area with white light and excitation light; a filter switcher arranged in an imaging light path and configured to allow light having a wavelength within a white light wavelength range to pass after switching to a first white light filter, and to allow light having a wavelength within a fluorescent light wavelength range to pass after switching to a fluorescent light filter; a first lens group configured to collect light in a light path in which the filter switcher is arranged and focus the light into a first image sensor, and a second lens group configured to collect light in a light path in which a second white light filter is arranged and focus the light into a second image sensor; a first image sensor configured to acquire first white light data of the light path in which the filter switcher is arranged when the filter switcher is switched to the first white light filter; acquire fluorescent environmental noise when the filter switcher is switched to the fluorescent light filter and the excitation light is in an off state; and acquire fluorescent raw data when the filter switcher is switched to the fluorescent light filter and the excitation light is in an on state; a second image sensor configured to acquire second white light data of the light path in which the second white light filter is arranged; an image processor configured to generate a 3D white light image based on the first white light data and the second white light data in a white light mode; acquire effective fluorescent data by removing the fluorescent environmental noise from the fluorescent raw data in a fluorescent light mode, and generate a 2D fluorescent light image based on the effective fluorescent data; and fuse the 2D fluorescent light image with a 2D white light image to generate a 3D fluorescent light image. The white light is used to illuminate the target area, and the excitation light is used to induce the fluorescent light emitted by fluorescent substances in tumor tissues of the target area, the fluorescent substances including protoporphyrin induced by 5-aminolevulinic acid administered in tumor tissue cells, or fluorin administered in tumor tissue cells. In the case where the fluorescent substances are the protoporphyrin, the peak value of the excitation light is within a range of 380 nm to 420 nm, and the peak value of fluorescent emission light of the protoporphyrin is within a range of 600 nm to 700 nm.

2. The exterior mirror system with fluorescence imaging according to claim 1, characterized in that, In the case where the fluorescent substances are the fluorin, the peak value of the excitation light is within a range of 460 nm to 490 nm, and the peak value of fluorescent emission light of the fluorin is within a range of 510 nm to 530 nm.

3. The exterior mirror system with fluorescence imaging according to claim 2, characterized in that, The removal of the fluorescent environmental noise from the fluorescent raw data to obtain effective fluorescent data comprises:

4. The exterior mirror system with fluorescence imaging according to claim 2, characterized in that, performing pixel difference operation on the fluorescent raw data and the fluorescent environmental noise to obtain the effective fluorescent data.

5. The exterior mirror system with fluorescence imaging according to claim 1, characterized in that, The fusion of the 2D fluorescent light image with the 2D white light image to generate the 3D fluorescent light image comprises: after stereo correction of the 2D fluorescent light image and the 2D white light image, alternately outputting odd row pixels of the stereo corrected 2D fluorescent light image and even row pixels of the stereo corrected 2D white light image row by row to generate the 3D fluorescent light image.

6. The exterior mirror system with fluorescence imaging according to claim 1, characterized in that, The image processor is further configured to superimpose a pseudo color on pixel values in the effective fluorescent data, wherein the color intensity of the pseudo color is positively correlated with the size of the pixel values in the effective fluorescent data. The application further comprises:

7. The exterior mirror system with fluorescence imaging according to claim 1, characterized in that, ​ 8. The exterior mirror system with fluorescence imaging according to claim 1, characterized in that, ​ A display screen configured to cooperate with 3D polarized light glasses, the display screen displays 3D fluorescent images; The left lens of the 3D polarized light glasses assists the left eye to receive 2D fluorescent images, and the right lens of the 3D polarized light glasses assists the right eye to receive 2D white light images, wherein the 2D white light images and 2D fluorescent images are received simultaneously, and the 2D white light images are used to provide clarity, brightness and tissue color information, and the 2D fluorescent images are used to provide tumor tissue fluorescent information.

9. A method of fluorescence imaging, characterized by, The method comprises: In the white light mode, a white light is irradiated to the target area by a light source generator; A filter switcher located in the imaging light path is controlled to switch to a first white light filter, light rays in the light path of the filter switcher are collected by a first lens group and focused into a first image sensor, and first white light data are collected by the first image sensor; at the same time, light rays in the light path of a second white light filter are collected by a second lens group and focused into a second image sensor, and second white light data are collected by the second image sensor; The first white light data and the second white light data are converted into 3D white light images by an image processor; The filter switcher is controlled to switch to a fluorescent filter, and fluorescent environmental noise is collected by the first image sensor; After the tumor tissue cells in the target area contain fluorescent substances, in the fluorescent mode, an excitation light is irradiated to the target area by the light source generator; Fluorescent raw data are collected by the first image sensor; The fluorescent environmental noise is removed from the fluorescent raw data by the image processor to obtain effective fluorescent data, and 2D fluorescent images are generated based on the effective fluorescent data; The 2D fluorescent images and 2D white light images are fused by the image processor to generate 3D fluorescent images.

10. A storage medium, characterized by The storage medium stores a computer program, wherein the computer program is configured to execute the fluorescent imaging method of claim 9 when running. The storage medium stores a computer program, wherein the computer program is configured to execute the fluorescent imaging method of claim 9 when running.

Images