Multi-drug fluorescence spectrum adaptive imaging analysis method and device

Abstract

The invention provides a multi-drug fluorescence spectrum adaptive imaging analysis method and device. Comprising the following steps: performing incident wavelength search on an original fluorescence image acquired by a fluorescence camera by applying a Bayesian optimization algorithm to obtain an optimal incident wavelength; and under the optimal incident wavelength, acquiring an original fluorescence image again by the fluorescence camera, acquiring an original visible light image by the visible light camera, performing deep learning filtering noise reduction on the original visible light image and the original fluorescence image, and inputting the images into an image fusion network at the same time to obtain an image fusion fluorescence image. The fluorescent drug excitation wavelength can be automatically matched, the fluorescent brightness can be automatically adjusted, the debugging time of a machine is shortened, and the utilization rate of equipment is improved.

Description

Technical Field

[0001] This invention relates to the field of medical imaging and digital image signal processing technology, specifically to a multi-drug fluorescence spectroscopy adaptive imaging analysis method and apparatus. Background Technology

[0002] In traditional medical diagnosis, fluorescence images reflect information such as tumor size and outline, while visible light images reflect information such as tissue details and location. Fusing these two sets of features can provide accurate judgment for tumor identification and precise localization. Therefore, image fusion technology effectively combines the advantages of fluorescence and visible light images. Through precise navigation of the fused image, disease treatment can be more accurate, systematic, and comprehensive. By fluorescently labeling tumor tissue structures and displaying real-time fluorescence fusion images of the tumor area, surgeons can be assisted in identifying tumor shape and other characteristics, thereby distinguishing it from normal healthy tissue.

[0003] However, there are currently hundreds of fluorescently labeled drugs, and more than a dozen known incident wavelengths with a wide range. Supporting only one or two drugs would significantly limit the instrument's applicability. Furthermore, manually calibrating the incident wavelength each time would greatly increase the instrument's operating costs. Additionally, due to differences in manufacturing processes, batch variations, and the inconsistent techniques and tools used by different manufacturers to test the optimal excitation wavelength, the optimal excitation wavelength for fluorescently labeled drugs may not be accurately measured. Fluorescence images acquired using non-optimal excitation wavelengths may lead to distortion of the drug's fluorescent molecular labeling, significantly increasing the risk of misdiagnosis of lesions by physicians.

[0004] Based on this, the present invention provides a method and apparatus for adaptive imaging analysis of multi-drug fluorescence spectroscopy. Summary of the Invention

[0005] To address the issues of inconsistent excitation wavelengths and varying excitation intensities of fluorescent drugs in existing medical image stitching and fusion processes, this invention provides a multi-drug fluorescence spectroscopy adaptive imaging analysis method and related apparatus. The method includes:

[0006] Step S100: Turn on the device power and start the fluorescence acquisition camera;

[0007] Step S200: Set the upper and lower limits of the excitation wavelength of the fluorescently labeled drug for the filter scanning;

[0008] Step S300: Turn on the laser generator, adjust the cavity length of the filter to move through the excitation wavelength of the fluorescently labeled drug in a certain step size, use the fluorescence acquisition camera to acquire fluorescence images, and use filtering and noise reduction technology to filter and reduce noise to obtain fluorescence noise-reduced images.

[0009] Step S400, change the incident wavelength, collect a series of fluorescence images, apply a traditional optimization algorithm (such as Bayesian optimization algorithm) to analyze the fluorescence intensity of the fluorescence denoising image, and calculate the optimal excitation wavelength of the fluorescently labeled drug;

[0010] Step S500, set the filter wavelength of the filter to the optimal excitation wavelength of the fluorescently labeled drug obtained in step S400;

[0011] Step S600, start the visible light acquisition camera, and simultaneously acquire visible light images and fluorescence images using the visible light acquisition camera and the fluorescence acquisition camera, respectively;

[0012] Step S700, apply filter denoising technology to filter and denoise the visible light images and fluorescence images obtained in step S600, respectively, to obtain visible light denoising images and fluorescence denoising images;

[0013] Step S800, apply image matching fusion technology to match and fuse the visible light denoising images and fluorescence denoising images obtained in step S700, to obtain a fluorescence fusion image, wherein the fluorescence image is presented in pseudo-color form.

[0014] As an embodiment of the present application, the present application provides a multi-drug fluorescence spectrum adaptive imaging analysis device. The multi-drug near-infrared fluorescence spectrum adaptive imaging analysis device mainly comprises a camera lens 101, a light splitting prism 201, a visible light camera 301, a fluorescence camera 302, an adjustable optical filter 401, an optical filter controller 402, and a fluorescence intensity analysis module 501.

[0015] The camera lens 101 is used to receive visible light reflected from biological tissues and fluorescence emitted by fluorescently labeled drugs.

[0016] The light splitting prism 201 is used to split the mixed light obtained by the camera lens 101 into two beams, one of which is reflected light and the other of which is refracted light.

[0017] The visible light camera 301 is used to acquire visible light images.

[0018] The fluorescence camera 302 is used to acquire fluorescence images.

[0019] The adjustable optical filter 401 is used to filter fluorescence of a specific wavelength. Its basic structure is a Fabry-Perot resonant cavity, which is composed of a plane mirror and a concave lens.

[0020] The optical filter controller 402 is used to adjust the cavity length of the Fabry-Perot resonant cavity in the adjustable optical filter 401, thereby controlling the transmission wavelength of the adjustable optical filter.

[0021] The fluorescence intensity analysis module 501 is used to calculate the fluorescence intensity related parameters of the fluorescence image.

[0022] Compared with the prior art, the present application has the beneficial effects that:

[0023] (1) The present application automatically searches for the drug excitation wavelength, so the device (fluorescence endoscope, etc.) is no longer limited to a certain drug, and the applicability is more extensive.

[0024] (2) The present application automatically searches for the drug excitation wavelength, which is shorter in time and better in effect compared with manual setting and checking. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of a multi-drug fluorescence spectrum self-adaptive imaging analysis method of the present application.

[0026] Figure 2 is a structural schematic diagram of a multi-drug fluorescence spectrum self-adaptive imaging analysis device of the present application. DETAILED DESCRIPTION

[0027] The present application will be further described in detail below in combination with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. It should be noted that, for the sake of description, only the parts related to the application are shown in the drawings. The embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0028] Referring to Figure 1 , a multi-drug fluorescence spectrum self-adaptive imaging analysis method of the first embodiment of the present application comprises the following steps:

[0029] Step S100, turn on the power of the device and start the fluorescence acquisition camera;

[0030] Step S200, set the upper and lower boundaries of the filter scanning fluorescence marker drug excitation wavelength;

[0031] Step S300, turn on the laser generator, adjust the cavity length of the filter to move through the excitation wavelength of the fluorescence marker drug at a certain step, use the fluorescence acquisition camera to collect the fluorescence image, and use the filtering and noise reduction technology to filter and reduce the noise to obtain the fluorescence noise reduction image;

[0032] Step S400, change the incident wavelength, collect a series of fluorescence images, apply a traditional optimization algorithm (such as Bayesian optimization algorithm) to analyze the fluorescence intensity of the fluorescence noise reduction image, and calculate the optimal excitation wavelength of the fluorescence marker drug;

[0033] Step S500, set the filter wavelength of the filter to the optimal excitation wavelength of the fluorescence marker drug obtained in step S400;

[0034] Step S600, start the visible light acquisition camera, and simultaneously acquire a visible light image and a fluorescent image using the visible light acquisition camera and the fluorescent acquisition camera respectively;

[0035] Step S700, apply a filter denoising technology to filter and denoise the visible light image and the fluorescent image obtained in step S600, to obtain a visible light denoised image and a fluorescent denoised image;

[0036] Step S800, apply an image matching fusion technology to match and fuse the visible light denoised image and the fluorescent denoised image obtained in step S700, to obtain a fluorescent fusion image, wherein the fluorescent image is presented in a pseudo-color form.

[0037] As an embodiment of the present application, according to the multi-drug fluorescent spectrum adaptive imaging analysis method and device provided by the present application, the image filter denoising technology can be a deep learning denoising algorithm, or a traditional guided filter denoising algorithm, and the present application is not limited thereto.

[0038] As an embodiment of the present application, according to the multi-drug fluorescent spectrum adaptive imaging analysis method and device provided by the present application, the image matching fusion technology can be a deep learning image matching fusion algorithm, or a traditional SIFT algorithm or SURF algorithm plus a seam line stitching algorithm, and the present application is not limited thereto.

[0039] Referring to Figure 2 The multi-drug fluorescent spectrum adaptive imaging analysis device of the second embodiment of the present application mainly comprises a camera lens 101, a light splitting prism 201, a visible light camera 301, a fluorescent camera 302, an adjustable filter 401, a filter controller 402, and a fluorescent intensity analysis module 501.

[0040] The camera lens 101 is used to receive visible light reflected from a biological tissue and fluorescent light excited by a fluorescent marker drug.

[0041] The light splitting prism 201 is used to split the mixed light obtained by the camera lens 101 into two beams, one of which is reflected light and the other of which is refracted light.

[0042] The visible light camera 301 is used to acquire a visible light image.

[0043] The fluorescent camera 302 is used to acquire a fluorescent image.

[0044] The adjustable filter 401 is used to filter fluorescent light of a specific wavelength. The basic structure thereof is a Fabry-Perot resonant cavity, which is composed of a plane mirror and a concave lens.

[0045] The filter controller 402 is configured to adjust the cavity length of the Fabry-Perot resonant cavity in the tunable optical filter 401, so as to control the transmission wavelength of the tunable optical filter.

[0046] The fluorescence intensity analysis module 501 is configured to calculate the fluorescence intensity related parameters of the fluorescence image.

[0047] Referring to Figure 2 The optical path diagram of the multi-drug fluorescence spectrum adaptive imaging analysis device according to the third embodiment of the present application is shown in the following figure;

[0048] The specific light rays in the optical path diagram are as follows: Figure 2 The mixed light, which is composed of the fluorescence excited by the fluorescent marker drug and the visible light reflected by the biological tissue, is input to the camera lens 101 by the biological tissue, and the camera lens 101 outputs a first mixed light beam.

[0049] The first mixed light beam is input to the light splitting prism 201, and the light splitting prism 201 outputs two light beams, which are a first visible light beam and a first fluorescence light beam.

[0050] The first visible light beam is input to the visible light camera 301.

[0051] The first fluorescence light beam is input to the tunable optical filter 401, and the tunable optical filter 401 outputs a second fluorescence light beam.

[0052] The second fluorescence light beam is input to the fluorescence camera 302.

[0053] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0054] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0055] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable devices provide a process for implementing the flowchart Figure 1 One flowchart or multiple flowcharts and / or blocks Figure 1 One block or multiple blocks in the flowchart or multiple flowcharts and / or blocks

[0056] It should be noted that in the description of the present application, if there is a description of the orientation, such as the orientation or position relationship indicated by the above, below, front, back, left, right and the like, it is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and cannot be understood as indicating or implying that the device or element indicated must have a specific orientation, be constructed or operated in a specific orientation, and cannot be understood as a limitation on the present application.

[0057] In the description of the present application, the meaning of several is one or more, the meaning of multiple is two and more than two, greater than, less than, more than and the like are understood as not including the number, above, below, within and the like are understood as including the number. If there is a description of the first or second and the like, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the sequence of the indicated technical features.

[0058] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application according to the specific content of the technical scheme.

[0059] The principles and implementation modes of the present application are described in the specific embodiments in the present application, and the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in specific implementation modes and application scope, and the above description should not be understood as a limitation on the present application.

Claims

1. A multi-drug fluorescence spectral self-adaptive imaging analysis method, characterized in that, The method comprises the following steps: Step S100, turning on the power of the device and starting the fluorescence acquisition camera; Step S200, setting the upper and lower boundaries of the filter scanning the excitation wavelength of the fluorescent marker drug; Step S300, turning on the laser generator, adjusting the cavity length of the filter, moving through the excitation wavelength of the fluorescent marker drug at a certain step, using the fluorescence acquisition camera to acquire the fluorescence image, and using the filtering and denoising technology to filter and denoise to obtain the fluorescence denoising image; Step S400, applying the Bayesian optimization algorithm to analyze the fluorescence intensity of the fluorescence denoising image, and calculating the optimal excitation wavelength of the fluorescent marker drug; Step S500, setting the filtering wavelength of the filter to the optimal excitation wavelength of the fluorescent marker drug obtained in step S400; Step S600, starting the visible light acquisition camera, and using the visible light acquisition camera and the fluorescence acquisition camera to acquire the visible light image and the fluorescence image respectively; Step S700, applying the filtering and denoising technology to filter and denoise the visible light image and the fluorescence image obtained in step S600 respectively to obtain the visible light denoising image and the fluorescence denoising image; Step S800, applying the image matching fusion technology to the visible light denoising image and the fluorescence denoising image obtained in step S700 to obtain the fluorescence fusion image, wherein the fluorescence image is presented in the form of pseudo color; As an embodiment of the present application, according to the multi-drug fluorescence spectrum adaptive imaging analysis method and device provided by the present application, the image filtering and denoising technology can be a deep learning denoising algorithm or a traditional guided filtering and denoising algorithm, and the present application is not limited thereto; As an embodiment of the present application, according to the multi-drug fluorescence spectrum adaptive imaging analysis method and device provided by the present application, the image matching fusion technology can be a deep learning image matching fusion algorithm, or a traditional SIFT algorithm or SURF algorithm plus stitching line splicing algorithm, and the present application is not limited thereto.

2. A multi-drug fluorescence spectral self-adaptive imaging analysis device, characterized in that, The method comprises the following components: The camera lens 101, the light splitting prism 201, the visible light camera 301, the fluorescence camera 302, the adjustable filter 401, the filter controller 402, and the fluorescence intensity analysis module 501; The camera lens 101 is used to receive the visible light reflected from the biological tissue and the fluorescence excited by the fluorescent marker drug; The light splitting prism 201 is used to split the mixed light obtained by the camera lens 101 into two beams, one of which is reflected light and the other of which is refracted light; The visible light camera 301 is used to acquire the visible light image; The fluorescence camera 302 is used to acquire the fluorescence image; The adjustable filter 401 is used to filter the fluorescence of a specific wavelength; The basic structure is a Fabry-Perot resonant cavity, which is composed of a plane mirror and a concave lens; The filter controller 402 is used to adjust the cavity length of the Fabry-Perot resonant cavity in the adjustable filter 401, thereby adjusting the transmission wavelength of the adjustable filter; The fluorescence intensity analysis module 501 is used to calculate the fluorescence intensity related parameters of the fluorescence image.

Images