Fluorescence molecular imaging system, device, method and storage medium

Abstract

The embodiments of the present application provide a fluorescent molecular imaging system, device, method and storage medium. The system includes: a multi-spectral light source excitation module for synchronously emitting visible light and near-infrared light to a target detection area containing a near-infrared fluorescent contrast agent; an optical detector module for collecting a mixed light image of the target detection area; an image processing module for generating a visible fluorescent image based on the mixed light image; and a display module for displaying the visible fluorescent image in real time. The embodiments of the present application can more accurately identify normal biological tissue structures and abnormal biological tissue structures.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to a fluorescent molecular imaging system, device, method and storage medium. Background Art

[0002] Tumors are a major threat to human health. With the development of medical technology, surgeons can now remove tumors without affecting the normal vital signs of tissues and organs. However, it is difficult for existing medical imaging systems to accurately locate the boundaries of tumors, which results in surgeons being unable to accurately remove tumors during surgery. Therefore, how to accurately identify normal and abnormal biological tissue structures has become a technical problem that needs to be solved in the field of medical imaging technology. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a fluorescent molecular imaging system, device, method and storage medium to achieve more accurate identification of normal biological tissue structures and abnormal biological tissue structures.

[0004] To achieve the above objectives, on the one hand, an embodiment of the present application provides a fluorescent molecular imaging system, comprising:

[0005] A multi-spectral light source excitation module is used to simultaneously emit visible light and near-infrared light to a target detection area containing a near-infrared fluorescent contrast agent;

[0006] An optical detector module, used for collecting a mixed light image of the target detection area;

[0007] An image processing module, used for generating a visible fluorescence image according to the mixed light image;

[0008] A display module is used to display the visible fluorescent image in real time.

[0009] In one embodiment of the present application, the optical detector module includes:

[0010] An imaging lens, used for acquiring a first mixed light signal of the target detection area;

[0011] A filter, used for filtering out the near-infrared emission light in the first mixed light signal to obtain a second mixed light signal;

[0012] The photoelectric conversion module is used to convert the second mixed light signal into a mixed light image.

[0013] In one embodiment of the present application, the photoelectric conversion module includes any one of the following:

[0014] Charge coupled device;

[0015] Complementary Metal Oxide Semiconductor Image Sensor.

[0016] In one embodiment of the present application, the multi-spectral light source excitation module includes: an LED white light source and a near-infrared laser light source.

[0017] In one embodiment of the present application, the LED white light source and the near-infrared laser light source are close to each other and are independently arranged.

[0018] On the other hand, an embodiment of the present application further provides a multi-spectral light source excitation module, which is applied to a fluorescent molecular imaging system and is used to synchronously emit visible light and near-infrared light to a target detection area containing a near-infrared fluorescent contrast agent.

[0019] In one embodiment of the present application, the multi-spectral light source excitation module includes: an LED white light source and a near-infrared laser light source.

[0020] On the other hand, an embodiment of the present application further provides an optical detector module, which is applied to a fluorescent molecular imaging system and is used to collect a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent.

[0021] In one embodiment of the present application, the optical detector module includes:

[0022] An imaging lens, used for acquiring a first mixed light signal of the target detection area;

[0023] A filter, used for filtering out the near-infrared emission light in the first mixed light signal to obtain a second mixed light signal;

[0024] The photoelectric conversion module is used to convert the second mixed light signal into a mixed light image.

[0025] On the other hand, an embodiment of the present application further provides an image processing module, which is applied to a fluorescent molecular imaging system and includes:

[0026] An image acquisition unit, used to acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent;

[0027] An image synthesis unit is used to generate a visible fluorescent image according to the mixed light image.

[0028] On the other hand, the embodiment of the present application further provides an image processing method, including:

[0029] Acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent;

[0030] A visible fluorescent image is generated according to the mixed light image.

[0031] In an embodiment of the present application, generating a visible fluorescent image according to the mixed light image includes:

[0032] Extracting a fluorescence intensity image from the mixed light image, and performing adaptive brightness conversion on the mixed light image to obtain a second mixed light image;

[0033] Performing image enhancement on the fluorescence intensity image and the second mixed light image to obtain a third mixed light image;

[0034] Perform image noise reduction on the third mixed light image to obtain a visible fluorescence image.

[0035] In one embodiment of the present application, extracting the fluorescence intensity image from the mixed light image includes:

[0036] According to the formula Extracting a fluorescence intensity image from the mixed light image;

[0037] Among them, f1(i,j) is the grayscale value of position i,j in the fluorescence intensity image, f(i,j) is the grayscale value of position i,j in the mixed light image, A1 and A2 are the lower and upper thresholds of the preset grayscale range, respectively, representing the image.

[0038] In an embodiment of the present application, the step of performing adaptive brightness conversion on the mixed light image includes:

[0039] Performing adaptive brightness conversion on the mixed light image according to the formula f2(i,j)=a*f(i,j)+b to obtain a second mixed light image;

[0040] Among them, f(i,j) is the grayscale value of position i,j in the mixed light image, f2(i,j) is the grayscale value of position i,j in the second mixed light image, a is the coefficient of f(i,j), and b is a constant.

[0041] In one embodiment of the present application, the step of performing image enhancement on the fluorescence intensity image and the second mixed light mixed light image includes:

[0042] According to the formula Performing image enhancement on the fluorescence intensity image and the second mixed light mixed light image;

[0043] Among them, g(i,j) is the grayscale value of position i,j in the third mixed light image, f2(i,j) is the grayscale value of position i,j in the second mixed light image, f1(i,j) is the grayscale value of position i,j in the fluorescence intensity image, and B is a constant.

[0044] In an embodiment of the present application, performing image noise reduction on the third mixed light image includes:

[0045] According to the formula performing image noise reduction on the third mixed light image;

[0046] Among them, J(i,j) is the grayscale value of position i,j in the visible fluorescence image, g(i,j) is the grayscale value of position i,j in the third mixed light image, A is a constant, t(i,j) is the transmittance, t0 is the degree of image noise reduction, and w is a constant coefficient.

[0047] On the other hand, an embodiment of the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent;

[0049] A visible fluorescent image is generated according to the mixed light image.

[0050] It can be seen from the technical solutions provided by the above embodiments of the present application that the embodiments of the present application can obtain a mixed light image of the target detection area (which contains a near-infrared fluorescent contrast agent), and can generate a visible fluorescence image based on the mixed light image; and the visible light image part of the visible fluorescence image can obtain visible environmental information such as the surgical environment, tumors and tissues; and the fluorescence image part of the visible fluorescence image can obtain information on areas of interest such as diseased tissues and tumors. Therefore, the embodiments of the present application can be conducive to more accurate identification of normal biological tissues and diseased tissues in the target detection area, and thus can provide real-time and accurate navigation images in non-contact surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0052] Figure 1 A structural block diagram of a fluorescent molecular imaging system according to some embodiments of the present application;

[0053] Figure 2 This is a schematic diagram of the structure of the image processing module in some embodiments of the present application;

[0054] Figure 3 This is a schematic diagram of the structure of the image processing module in some other embodiments of the present application;

[0055] Figure 4 A flowchart of an image processing method according to some embodiments of the present application;

[0056] Figure 5 This is a flow chart of generating a visible fluorescence image in the image processing method in some embodiments of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application. For example, in the following description, the second component is formed above the first component, which may include an embodiment in which the first component and the second component are formed in direct contact, and may also include an embodiment in which the first component and the second component are formed in non-direct contact (that is, additional components may be included between the first component and the second component), etc.

[0058] Moreover, for ease of description, some embodiments of the present application may use spatially relative terms such as "above", "below", "top", "below", etc. to describe the relationship between one element or component and another (or other) elements or components as shown in the various figures of the embodiments. It should be understood that in addition to the orientations described in the figures, the spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the figures is turned over, the elements or components described as "below" or "beneath" other elements or components will subsequently be positioned as "above" or "on" the other elements or components.

[0059] See also Figure 1 As shown, the fluorescent molecular imaging system of some embodiments of the present application may include a multi-spectral light source excitation module, an optical detector module, an image processing module, a display module, etc. Among them, the multi-spectral light source excitation module can be used to synchronously emit visible light and near-infrared light to a target detection area containing a near-infrared fluorescent contrast agent. The optical detector module can be used to collect a mixed light image of the target detection area. The image processing module can be used to generate a visible fluorescence image based on the mixed light image. The display module can be used to display the visible fluorescence image in real time.

[0060] In the fluorescent molecular imaging system of the above-mentioned embodiment of the present application, the visible light image part (wavelength range: 450nm~650nm) in the visible fluorescent image can obtain visible environmental information such as the environment, tumors and tissues during surgery; and the fluorescent image part (wavelength range: 820nm~900nm) in the visible fluorescent image can obtain information on areas of interest such as diseased tissues and tumors. Therefore, the visible fluorescent image output and displayed by the fluorescent molecular imaging system of the above-mentioned embodiment of the present application can be conducive to more accurate identification of normal biological tissues and diseased tissues in the target detection area, thereby providing real-time and accurate navigation images in non-contact surgery.

[0061] In one embodiment of the present application, the multi-spectrum light source excitation module may be an LED white light source and a near-infrared laser light source that are close to each other and independently arranged, and the LED white light source and the near-infrared laser light source may synchronously emit visible light and near-infrared light to the target detection area. In another embodiment of the present application, the multi-spectrum light source excitation module may also be a single laser source, and the single laser source may emit a laser with a wavelength range that at least covers the visible light band and the near-infrared light band. Therefore, in some embodiments of the present application, the structure of the multi-spectrum light source excitation module is not limited, and may be selected according to needs.

[0062] In one embodiment of the present application, the optical detector module may include an imaging lens, a filter, and a photoelectric conversion module. The imaging lens may be used to obtain a first mixed light signal of the target detection area. The filter may be used to filter out near-infrared emission light in the first mixed light signal to obtain a second mixed light signal. The photoelectric conversion module may be used to convert the second mixed light signal into a mixed light image.

[0063] In some embodiments of the present application, the imaging lens may be of any suitable structure as required. For example, in an exemplary embodiment, the lens may be an endoscope lens, etc. The photoelectric conversion module may adopt any suitable existing photosensitive element, such as a charge coupled device (i.e., a CCD photosensitive element) or a complementary metal oxide semiconductor image sensor (i.e., a CMOS photosensitive element), etc.

[0064] In some embodiments of the present application, as needed, the image processing module can adopt any suitable hardware or a combination of hardware and software. For example, in some exemplary embodiments, the image processing module can be implemented using a hardware structure such as a Field Programmable Gate Array (FPGA) or a Complex Programmable Logic Device (CPLD). Among them, FPGA and CPLD are both hardware-based processing systems with fast data processing speeds. They can process multiple data simultaneously within one clock cycle, thereby shortening the processing time and reducing image delay. In the above hardware implementation, if Figure 2 As shown, the image processing module may specifically include an image acquisition unit 21 and an image synthesis unit 22. The image acquisition unit 21 may be used to acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent; and the image synthesis unit 22 may be used to generate a visible fluorescent image based on the mixed light image.

[0065] In some embodiments of the present application, for ease of use, the multi-spectral light source excitation module and the image processing module can be integrated into a system host, and the optical detector module can be integrated into a handle and connected to the system host through optical fibers, communication cables, etc. The optical fiber can introduce the near-infrared laser and LED white light of the multi-spectral light source excitation module through the handle and can be emitted from the front end of the handle. The image data collected by the optical detector module can be transmitted back to the system host for processing through the communication cable, and the image processing module can process and output it to the display module for display.

[0066] For the convenience of description, the above device is described in terms of functions and is divided into various units and described separately. Of course, when implementing the present application, the functions of each unit can be implemented in the same or more hardware modules.

[0067] In some other exemplary embodiments, Figure 3 As shown, the image processing module may also include a processor and a computer program stored in a memory, and when the computer program is executed by the processor, the following steps are performed:

[0068] Acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent;

[0069] A visible fluorescent image is generated according to the mixed light image.

[0070] Although the process flows described above include multiple operations that occur in a particular order, it should be clearly understood that the processes may include more or fewer operations, which may be performed sequentially or in parallel (eg, using parallel processors or a multi-threaded environment).

[0071] refer to Figure 4 As shown, the image processing method of some embodiments of the present application may include:

[0072] S401, acquiring a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent.

[0073] S402: Generate a visible fluorescent image according to the mixed light image.

[0074] In some embodiments of the present application, reference Figure 5 As shown, generating a visible fluorescent image according to the mixed light image may specifically include:

[0075] S4021. Extract a fluorescence intensity image from the mixed light image, and perform adaptive brightness conversion on the mixed light image to obtain a second mixed light image.

[0076] In an embodiment of the present application, extracting a fluorescence intensity image from a mixed light image may include:

[0077] According to the formula Extracting a fluorescence intensity image from the mixed light image;

[0078] Among them, f1(i,j) is the grayscale value of position i,j in the fluorescence intensity image, f(i,j) is the grayscale value of position i,j in the mixed light image, A1 and A2 are the lower and upper thresholds of the preset grayscale range, respectively, representing the image.

[0079] In an embodiment of the present application, the step of performing adaptive brightness conversion on the mixed light image includes:

[0080] Performing adaptive brightness conversion on the mixed light image according to the formula f2(i,j)=a*f(i,j)+b to obtain a second mixed light image;

[0081] Among them, f(i,j) is the grayscale value of position i,j in the mixed light image, f2(i,j) is the grayscale value of position i,j in the second mixed light image, a is the coefficient of f(i,j), and b is a constant.

[0082] S4022. Perform image enhancement on the fluorescence intensity image and the second mixed light image to obtain a third mixed light image.

[0083] In one embodiment of the present application, the step of performing image enhancement on the fluorescence intensity image and the second mixed light mixed light image includes:

[0084] According to the formula Performing image enhancement on the fluorescence intensity image and the second mixed light mixed light image;

[0085] Among them, g(i,j) is the grayscale value of position i,j in the third mixed light image, f2(i,j) is the grayscale value of position i,j in the second mixed light image, f1(i,j) is the grayscale value of position i,j in the fluorescence intensity image, and B is a constant.

[0086] S4023. Perform image noise reduction on the third mixed light image to obtain a visible fluorescence image.

[0087] In an embodiment of the present application, performing image noise reduction on the third mixed light image includes:

[0088] According to the formula performing image noise reduction on the third mixed light image;

[0089] Among them, J(i,j) is the grayscale value of position i,j in the visible fluorescence image, g(i,j) is the grayscale value of position i,j in the third mixed light image, A is a constant, t(i,j) is the transmittance, t0 is the degree of image noise reduction, w is a constant coefficient, and w can be taken as 0.95.

[0090] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0095] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, or device including the element.

[0097] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0098] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0099] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0100] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A fluorescent molecular imaging system, characterized in that: include: A multi-spectral light source excitation module, used to synchronously emit visible light and near-infrared light to a target detection area containing a near-infrared fluorescent contrast agent, wherein the target detection area includes normal biological tissue and tumor tissue; An optical detector module, used for collecting a mixed light image of the target detection area; An image processing module, used to generate a visible fluorescence image according to the mixed light image, comprising: Generate a fluorescence intensity image from the image of the mixed light image within a preset pixel range; generate a second mixed light image from the image of the mixed light image outside the preset pixel range after adaptive brightness conversion; combine the fluorescence intensity image and the second mixed light image into a fluorescence image part and a visible light image part of the visible fluorescence image; the fluorescence image part is used to obtain tumor interest area information of the target detection area, and the visible light image part is used to obtain environmental information of the target detection area; A display module, used for displaying the visible fluorescent image in real time; Wherein, the optical detector module comprises: An imaging lens, used for acquiring a first mixed light signal of the target detection area; A filter, used for filtering out the near-infrared emission light in the first mixed light signal to obtain a second mixed light signal; The photoelectric conversion module is used to convert the second mixed light signal into a mixed light image.

2. The fluorescent molecular imaging system according to claim 1, characterized in that: The photoelectric conversion module includes any one of the following: Charge coupled device; Complementary Metal Oxide Semiconductor Image Sensor.

3. The fluorescent molecular imaging system according to claim 1, characterized in that: The multi-spectral light source excitation module includes: an LED white light source and a near-infrared laser light source.

4. The fluorescent molecular imaging system according to claim 3, characterized in that: The LED white light source and the near-infrared laser light source are close to each other and are independently arranged.

5. The fluorescent molecular imaging system according to claim 1, characterized in that: The image processing module comprises: An image acquisition unit, used to acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent; An image synthesis unit is used to generate a visible fluorescent image according to the mixed light image.

6. An image processing method for the fluorescent molecular imaging system according to claim 1, characterized in that: include: Acquire a mixed light image of a target detection area; the target detection area contains a near-infrared fluorescent contrast agent; A visible fluorescent image is generated according to the mixed light image.

7. The image processing method according to claim 6, characterized in that: The generating a visible fluorescent image according to the mixed light image comprises: Extracting a fluorescence intensity image from the mixed light image, and performing adaptive brightness conversion on the mixed light image to obtain a second mixed light image; Performing image enhancement on the fluorescence intensity image and the second mixed light image to obtain a third mixed light image; Perform image noise reduction on the third mixed light image to obtain a visible fluorescence image.

8. The image processing method according to claim 7, characterized in that: The step of extracting a fluorescence intensity image from a mixed light image comprises: According to the formula Extracting a fluorescence intensity image from the mixed light image; in, is the fluorescence intensity image The gray value of the position, For mixed light images The gray value of the position, They are respectively the lower threshold and upper threshold of the preset grayscale range.

9. The image processing method according to claim 7, characterized in that: The step of adaptively converting the brightness of the mixed light image comprises: According to the formula Performing adaptive brightness conversion on the mixed light image to obtain a second mixed light image; in, For mixed light images The gray value of the position, In the second mixed light image The gray value of the position, for The coefficient of is a constant.

10. The image processing method according to claim 7, characterized in that: The step of performing image enhancement on the fluorescence intensity image and the second mixed light image comprises: According to the formula Performing image enhancement on the fluorescence intensity image and the second mixed light image; in, In the third mixed light image The gray value of the position, In the second mixed light image The gray value of the position, Fluorescence intensity image The gray value of the position, is a constant.

11. The image processing method according to claim 7, characterized in that: The performing image noise reduction on the third mixed light image includes: According to the formula performing image noise reduction on the third mixed light image; in, For visible fluorescence images The gray value of the position, In the third mixed light image The gray value of the position, is a constant, is the transmittance, is the degree of image noise reduction, is a constant coefficient.

12. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to claim 6 is implemented.

Images