White light background image fused medical dual-fluorescence imaging method and device

Abstract

The invention provides a medical dual-fluorescence imaging method and device fusing white light background images, and the method comprises the steps: setting the light-emitting logic of a fusion light source, irradiating an imaging region according to the light-emitting logic, carrying out the time-sharing collection of white light, first long-wavelength fluorescence, short-wavelength fluorescence and second long-wavelength fluorescence, and carrying out the light splitting of the white light, the first long-wavelength fluorescence, the short-wavelength fluorescence and the second long-wavelength fluorescence, and performing time-sharing acquisition on the white light and the short-wavelength fluorescence by using the first camera, performing time-sharing acquisition on the first long-wavelength fluorescence and the second long-wavelength fluorescence by using the second camera, and performing image fusion processing on the white light image data, the first long-wavelength fluorescence image data, the short-wavelength fluorescence image data and the second long-wavelength fluorescence image data to obtain a fused image. Short-wavelength fluorescent light and white light are output in the same path and long-wavelength fluorescent light is independently output by designing a mode of combining time-sharing light emitting with prism light splitting, so that white light background images are fused during double-fluorescence simultaneous development imaging, the prism structure is relatively simple, fluorescence signal crosstalk can be avoided, and the overall fused image is bright and clear in color.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a medical dual-fluorescence imaging method and device integrating a white light background image. Background Art

[0002] With the development of modern medicine, the application of fluorescence technology in the medical field is becoming more and more extensive and playing an increasingly important role. Through fluorescence technology, DNA sequencing, lymph and tumor tracing, drug metabolism and pesticide residue detection can be achieved. Fluorescence technology mainly collects fluorescence for fluorescence development, thereby tracing drugs, tumors, etc. However, since fluorescence development can usually only display fluorescent areas, other tissue areas and instrument positions cannot be developed, most of the current fluorescence surgical navigation imaging equipment will provide a fusion image of white light and fluorescence, so that doctors can easily identify fluorescent development areas, instruments and other tissues. Currently, there are usually three ways to achieve single fluorescence imaging fused with white light. The first way is to use a dichroic prism, such as using a double prism to split the fluorescence image and the white light image into two paths, corresponding to fluorescence and white light respectively, and then use two cameras to collect the fluorescence signal and the white light signal respectively, and then display the image after image fusion; the second way is to collect white light and fluorescence images by time-sharing, fuse the two images into one frame, and then display the image; the third way is to use a Bayer camera (a color camera) to collect the mixed signal of white light and fluorescence and then perform color gamut conversion, specifically, to identify the fluorescence by the color corresponding to the fluorescence in the color gamut space, and then display the image after image fusion.

[0003] However, due to the different characteristics of fluorescent drugs, some surgeries require the use of two fluorescent drugs in combination. The existing dual-fluorescence imaging scheme is improved based on the above-mentioned single fluorescence imaging method of fusion white light. The difference is that the white light is replaced by another fluorescence. Obviously, such a dual-fluorescence imaging scheme cannot simultaneously meet the needs of fluorescence development area, instruments and other tissue development. However, if the above-mentioned single fluorescence imaging method of fusion of white light is still used to fuse the dual fluorescence and white light images, the following problems will arise: Based on the above-mentioned method one, it is necessary to add a white light prism on the basis of the dual prism, at least a three-prism or even a five-prism, which makes the design and production of the dichroic prism very complicated; Based on the above-mentioned method two, when one of the fluorescent drugs is excited, the spectrum with a relative excitation peak blue shift (deviation to the blue light direction) will also excite fluorescence, and even if the white light is not fused, crosstalk between the fluorescences will occur, causing interference to fluorescence recognition; Based on the above-mentioned method three, since the fluorescence signal itself is very weak and easily "submerged" in the white light signal, thereby affecting the color recognition corresponding to the two fluorescences, fluorescence recognition is performed by lowering the brightness of the white light, but the fused image obtained in this way will be dark and not bright enough as a whole, and some white light image details may even be unclear, resulting in poor picture quality and detail clarity.

[0004] Therefore, it is necessary to design a medical dual-fluorescence imaging method and device that can fuse the white light background image, make the prism structure relatively simple, avoid fluorescence signal interference, and make the overall fused image bright and clear. Summary of the invention

[0005] The present invention provides a medical dual-fluorescence imaging method and device for fusing a white light background image, which are mainly used to solve the problems that the existing dual-fluorescence imaging cannot fuse the white light background image, the prism design and production based on the traditional white light fusion solution are complex, there is fluorescence signal interference and the fused image quality is poor, so as to achieve the effect of fusing the white light background image when effectively performing dual fluorescence simultaneous development imaging, the prism structure is relatively simple, the fluorescence signal interference can be avoided, and the overall fused image is bright and clear.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] A medical dual fluorescence imaging method integrating a white light background image, comprising:

[0008] S1: Setting the light emitting logic of the fused light source and irradiating the imaging area according to the light emitting logic, wherein the light emitting logic is: white light and long-wavelength excitation light are emitted simultaneously, and are emitted in a time-sharing manner with short-wavelength excitation light.

[0009] S2: simultaneously collecting the reflected white light and the first long-wavelength fluorescence excited by the long-wavelength excitation light, and splitting the white light and the first long-wavelength fluorescence by using a beam splitter prism.

[0010] S3: simultaneously collecting the short-wavelength fluorescence and the second long-wavelength fluorescence excited by the short-wavelength excitation light, and using the beam splitting prism to split the short-wavelength fluorescence and the second long-wavelength fluorescence.

[0011] Steps S2 and S3 are performed in time division according to the illumination light currently received by the imaging area.

[0012] S4: using the first camera and the second camera to collect the white light and the first long-wavelength fluorescence respectively, and obtaining white light image data and the first long-wavelength fluorescence image data respectively.

[0013] S5: using the first camera and the second camera to collect the short-wavelength fluorescence and the second long-wavelength fluorescence respectively, and obtaining short-wavelength fluorescence image data and the second long-wavelength fluorescence image data respectively.

[0014] Steps S4 and S5 are performed in different time periods according to the current light splitting conditions of the light splitting prism.

[0015] S6: performing image fusion processing on the white light image data, the first long wavelength fluorescence image data, the short wavelength fluorescence image data and the second long wavelength fluorescence image data to obtain a fused image.

[0016] A further solution is that the setting of the splitting logic in step S1 includes the following steps:

[0017] S11: The fusion light source emits the white light and the long-wavelength excitation light, and is turned off after emission.

[0018] S12: Then, the short-wavelength excitation light is emitted, and then turned off after emission.

[0019] S13: Repeat steps S11-S12 to alternately emit light.

[0020] A further solution is that the light collection in step S2 includes the following steps:

[0021] S21: The white light and the long-wavelength excitation light are transmitted to the vicinity of the lens through the light guide, the long-wavelength excitation light illuminates the imaging area and excites the first long-wavelength fluorescence, and the white light is reflected.

[0022] S22: The lens collects the white light and the first long-wavelength fluorescence, and projects them to the beam splitter prism.

[0023] S23: The beam splitter prism projects the white light to the first camera through a first optical path, and projects the first long-wavelength fluorescence to the second camera through a second optical path.

[0024] A further solution is that the light collection in step S3 includes the following steps:

[0025] S31: The short-wavelength excitation light is transmitted to the vicinity of the lens through the light guide, and the short-wavelength excitation light illuminates the imaging area and excites the short-wavelength fluorescence and the second long-wavelength fluorescence.

[0026] S32: The lens collects the short-wavelength fluorescence and the second long-wavelength fluorescence, and projects them to the beam splitter prism.

[0027] S33: The beam splitter prism projects the short-wavelength fluorescence to the first camera through a first optical path, and projects the second long-wavelength fluorescence to the second camera through a second optical path.

[0028] A further solution is to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence through a filter arranged at the front end of the lens, and to cut off the long-wavelength excitation light or the short-wavelength excitation light.

[0029] A further solution is that the light-sharing acquisition of the first camera and the second camera in steps S4 and S5 further includes the following steps:

[0030] The light collection logic of the first camera and the second camera are respectively set and light collection is performed according to the light collection logic, wherein the first camera collects the white light and the short-wavelength fluorescence in a time-sharing manner, and the second camera collects the first long-wavelength fluorescence and the second long-wavelength fluorescence in a time-sharing manner.

[0031] A further solution is that the image data processing in step S6 includes:

[0032] S61: performing image addition processing on the first long-wavelength fluorescence image data and the second long-wavelength fluorescence image data to obtain long-wavelength fluorescence image data.

[0033] S62: performing image enhancement on the white light image data, the short wavelength fluorescence image data and the long wavelength fluorescence image data respectively, and performing image registration and image fusion to obtain the fused image.

[0034] A further solution is that the imaging region is a near-infrared spectral region, in which a mixed first fluorescent drug and a second fluorescent drug are placed, and the excitation peak of the first fluorescent drug is greater than the excitation peak of the second fluorescent drug.

[0035] The first long-wavelength fluorescence and the second long-wavelength fluorescence are both fluorescence generated after the first fluorescent drug is excited, and the second long-wavelength fluorescence is weaker than the first long-wavelength fluorescence.

[0036] The wavelength of the long-wavelength excitation light is near the excitation peak of the first fluorescent drug, and the wavelength of the short-wavelength excitation light is near the excitation peak of the second fluorescent drug.

[0037] A medical dual fluorescence imaging device integrating a white light background image, using the medical dual fluorescence imaging method integrating a white light background image, comprising:

[0038] A main control module, a fusion light source module, a first camera, a second camera, a lens and a spectroscopic prism, wherein the main control module establishes a communication connection with the fusion light source module, and respectively establishes data communication connections with the first camera and the second camera, the main control module outputs a light emission instruction to the fusion light source module, and respectively sends a collection instruction to the first camera and the second camera; the fusion light source module is pre-set with a light emission logic for emitting white light and long-wavelength excitation light or a single short-wavelength excitation light to an imaging area according to the light emission instruction and the light emission logic; the lens is placed above the imaging area for time-sharing collection and output of the reflected light of the white light and the excited first long-wavelength fluorescence, the excited short-wavelength fluorescence and the second long-wavelength fluorescence; the spectroscopic prism is placed in the light emission direction of the lens, and is provided with a first light path and a second light path for time-sharing reception of the reflected light of the white light and the excited first long-wavelength fluorescence, the excited short-wavelength fluorescence and the second long-wavelength fluorescence. The white light and the first long-wavelength fluorescence, the short-wavelength fluorescence and the second long-wavelength fluorescence are projected to the first camera through the first optical path in a time-sharing manner, and the first long-wavelength fluorescence and the second long-wavelength fluorescence are projected to the second camera in a time-sharing manner through the second optical path; the first camera is used to perform signal conversion and processing on the white light and the short-wavelength fluorescence in a time-sharing manner, and output white light image data and short-wavelength fluorescence data to the main control module, the second camera is used to perform signal conversion and processing on the first long-wavelength fluorescence and the second long-wavelength fluorescence in a time-sharing manner, and output first long-wavelength fluorescence data and second long-wavelength fluorescence data to the main control module; the main control module is used to perform image processing on the white light image data, the short-wavelength fluorescence data, the first long-wavelength fluorescence data and the second long-wavelength fluorescence data, and output a fused image.

[0039] A further solution is to further include a filter, which is placed at the front end of the lens and is used to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence, with a transmittance greater than 85%; and is used to cut off the long-wavelength excitation light or the short-wavelength excitation light, with a cutoff degree greater than OD4.

[0040] It can be seen that the present invention has the following beneficial effects:

[0041] 1. The present invention emits white light and long-wavelength fluorescence excitation light together according to a preset light emission sequence, and emits them alternately with short-wavelength excitation light in a time-sharing manner, and performs time-sharing, light splitting and collection. Specifically, a beam splitter prism is used to split white light and long-wavelength fluorescence, and then the two cameras collect them respectively. Then, short-wavelength fluorescence and weaker long-wavelength fluorescence stimulated by a single short-wavelength excitation light are split and then collected respectively by the two cameras. The white light and short-wavelength fluorescence are collected by the same camera to generate corresponding image data, and the two long-wavelength fluorescence collected successively are collected by the same camera to generate corresponding image data, thereby realizing the fusion of the white light background image during the simultaneous development and imaging of dual fluorescence, so as to meet the needs of the fluorescence development area, instruments and other tissue development, so that doctors can identify the fluorescence development area, instruments and other tissues, which is convenient for operation.

[0042] 2. The present invention only needs to design a double prism with two light paths to achieve light splitting, and can be implemented based on the currently common double prism and four-prism single fluorescence surgical navigation imaging equipment. Compared with the existing prism design scheme, the structure and production are relatively simple.

[0043] 3. The present invention can avoid crosstalk between fluorescence signals to the greatest extent by splitting the short-wavelength fluorescence and the weaker long-wavelength fluorescence stimulated by the short-wavelength excitation light and using two cameras to collect them respectively.

[0044] 4. The present invention uses a prism to split the light and adopts two cameras to collect short-wave and long-wave light signals respectively according to the collection sequence, thereby avoiding the problem of dark images and poor clarity caused by lowering the brightness of white light by existing color cameras, and can maintain the brightness of the fused image that is consistent with the currently common dual-prism and four-prism single fluorescence surgical navigation imaging equipment.

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the medical dual fluorescence imaging method of the present invention integrating white light background image.

[0047] Figure 2 It is a schematic diagram of the process of integrating the time-sharing light emission of the light source and the time-sharing data collection of the camera in the present invention.

[0048] Figure 3 It is a schematic diagram of light splitting by the light splitting prism of the present invention.

[0049] Figure 4 It is a schematic diagram of the transmittance enhancement and cutoff spectrum of the optical filter of the present invention.

[0050] Figure 5 It is a schematic diagram of a medical dual-fluorescence imaging device that fuses a white light background image according to the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] A medical dual fluorescence imaging method and device embodiment integrating white light background image

[0053] See also Figure 1 The present invention relates to a medical dual-fluorescence imaging method for integrating a white light background image, comprising:

[0054] S1: Setting the light emitting logic of the fused light source and irradiating the imaging area according to the light emitting logic, wherein the light emitting logic is: white light and long-wavelength excitation light are emitted simultaneously, and are emitted in a time-sharing manner with short-wavelength excitation light.

[0055] S2: simultaneously collecting the reflected white light and the first long-wavelength fluorescence excited by the long-wavelength excitation light, and splitting the white light and the first long-wavelength fluorescence by using a beam splitter prism.

[0056] S3: simultaneously collecting the short-wavelength fluorescence and the second long-wavelength fluorescence excited by the short-wavelength excitation light, and using the beam splitting prism to split the short-wavelength fluorescence and the second long-wavelength fluorescence.

[0057] Steps S2 and S3 are performed in time division according to the illumination light currently received by the imaging area.

[0058] S4: using the first camera and the second camera to collect the white light and the first long-wavelength fluorescence respectively, and obtaining white light image data and the first long-wavelength fluorescence image data respectively.

[0059] S5: using the first camera and the second camera to collect the short-wavelength fluorescence and the second long-wavelength fluorescence respectively, and obtaining short-wavelength fluorescence image data and the second long-wavelength fluorescence image data respectively.

[0060] Steps S4 and S5 are performed in different time periods according to the current light splitting conditions of the light splitting prism.

[0061] S6: performing image fusion processing on the white light image data, the first long wavelength fluorescence image data, the short wavelength fluorescence image data and the second long wavelength fluorescence image data to obtain a fused image.

[0062] Specifically, this embodiment takes the scenario where two fluorescent drugs, indocyanine green and methylene blue, are used together as an example to illustrate:

[0063] The excitation peak of the indocyanine green (ICG) is about 780nm, and the emission peak is about 830nm, that is, the fluorescence generated by the indocyanine green when it is excited is long-wavelength fluorescence; the excitation peak of the methylene blue (MB) is about 665nm, and the emission peak is about 688nm, that is, the fluorescence generated by the methylene blue when it is excited is short-wavelength fluorescence; these two fluorescent drugs are both in the near-infrared spectral band.

[0064] Among them, indocyanine green is commonly used for tracing blood vessels, lymph nodes, and tumors, and observing anastomotic fistulas during surgery. It is injected through veins, lymphatic vessels, and around tumors, metabolized by the liver and intestines, and finally excreted from the body in the form of feces. Therefore, it has disadvantages in ureteral imaging. However, most of methylene blue is metabolized by the kidneys after intravenous injection, and finally excreted from the body in the form of urine, which can be used for ureteral imaging. Therefore, in some urological surgeries, it is necessary to inject indocyanine green and methylene blue at the same time, and use the two drugs together for dual fluorescence imaging.

[0065] In this embodiment, the setting of the splitting logic in step S1 includes the following steps:

[0066] S11: The fusion light source emits the white light and the long-wavelength excitation light, and is turned off after emission.

[0067] S12: Then, the short-wavelength excitation light is emitted, and then turned off after emission.

[0068] S13: Repeat steps S11-S12 to alternately emit light.

[0069] See also Figure 2 Specifically, in this embodiment, the light emitting logic of the fused light source corresponds to a set of light emitting sequences, and the light collection logic of the first camera and the second camera respectively corresponds to a set of light collection sequences, and the light emitting sequences and the light collection sequences correspond to each other in the time series of acquiring corresponding image data.

[0070] For example: in the acquisition of the first frame of the fused image, according to the light emission sequence of the fused light source, white light and indocyanine green excitation light are first emitted, and the indocyanine green excitation light is the above-mentioned long-wavelength excitation light. At this time, the white light will be reflected, and the indocyanine green will be excited by the long-wavelength excitation light to emit indocyanine green fluorescence, and the indocyanine green fluorescence is the above-mentioned first long-wavelength fluorescence. Correspondingly, at this time, the first camera collects the white light after the spectroscopic analysis according to its light collection sequence, and the second camera collects the first long-wavelength fluorescence after the spectroscopic analysis according to its light collection sequence, and generates white light image data and first long-wavelength fluorescence image data respectively after data processing; then, after turning off the above white light and long-wavelength excitation light, the fused light source emits the excitation light of methylene blue alone, and the excitation light of methylene blue is the above short-wavelength excitation light. At this time, the methylene blue and indocyanine green are excited by the short-wavelength excitation light, and emit short-wavelength fluorescence and second long-wavelength fluorescence respectively. Correspondingly, at this time, the first camera collects the short-wavelength fluorescence after the spectroscopic analysis according to its light collection sequence, and the second camera collects the second long-wavelength fluorescence after the spectroscopic analysis according to its light collection sequence, and generates short-wavelength fluorescence image data and second long-wavelength fluorescence image data respectively after data processing. By performing image fusion processing on the above white light image data, the first long-wavelength fluorescence image data, the short-wavelength fluorescence image data and the second long-wavelength fluorescence image data, a fused image of this frame is obtained. After that, the acquisition of each frame of the fused image is obtained by repeating the above process according to the sequence.

[0071] In this embodiment, the light collection in step S2 includes the following steps:

[0072] S21: The white light and the long-wavelength excitation light are transmitted to the vicinity of the lens through the light guide, the long-wavelength excitation light illuminates the imaging area and excites the first long-wavelength fluorescence, and the white light is reflected.

[0073] S22: The lens collects the white light and the first long-wavelength fluorescence, and projects them to the beam splitter prism.

[0074] S23: The beam splitter prism projects the white light to the first camera through a first optical path, and projects the first long-wavelength fluorescence to the second camera through a second optical path.

[0075] See also Figure 3 Specifically, the spectroscopic prism described in this embodiment adopts a double prism, the first light path is used to transmit light with a wavelength of 420nm-720nm, the second light path is used to reflect light with a wavelength of 800nm-900nm, and the light output interval between the first light path and the second light path is about 80nm.

[0076] Specifically, in this embodiment, the fusion light source adopts LED white light and excitation light with a wavelength of 785nm. The wavelength of the LED white light is 450-460nm, wherein a spectrum close to the excitation peak of indocyanine green (about 780nm) is used as the excitation light. Under the excitation light of this wavelength, indocyanine green can absorb the maximum amount of light energy and reach the highest excited state, thereby being able to stimulate a higher emission light intensity with the lowest possible excitation light power. At this time, the excitation efficiency of the excitation light is the highest.

[0077] Next, after the imaging area is irradiated with LED white light and 785nm excitation light, the LED white light is reflected, and indocyanine green is excited near its excitation peak to emit a relatively strong fluorescence with a wavelength of about 840nm, that is, the above-mentioned first long-wavelength fluorescence. The LED white light and the first long-wavelength fluorescence are collected by the lens and projected onto the double prism, and the double prism is used to perform spectral separation on the refraction characteristics of the light waves, specifically, the LED white light is transmitted through the first light path, and the first long-wavelength fluorescence is reflected through the second light path.

[0078] In this embodiment, the light collection in step S3 includes the following steps:

[0079] S31: The short-wavelength excitation light is transmitted to the vicinity of the lens through the light guide, and the short-wavelength excitation light illuminates the imaging area and excites the short-wavelength fluorescence and the second long-wavelength fluorescence.

[0080] S32: The lens collects the short-wavelength fluorescence and the second long-wavelength fluorescence, and projects them to the beam splitter prism.

[0081] S33: The beam splitter prism projects the short-wavelength fluorescence to the first camera through a first optical path, and projects the second long-wavelength fluorescence to the second camera through a second optical path.

[0082] Specifically, in this embodiment, the fusion light source adopts 660nm excitation light, wherein a spectrum close to the excitation peak of methylene blue (about 665nm) is used as the excitation light. Under the excitation light of this wavelength, methylene blue can absorb the maximum amount of light energy and reach the highest excited state, thereby being able to achieve a higher emission light intensity with the lowest possible excitation light power. At this time, the excitation efficiency of the excitation light is the highest.

[0083] Next, after the imaging area is irradiated with an excitation light of 660nm, methylene blue is excited near its excitation peak to emit fluorescence with a wavelength of about 694nm, i.e., the above-mentioned short-wavelength fluorescence. At the same time, indocyanine green is also excited to emit a weaker fluorescence with a wavelength of about 840nm, i.e., the above-mentioned second long-wavelength fluorescence. The short-wavelength fluorescence and the second long-wavelength fluorescence are collected by the lens and projected onto the biprism. The biprism is used to perform spectral separation on the refraction characteristics of the light waves, specifically, the short-wavelength fluorescence is transmitted through the first light path, and the second long-wavelength fluorescence is reflected through the second light path.

[0084] In this embodiment, the filter provided at the front end of the lens is used to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence, and to cut off the long-wavelength excitation light or the short-wavelength excitation light.

[0085] See also Figure 4 Specifically, the filter of this embodiment can selectively transmit or reflect light of different wavelengths, and the cutoff band transmittance for excitation light of 660nm and 785nm is less than 0.01%, that is, the corresponding cutoff effect reaches above OD4, and the transmittance for the above-mentioned LED white light, long-wavelength fluorescence and short-wavelength fluorescence reaches above 85%, which can effectively transmit white light and fluorescence and filter out the interference of excitation light.

[0086] In this embodiment, the light-sharing acquisition of the first camera and the second camera in steps S4 and S5 further includes the following steps:

[0087] The light collection logic of the first camera and the second camera are respectively set and light collection is performed according to the light collection logic, wherein the first camera collects the white light and the short-wavelength fluorescence in a time-sharing manner, and the second camera collects the first long-wavelength fluorescence and the second long-wavelength fluorescence in a time-sharing manner.

[0088] In this embodiment, the image data processing in step S6 includes:

[0089] S61: performing image addition processing on the first long-wavelength fluorescence image data and the second long-wavelength fluorescence image data to obtain long-wavelength fluorescence image data.

[0090] S62: performing image enhancement on the white light image data, the short wavelength fluorescence image data and the long wavelength fluorescence image data respectively, and performing image registration and image fusion to obtain the fused image.

[0091] In this embodiment, the imaging region is a near-infrared spectral region, in which a mixed first fluorescent drug and a second fluorescent drug are placed, and the excitation peak of the first fluorescent drug is greater than the excitation peak of the second fluorescent drug.

[0092] The first long-wavelength fluorescence and the second long-wavelength fluorescence are both fluorescence generated after the first fluorescent drug is excited, and the second long-wavelength fluorescence is weaker than the first long-wavelength fluorescence.

[0093] The wavelength of the long-wavelength excitation light is near the excitation peak of the first fluorescent drug, and the wavelength of the short-wavelength excitation light is near the excitation peak of the second fluorescent drug.

[0094] Specifically, in this embodiment, the first fluorescent drug is indocyanine green, whose excitation peak is about 780nm, and the second fluorescent drug is methylene blue, whose excitation peak is about 665nm. The first long-wavelength fluorescence is a stronger fluorescence excited by indocyanine green after irradiation with excitation light of 785nm, and the second long-wavelength fluorescence is a weaker fluorescence excited by indocyanine green after irradiation with excitation light of 660nm. The wavelength of the long-wavelength excitation light is 785nm, which is near the excitation peak of indocyanine green at 780nm, and the wavelength of the short-wavelength excitation light is 660nm, which is near the excitation peak of methylene blue at 665nm.

[0095] Specifically, the above scenario of using two fluorescent drugs, indocyanine green and methylene blue, together in this embodiment is only illustrative, and not the only way. Different fluorescent drugs can be selected according to the application scenario, and excitation light near each excitation peak can be selected according to the fluorescent drugs for fluorescence excitation. The prism can be designed according to the principle of spectral separation of the wavelength of the excited fluorescence and the refraction characteristics of the prism for light waves to better achieve light splitting.

[0096] Embodiment of a medical dual fluorescence imaging device integrating white light background image

[0097] See also Figure 5 A medical dual fluorescence imaging device integrating a white light background image, and a medical dual fluorescence imaging method integrating a white light background image, comprising:

[0098] A main control module 10, a fusion light source module 20, a first camera 30, a second camera 40, a lens 50 and a spectroscopic prism 60. The main control module 10 establishes a communication connection with the fusion light source module 20, and respectively establishes data communication connections with the first camera 30 and the second camera 40. The main control module 10 outputs a light emission instruction to the fusion light source module 20, and respectively sends a collection instruction to the first camera 30 and the second camera 40; the fusion light source module 20 is pre-set with a light emission logic for emitting white light and long-wavelength excitation light or a single short-wavelength excitation light to the imaging area 200 according to the light emission instruction and the light emission logic; the lens 50 is placed above the imaging area 200, for time-sharing collection and output of the reflected light of the white light and the excited first long-wavelength fluorescence, the excited short-wavelength fluorescence and the second long-wavelength fluorescence; the spectroscopic prism 60 is placed in the light emission direction of the lens 50, and is provided with a first optical path and a second The optical path is used to receive the white light and the first long wavelength fluorescence, the short wavelength fluorescence and the second long wavelength fluorescence in a time-sharing manner, and to project the white light and the short wavelength fluorescence to the first camera 30 in a time-sharing manner through the first optical path, and to project the first long wavelength fluorescence and the second long wavelength fluorescence to the second camera 40 in a time-sharing manner through the second optical path; the first camera 30 is used to perform signal conversion and processing on the white light and the short wavelength fluorescence in a time-sharing manner, and output white light image data and short wavelength fluorescence data to the main control module 10, and the second camera 40 is used to perform signal conversion and processing on the first long wavelength fluorescence and the second long wavelength fluorescence in a time-sharing manner, and output first long wavelength fluorescence data and second long wavelength fluorescence data to the main control module 10; the main control module 10 is used to perform image processing on the white light image data, the short wavelength fluorescence data, the first long wavelength fluorescence data and the second long wavelength fluorescence data, and output a fused image.

[0099] Specifically, the main control module 10 of this embodiment includes at least an interface module, a communication module, an image processor GPU, a central processing unit CPU and peripheral circuits. A data communication connection is established with the first camera 30 and the second camera 40 through the interface module for image data reception and communication; RS232 serial communication is established with the fusion light source module 20 through the communication module; the image processor GPU is used for image data calculation and image display output, and the central processing unit CPU is used for logic control.

[0100] Specifically, the fusion light source module 20 of this embodiment includes a light source generating module, a plurality of control switches, a light combining module and a light output module, wherein the light source generating module is used to emit an excitation light source, and the excitation light source may be a laser light source, or a monochromatic LED light source or other light source, so as to meet the requirements of the excitation spectrum range, the excitation light intensity and the light emission timing control; the plurality of control switches are used to control the emission or shutoff of white light and long-wavelength excitation light, and the emission or shutoff of short-wavelength excitation light, respectively, and the light combining module is used to combine the optical paths of the white light, long-wavelength excitation light and short-wavelength excitation light into one, and output through one optical path; the light output module includes a light output interface, a light guide and a beam expander, and the light output interface adopts an SMA905 interface to output light to ensure the effective transmission of the optical signal; the light guide is used to efficiently transmit the white light and long-wavelength excitation light or a single short-wavelength excitation light to the vicinity of the lens 50; the beam expander is arranged on the light output port, and is used to expand the diameter of the laser beam and reduce the divergence angle of the laser beam, so that the laser can be transmitted to the imaging area 200 in a more concentrated manner.

[0101] Specifically, the main control module 10 and the fusion light source module 20 of this embodiment can be independent devices that communicate through external interfaces; or they can be integrated into the same device and communicate through internal interfaces. For example, the main control module 10 can be integrated into a computer host, and the fusion module is integrated into an independent device. The host can be based on an operating system such as Windows or Linux. The data processing and calculation can be based on a CPU, GPU or FPGA, equipped with an independent graphics card, connected to the first camera 30 and the second camera 40 through the USB interface of the host to receive and communicate image data; connected to the fusion light source module 20 through its RS232 serial communication interface to establish serial communication; connected to the display device 300 through an HDMI interface to display the fused image.

[0102] Specifically, the first camera 30 and the second camera 40 of this embodiment use the same type of CMOS Mono cameras, which are low in cost and have higher quantum efficiency than Bayer cameras. They are integrated with a variety of peripheral interfaces such as USB interface and BNC interface, and are connected to the host through the USB interface for data transmission and communication, and are externally triggered through the BNC interface. The camera frame rate is greater than 120fps to meet the 60fps video output of the entire device; its minimum exposure time is 10us, which is convenient for overexposure adjustment.

[0103] Specifically, the lens 50 of this embodiment adopts an infrared anti-reflection 8mm fixed-focus C-mount lens, which ensures that a certain field of view is met while facilitating more effective collection of near-infrared fluorescence.

[0104] Specifically, the spectroscopic prism 60 of this embodiment adopts a double prism, the first light path is used to transmit light with a wavelength of 420nm-720nm, the second light path is used to reflect light with a wavelength of 800nm-900nm, and the light output interval between the first light path and the second light path is about 80nm, so as to better avoid interference between the two light paths.

[0105] Specifically, the use of a double prism in this embodiment is only exemplary and not the only way. For example, the beam splitter prism 60 can use a four-prism. When white light and the first long-wavelength fluorescence are incident, the white light is split into three light outputs of R, G, and B (three primary colors of red, green, and blue), and the first long-wavelength fluorescence is output as one light output; when short-wavelength fluorescence is incident, the short-wavelength fluorescence is output using the same light path as the above-mentioned R light, and the second long-wavelength fluorescence is output using the same light path as the above-mentioned first long-wavelength fluorescence.

[0106] In this embodiment, a filter 70 is also included. The filter 70 is placed at the front end of the lens and is used to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence, with a transmittance greater than 85%; and is used to cut off the long-wavelength excitation light or the short-wavelength excitation light, with a cutoff degree greater than OD4.

[0107] Specifically, the filter 70 of this embodiment can be installed in front of the lens 50, or it can be simplified in design and divided into two or four pieces, which are installed between the beam splitter prism 60 and the first camera 30 and between the beam splitter prism 60 and the second camera 40 respectively.

[0108] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A medical dual fluorescence imaging method integrating a white light background image, characterized in that: include: S1: Setting the light emission logic of the fusion light source and irradiating the imaging area according to the light emission logic, wherein the light emission logic is: white light and long-wavelength excitation light are emitted simultaneously, and are emitted in a time-sharing manner with short-wavelength excitation light; S2: simultaneously collecting the reflected white light and the first long-wavelength fluorescence excited by the long-wavelength excitation light, and splitting the white light and the first long-wavelength fluorescence by using a beam splitter prism; S3: simultaneously collecting the short-wavelength fluorescence and the second long-wavelength fluorescence excited by the short-wavelength excitation light, and splitting the short-wavelength fluorescence and the second long-wavelength fluorescence by using the beam splitting prism; Steps S2 and S3 are performed in different time periods according to the illumination light currently received by the imaging area; S4: using the first camera and the second camera to collect the white light and the first long-wavelength fluorescence respectively, and obtaining white light image data and the first long-wavelength fluorescence image data respectively; S5: using the first camera and the second camera to collect the short-wavelength fluorescence and the second long-wavelength fluorescence respectively, and obtaining short-wavelength fluorescence image data and the second long-wavelength fluorescence image data respectively; Steps S4 and S5 are performed in different time periods according to the current splitting conditions of the splitting prism; S6: performing image fusion processing on the white light image data, the first long wavelength fluorescence image data, the short wavelength fluorescence image data and the second long wavelength fluorescence image data to obtain a fused image.

2. The medical dual fluorescence imaging method for fusion of white light background image according to claim 1, characterized in that: The setting of the splitting logic in step S1 includes the following steps: S11: the fusion light source emits the white light and the long-wavelength excitation light, and turns off after emitting; S12: then emitting the short-wavelength excitation light, and turning it off after emission; S13: Repeat steps S11-S12 to alternately emit light.

3. The medical dual fluorescence imaging method for fusion of white light background image according to claim 1, characterized in that: The light collection in step S2 includes the following steps: S21: The white light and the long-wavelength excitation light are transmitted to the vicinity of the lens through the light guide, the long-wavelength excitation light irradiates the imaging area and excites the first long-wavelength fluorescence, and the white light is reflected; S22: The lens collects the white light and the first long-wavelength fluorescence and projects them to the beam splitter prism; S23: The beam splitter prism projects the white light to the first camera through a first optical path, and projects the first long-wavelength fluorescence to the second camera through a second optical path.

4. The medical dual fluorescence imaging method for fusion of white light background image according to claim 1, characterized in that: The light collection in step S3 includes the following steps: S31: The short-wavelength excitation light is transmitted to the vicinity of the lens through the light guide, and the short-wavelength excitation light irradiates the imaging area and excites the short-wavelength fluorescence and the second long-wavelength fluorescence; S32: The lens collects the short-wavelength fluorescence and the second long-wavelength fluorescence, and projects them to the beam splitter prism; S33: The beam splitter prism projects the short-wavelength fluorescence to the first camera through a first optical path, and projects the second long-wavelength fluorescence to the second camera through a second optical path.

5. The medical dual fluorescence imaging method for fusion of white light background image according to any one of claims 3 to 4, characterized in that: The filter arranged at the front end of the lens is used to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence, and to cut off the long-wavelength excitation light or the short-wavelength excitation light.

6. The medical dual fluorescence imaging method for fusion of white light background image according to claim 1, characterized in that: The light-sharing acquisition of the first camera and the second camera in steps S4 and S5 further includes the following steps: The light collection logic of the first camera and the second camera are respectively set and light collection is performed according to the light collection logic, wherein the first camera collects the white light and the short-wavelength fluorescence in a time-sharing manner, and the second camera collects the first long-wavelength fluorescence and the second long-wavelength fluorescence in a time-sharing manner.

7. The medical dual fluorescence imaging method for fusion of white light background image according to claim 1, characterized in that: The image data processing in step S6 includes: S61: performing image addition processing on the first long-wavelength fluorescence image data and the second long-wavelength fluorescence image data to obtain long-wavelength fluorescence image data; S62: performing image enhancement on the white light image data, the short wavelength fluorescence image data and the long wavelength fluorescence image data respectively, and performing image registration and image fusion to obtain the fused image.

8. The medical dual fluorescence imaging method for fusion of white light background image according to any one of claims 1 to 7, characterized in that: The imaging region is a near-infrared spectral region, in which a mixed first fluorescent drug and a second fluorescent drug are placed, and the excitation peak of the first fluorescent drug is greater than the excitation peak of the second fluorescent drug; The first long-wavelength fluorescence and the second long-wavelength fluorescence are both fluorescence generated after the first fluorescent drug is excited, and the second long-wavelength fluorescence is weaker than the first long-wavelength fluorescence; The wavelength of the long-wavelength excitation light is near the excitation peak of the first fluorescent drug, and the wavelength of the short-wavelength excitation light is near the excitation peak of the second fluorescent drug.

9. A medical dual fluorescence imaging device integrating a white light background image, characterized in that: A medical dual fluorescence imaging method for fusion of a white light background image as described in any one of claims 1 to 8 comprises: A main control module, a fusion light source module, a first camera, a second camera, a lens and a spectroscopic prism, wherein the main control module establishes a communication connection with the fusion light source module, and respectively establishes data communication connections with the first camera and the second camera, the main control module outputs a light emission instruction to the fusion light source module, and respectively sends a collection instruction to the first camera and the second camera; the fusion light source module is pre-set with a light emission logic for emitting white light and long-wavelength excitation light or a single short-wavelength excitation light to an imaging area according to the light emission instruction and the light emission logic; the lens is placed above the imaging area for time-sharing collection and output of the reflected light of the white light and the excited first long-wavelength fluorescence, the excited short-wavelength fluorescence and the second long-wavelength fluorescence; the spectroscopic prism is placed in the light emission direction of the lens, and is provided with a first light path and a second light path for time-sharing reception of the reflected light of the white light and the excited first long-wavelength fluorescence, the excited short-wavelength fluorescence and the second long-wavelength fluorescence. The white light and the first long-wavelength fluorescence, the short-wavelength fluorescence and the second long-wavelength fluorescence are projected to the first camera through the first optical path in a time-sharing manner, and the first long-wavelength fluorescence and the second long-wavelength fluorescence are projected to the second camera in a time-sharing manner through the second optical path; the first camera is used to perform signal conversion and processing on the white light and the short-wavelength fluorescence in a time-sharing manner, and output white light image data and short-wavelength fluorescence data to the main control module, the second camera is used to perform signal conversion and processing on the first long-wavelength fluorescence and the second long-wavelength fluorescence in a time-sharing manner, and output first long-wavelength fluorescence data and second long-wavelength fluorescence data to the main control module; the main control module is used to perform image processing on the white light image data, the short-wavelength fluorescence data, the first long-wavelength fluorescence data and the second long-wavelength fluorescence data, and output a fused image.

10. The medical dual fluorescence imaging method of fusion white light background image according to claim 9, characterized in that: It also includes a filter, which is placed at the front end of the lens and is used to enhance the transmittance of the white light and the first long-wavelength fluorescence, or the short-wavelength fluorescence and the second long-wavelength fluorescence, with a transmittance greater than 85%; and is used to cut off the long-wavelength excitation light or the short-wavelength excitation light, with a cutoff degree greater than OD4.

Images