Near-infrared two-area multi-channel living body macroscopic stereo imaging system and method

Abstract

The invention discloses a near-infrared two-region multi-channel living body macroscopic stereo imaging system and a near-infrared two-region multi-channel living body macroscopic stereo imaging method. According to the invention, visible light and infrared imaging technologies are integrated, and the coaxial dual-channel lens, the illumination and excitation light source and the stereo display system are adopted, so that high-quality stereo images can be captured by crossing different spectral intervals. According to the system, a coaxial dual-channel lens is used for splitting light, so that synchronous acquisition of near-infrared and visible light signals in the same view field is realized; the image quality is enhanced by using a specific image processing algorithm; integrating image information of different spectrums by using an image fusion technology; a stereo display system is adopted, and a final stereo imaging result is presented through a naked eye 3D display technology. In addition, a binocular depth estimation algorithm is adopted to accurately calculate depth information, and more specific target depth information is further provided to help operation. According to the technical scheme, utilization of macroscopic three-dimensional information of living bodies in multiple fields such as medical diagnosis, biological scientific research and industrial detection can be improved.

Description

Technical Field

[0001] The invention belongs to the field of biomedical imaging of applied optics, and relates to a near-infrared two-zone multi-channel living macroscopic stereoscopic imaging system and method. Background Art

[0002] Near-infrared II (NIR-II, 900-1880 nm band) fluorescence imaging is an advanced optical in vivo imaging method. Compared with traditional visible light (360-760nm) and near-infrared I (760-900 nm) imaging technologies, NIR-II imaging effectively reduces light scattering interference in the two-dimensional planar array imaging process due to the low scattering and moderate absorption characteristics of biological tissues for photons in this band, thereby significantly improving the imaging quality.

[0003] Multi-channel imaging technology is an imaging technology that has emerged with the development of optics, computer vision and image processing technology. Compared with single-wavelength window or single-modality image capture in traditional imaging, multi-channel imaging technology can obtain more dimensional information and provide a richer data foundation for the analysis of complex scenes.

[0004] Stereoscopic imaging technology is derived from the natural stereoscopic vision system of human beings. It can observe the same scene from slightly different angles through two eyes, and then analyze the depth information of the scene. It captures the image of an object or scene from two or more angles and reconstructs the three-dimensional structure of the scene using the parallax between the images. Stereoscopic imaging technology has been widely used in many fields such as machine vision, remote sensing, and medical imaging.

[0005] Existing near-infrared two-zone imaging systems used for biomedical imaging are mostly single-channel planar imaging systems, that is, such imaging systems only have a fluorescence channel but lack a visible light channel, which makes it impossible to image tissues other than fluorescent markers; and because they do not have depth resolution capabilities, the imaging cannot well guide users in performing biological operations. Summary of the invention

[0006] The purpose of the present invention is to provide a near-infrared two-zone multi-channel living macroscopic stereoscopic imaging system and method to address the shortcomings of the existing near-infrared two-zone imaging technology. It is used to solve the shortcomings of the current imaging system that cannot analyze the target depth information and lacks a visible light channel; based on a two-way dual-channel system, the calculation and three-dimensional display of the target object's depth information can be achieved.

[0007] In a first aspect, the present invention provides a near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system, the system comprising: Dual-channel image acquisition equipment, lighting and excitation light sources, and stereoscopic display system; The illumination and excitation light sources are facing the biological tissue to be observed, and the fluorescent substance in the biological tissue is excited by the excitation light source to emit a fluorescent signal in the near-infrared region II, while the part other than the fluorescent substance reflects a shadowless lamp signal as the illumination light source; There are two sets of dual-channel image acquisition equipment, each set includes a visible light camera, a near-infrared camera and a coaxial dual-channel lens; in each set of dual-channel image acquisition equipment, the near-infrared second-zone fluorescence and the reflected shadowless lamp signal are split and filtered by the coaxial dual-channel lens, and then synchronously imaged in the near-infrared camera and the visible light camera in the same field of view; The stereoscopic display system is used to process images output by a near-infrared camera and a visible light camera to obtain a multi-channel stereoscopic image with depth information.

[0008] In a second aspect, the present invention provides a real-time multi-channel in vivo macroscopic stereoscopic imaging method, using the above imaging system, comprising the following steps: Step 1: The fluorescent substance in the biological tissue is excited by the excitation light source to emit a fluorescent signal in the near-infrared region II, and at the same time, the part other than the fluorescent substance reflects a shadowless lamp signal as an illumination light source; Step 2: Two sets of dual-channel image acquisition devices are arranged in parallel to collect four signals at the same time; in each set of dual-channel image acquisition devices, the near-infrared second zone fluorescence and reflected shadowless lamp signals are split and filtered by the dual-channel lens, and then synchronously imaged on the near-infrared camera and the visible light camera in the same field of view; Step 3: The real-time image signals of the four cameras are transmitted to the computer, which are the left fluorescence image F L 、Fluorescence right view F R , Visible light left view V L , Visible light right view V R ; Use image enhancement algorithm to enhance the near-fluorescence left and right views F L 、F R Structural information of Step 4: Get the two sets of dual-channel left and right views F L 、V L and F R 、V R Perform image registration separately, give pseudo color to the fluorescence image and fuse it with the visible light image to obtain a dual-channel image FV L 、FV R ; The registered left and right dual channel images FV L 、FV R It is sent to the naked-eye 3D display, and the propagation direction of the left and right images is controlled through the cylindrical lens, so that the viewer's left and right eyes receive different pictures, thus forming parallax and obtaining a three-dimensional image.

[0009] Compared with the prior art, the present invention has the following significant advantages: 1) Compared with conventional single-channel imaging systems, the present invention uses a coaxial dual-channel lens to collect visible light signals and simultaneously image tissues other than fluorescent markers, providing richer biological tissue information; 2) Compared with conventional monocular cameras that can only provide two-dimensional image information, the present invention enables multiple cameras to work in a synchronous mode, and combines image processing algorithms and naked-eye 3D display technology to generate and display multi-channel stereo images with depth information in real time. It provides users with depth perception and achieves more accurate positioning of target tissue structures; 3) This system has precise depth calculation capabilities in both channels and can provide accurate depth information of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the optical path structure of the present invention.

[0011] Figure 2 Flowchart of the method for real-time multi-channel macroscopic in vivo imaging.

[0012] Figure 3 In the embodiment, the dual-channel left view FV L (a) Dual channel right view FV R (b), the mixed output left and right view images (c) (where green is the near-infrared second-zone fluorescence signal), and the dual-channel depth image (d) (where pseudo color indicates the depth size). DETAILED DESCRIPTION

[0013] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0014] Combination Figure 1 The present invention proposes a real-time multi-channel living macroscopic stereoscopic imaging system based on near-infrared second zone and visible light, which includes: visible light cameras 1, 6, infrared cameras 2, 5, coaxial dual-channel lenses 3, 7, illumination and excitation light source 8, and stereoscopic display system 9.

[0015] Further exemplarily, the illumination and excitation light sources include an illumination light source: a shadowless lamp, and an excitation light source: one of a laser or an LED light source.

[0016] Further illustratively, the visible light camera specifically adopts the same model of visible light sensitive industrial camera.

[0017] Further exemplarily, the infrared camera specifically adopts the same model of InGaAs deep-cooling camera that is sensitive to the infrared band.

[0018] Further exemplary, the dual-channel lens adopts a coaxial dual-channel lens that enhances the transmittance of a wide band from visible to near-infrared light (360-1700 nm). The visible light channel of the lens collects signals from 360-700 nm, and the near-infrared channel collects signals greater than 1000 nm, and the lens is optimized for various aberrations of the two channels.

[0019] Further exemplarily, the stereoscopic display system includes a computer and a stereoscopic display device. The computer is specifically a high-bandwidth computer that can support multiple high-definition video inputs. The stereoscopic display device is specifically a naked-eye 3D display, which is provided with a multi-lens screen, and uses vertically arranged cylindrical lenses to control the propagation direction of the left and right images, so that the left and right eyes of the viewer receive different images, thereby forming parallax to obtain a stereoscopic image.

[0020] Combination Figure 2 The real-time multi-channel in vivo macroscopic stereoscopic imaging method proposed by the present invention comprises the following steps: Step 1: The fluorescent substance in the biological tissue 4 is excited by the excitation light source to emit a fluorescent signal in the near-infrared region II, and at the same time, the part other than the fluorescent substance reflects a shadowless lamp signal as an illumination light source.

[0021] Step 2: Two sets of dual-channel image acquisition devices are arranged in parallel at a fixed distance b to simultaneously acquire four signals.

[0022] In each set of dual-channel image acquisition equipment, the near-infrared second-zone fluorescence and reflected shadowless lamp signals are split and filtered by the dual-channel lens, and then imaged synchronously on the infrared camera and visible light camera sensors in the same field of view; Step 3: The real-time image signals of the four cameras are transmitted to the computer, which are the left fluorescence image F L 、Fluorescence right view F R , Visible light left view V L , Visible light right view V R , the image enhancement algorithm based on Hessian feature and Frangi filter is used to enhance the near-fluorescence left and right views F L 、F R Structural information of Step 4: Get the two sets of dual-channel left and right views F L 、V L and F R 、V R Perform image registration separately, give the fluorescence image a suitable pseudo color and fuse it with the visible light image to obtain a dual-channel image FV L 、FV R Since there is parallax between the left and right views, the aligned left and right dual channel views FV L 、FV RThe image is sent to the naked-eye 3D display, and the propagation direction of the left and right images is controlled by the cylindrical lens, so that the left and right eyes of the viewer receive different images, thereby forming parallax and obtaining a three-dimensional image. Step 5: Use the binocular global matching algorithm to match FV L 、FV R Perform matching operations and calculate the dual-channel disparity map, perform weighted least squares filtering on the dual-channel disparity image, and perform connected domain detection to remove the small noise areas in the disparity map due to mismatching to obtain the processed disparity map FV D .

[0023] Based on the calibrated visible and infrared camera parameters and the fixed distance b between the cameras, a dual-channel specific depth image of the target is obtained, and an appropriate pseudo color is given to it to represent the depth output.

[0024] Example: Mice were depilated in advance and injected with 200 μL (concentration 0.2 mg / mL) of indocyanine green (ICG, FDA-approved, with a fluorescence center wavelength of 810 nm, but fluorescence tailing above 900 nm can still be used for clear near-infrared zone II imaging) aqueous solution in the orbit.

[0025] During imaging, the mouse was placed under a dual-channel lens and irradiated with 793 nm excitation light. Two dual-channel collection systems were arranged in parallel at a fixed distance b to collect four signals simultaneously. In each dual-channel collection system, the near-infrared second-zone fluorescence and reflected visible light signals were split by the dual-channel lens and imaged synchronously on the infrared camera and visible light camera sensors in the same field of view; the real-time image signals of the four cameras were transmitted to the computer, namely, the left view of the fluorescence F L 、Fluorescence right view F R , Visible light left view V L , Visible light right view V R .

[0026] The two sets of dual-channel left and right views F L 、V L and F R 、V R Perform image registration separately, give the fluorescence image a suitable pseudo color and fuse it with the visible light image to obtain a dual-channel image FV L 、FV R . Use binocular global matching algorithm to match FV L 、FV R Perform matching operations and calculate the dual-channel disparity map, perform weighted least squares filtering on the dual-channel disparity image, and perform connected domain detection to remove the small noise areas in the disparity map due to mismatching to obtain the processed disparity map FV DBased on the calibrated visible and infrared camera parameters and the fixed distance b between the cameras, a dual-channel depth image of the mouse body and its liver is obtained, and appropriate pseudo colors are assigned to represent the depth and output, see Figure 3 .

[0027] The present invention discloses a real-time multi-channel in vivo macroscopic stereoscopic imaging system based on near-infrared second region and visible light and an imaging method thereof. The system integrates a visible light camera, a near-infrared camera, a coaxial dual-channel lens, an illumination and excitation light source, and a stereoscopic display system, and can capture high-quality stereoscopic images in different spectral intervals, providing depth perception for users. The system has broad application prospects in clinical fluorescent surgical navigation.

[0028] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system, characterized in that: The system includes: Dual-channel image acquisition equipment, lighting and excitation light sources, and stereoscopic display system; The illumination and excitation light sources are facing the biological tissue to be observed, and the fluorescent substance in the biological tissue is excited by the excitation light source to emit a fluorescent signal in the near-infrared region II, while the part other than the fluorescent substance reflects a shadowless lamp signal as the illumination light source; There are two sets of dual-channel image acquisition equipment, each set includes a visible light camera, a near-infrared camera and a coaxial dual-channel lens; in each set of dual-channel image acquisition equipment, the near-infrared second-zone fluorescence and the reflected shadowless lamp signal are split and filtered by the coaxial dual-channel lens, and then synchronously imaged in the near-infrared camera and the visible light camera in the same field of view; The stereoscopic display system is used to process images output by a near-infrared camera and a visible light camera to obtain a multi-channel stereoscopic image with depth information.

2. The near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system according to claim 1, characterized in that: The illumination and excitation light sources include an illumination light source and an excitation light source, wherein the illumination light source adopts a shadowless lamp, and the excitation light source adopts one of a laser or an LED light source.

3. The near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system according to claim 1, characterized in that: The visible light camera is an industrial camera sensitive to visible light.

4. The near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system according to claim 1, characterized in that: The near-infrared camera adopts an InGaAs deep-cooling camera that is sensitive to the near-infrared band.

5. A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system according to claim 3 or 4, characterized in that: The coaxial dual-channel lens adopts a coaxial dual-channel lens that enhances the transmittance of a wide band from visible to near-infrared light; the visible light channel of the lens collects signals of 360-700 nm, and the near-infrared channel collects signals greater than 1000 nm, and the lens optimizes the various aberrations of the two channels accordingly.

6. The near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system according to claim 1, characterized in that: The stereoscopic display system includes a computer and a stereoscopic display device, wherein the computer is specifically a large-bandwidth computer that can support multiple high-definition video inputs; the stereoscopic display device is specifically a naked-eye 3D display, which is provided with a multi-lens screen and uses vertically arranged cylindrical lenses to control the propagation direction of the left and right images, so that the left and right eyes of the viewer receive different images, thereby forming parallax to obtain a stereoscopic image.

7. A real-time multi-channel in vivo macroscopic stereoscopic imaging method, using the imaging system according to any one of claims 1 to 6, characterized in that: The following steps are involved: Step 1: The fluorescent substance in the biological tissue is excited by the excitation light source to emit a fluorescent signal in the near-infrared region II, and at the same time, the part other than the fluorescent substance reflects a shadowless lamp signal as an illumination light source; Step 2: Two sets of dual-channel image acquisition devices are arranged in parallel to collect four signals at the same time; in each set of dual-channel image acquisition devices, the near-infrared second zone fluorescence and reflected shadowless lamp signals are split and filtered by the dual-channel lens, and then synchronously imaged on the near-infrared camera and the visible light camera in the same field of view; Step 3: The real-time image signals of the four cameras are transmitted to the computer, which are the left fluorescence image F L 、Fluorescence right view F R , Visible light left view V L , Visible light right view V R ; Use image enhancement algorithm to enhance the near-fluorescence left and right views F L 、F R Structural information of Step 4: Get the two sets of dual-channel left and right views F L 、V L and F R 、V R Perform image registration separately, give pseudo color to the fluorescence image and fuse it with the visible light image to obtain a dual-channel image FV L 、FV R ; The registered left and right dual channel images FV L 、FV R It is sent to the naked-eye 3D display, and the propagation direction of the left and right images is controlled through the cylindrical lens, so that the viewer's left and right eyes receive different pictures, thus forming parallax and obtaining a three-dimensional image.

8. The real-time multi-channel in vivo macroscopic stereoscopic imaging method according to claim 1, characterized in that: In step 3, the image enhancement algorithm based on Hessian features and Frangi filtering is used to enhance the near-fluorescence left and right views F L 、F R structural information.

9. The real-time multi-channel in vivo macroscopic stereoscopic imaging method according to claim 1, characterized in that: The method also includes step 5, wherein the left and right dual channel images FV L 、FV R Perform matching operations and calculate the dual-channel disparity map, perform weighted least squares filtering on the dual-channel disparity image, and perform connected domain detection to remove small noise areas in the disparity map due to mismatching, and obtain the processed disparity map FV D ; Based on the calibrated camera parameters and the fixed distance between the cameras, a dual-channel specific depth image of the target is obtained, which is given a pseudo color to represent the depth and output.

10. The real-time multi-channel in vivo macroscopic stereoscopic imaging method according to claim 1, characterized in that: The binocular global matching algorithm is used to match the left and right dual channel images FV L 、FV R Perform matching operations and calculate a dual-channel disparity map.

Images