A near-infrared two-zone fluorescence-magnetic particle-CT three-modality fusion imaging method

Abstract

The application provides a near-infrared two-region fluorescence-magnetic particle-CT three-modality fusion imaging method, which can be applied to the field of biomedical imaging technology. The method comprises the following steps: performing threshold segmentation and connected domain screening on a CT image to obtain marked points of the CT image; performing a defocus blur removal operation on an original near-infrared two-region fluorescence image to obtain a near-infrared two-region fluorescence image, and obtaining marked points of the near-infrared two-region fluorescence image and marked points of a magnetic particle image through key point detection; performing registration on the marked points of the CT image, the marked points of the near-infrared two-region fluorescence image and the marked points of the magnetic particle image; performing region of interest segmentation on the registered near-infrared two-region fluorescence image, the registered magnetic particle image and the registered CT image respectively; and performing multi-modality fusion on the region of interest of the near-infrared two-region fluorescence image, the region of interest of the magnetic particle image and the region of interest of the CT image to obtain a lesion region of a target organism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical imaging, in particular to a near-infrared region-II fluorescence-magnetic particle-CT three-modality fusion imaging method. BACKGROUND

[0002] Near-infrared region-II fluorescence molecular imaging (NIR-II FMI) is an advanced molecular imaging technology with high resolution, high sensitivity and clinical translation, which can accurately detect early tumors and metastases. However, NIR-II FMI can only provide fluorescence photon distribution information on the surface of biological tissues, and cannot provide three-dimensional spatial distribution information of tumors. Moreover, due to the absorption and scattering of photons in tissues, the imaging depth is limited. The above problems limit the further development and wide application of NIR-II FMI in the field of tumor precise diagnosis and treatment.

[0003] Magnetic particle imaging (MPI) is a new molecular imaging technology, which has the advantages of no imaging depth limitation, linear quantification, high sensitivity, no background signal interference, and no ionizing radiation hazard, and has broad biomedical application prospects. However, MPI has low spatial resolution and anisotropic resolution, which hinders its clinical application.

[0004] Computed tomography (CT) uses X-ray tomography technology to provide high-resolution anatomical structure images, with a spatial resolution of microns, which can clearly present the morphology of biological tissues and organs, and is an important imaging means for disease diagnosis. However, CT can only provide structural information and lacks molecular functional information. SUMMARY

[0005] In view of the above problems, the present application provides a near-infrared region-II fluorescence-magnetic particle-CT three-modality fusion imaging method, which at least solves one of the problems of the prior art.

[0006] According to a first aspect of the present application, a near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method is provided, comprising: performing a multi-modal imaging operation on a target biological body injected with a fluorescence-magnetic particle dual-mode tracer by using a multi-modal image acquisition device to obtain an original near-infrared two-zone fluorescence image, a magnetic particle image and a CT image of the target biological body; performing an automatic threshold segmentation operation and a connected domain screening operation on the CT image to obtain a marker point representing the contour edge of the target biological body in the CT image; iteratively performing a deblurring operation based on deconvolution on the original near-infrared two-zone fluorescence image to obtain a near-infrared two-zone fluorescence image, and obtaining a marker point of the near-infrared two-zone fluorescence image and a marker point of the magnetic particle image through a key point detection operation;

[0007] Performing a registration operation of corresponding marker points on the marker point of the CT image, the marker point of the near-infrared two-zone fluorescence image and the marker point of the magnetic particle image; performing a region of interest segmentation operation on the registered near-infrared two-zone fluorescence image, the registered magnetic particle image and the registered CT image, respectively; performing multi-modal fusion processing on the region of interest of the near-infrared two-zone fluorescence image, the region of interest of the magnetic particle image and the region of interest of the CT image to obtain a lesion area of the target biological body.

[0008] According to an embodiment of the present application, the multi-modal image acquisition device described above comprises a near-infrared two-zone fluorescence imaging module, a magnetic particle imaging module and a CT imaging module; wherein the near-infrared two-zone fluorescence imaging module comprises a laser excitation unit, a fluorescence acquisition unit and a signal processing unit; wherein the magnetic particle imaging module comprises a gradient magnetic field generating unit, a driving unit and an excitation and receiving unit; wherein the CT imaging module comprises an X-ray generating unit, a detector unit and a three-dimensional spiral scanning mechanism.

[0009] According to an embodiment of the present application, the multi-modal imaging operation performed by using the multi-modal image acquisition device on the target biological body injected with the fluorescence-magnetic particle dual-mode tracer to obtain the original near-infrared two-zone fluorescence image, the magnetic particle image and the CT image of the target biological body comprises: in the magnetic particle image imaging process, by setting the gradient magnetic field generating unit, the driving unit and the excitation and receiving unit, a magnetic particle imaging module with a two-stage feed-through compensation circuit is obtained, and the target biological body is scanned by using the magnetic particle imaging module with the two-stage feed-through compensation circuit to obtain the magnetic particle image.

[0010] According to the embodiment of the present application, the automatic threshold segmentation operation and the connected domain screening operation on the CT image to obtain the marker points representing the contour edges of the target biological body in the CT image include: performing noise reduction processing and gray scale normalization processing on the CT image to obtain a preprocessed CT image, and calculating a gray scale histogram of the preprocessed CT image; performing traversal operation of candidate threshold value iteration by calculating the inter-class variance of the gray scale histogram to obtain a maximum threshold value, and generating a binary image of the preprocessed CT image based on the maximum threshold value; performing connectedness analysis processing on the binary image and calculating the pixel area of each connected domain in the binary image, and performing segmentation processing and screening processing on the connected domains based on the calculated pixel area; and performing edge detection processing and marker point sampling processing on the connected domain screening result by using a preset edge detection algorithm to obtain the marker points of the preprocessed CT image.

[0011] According to the embodiment of the present application, the noise reduction processing and the gray scale normalization processing on the CT image to obtain the preprocessed CT image include: performing noise reduction processing on the CT image by using a Gaussian filter algorithm and / or a median filter algorithm to obtain a CT image retaining edge structure in a uniform image area; correcting the gray scale distribution of the CT image by using a homomorphic filter algorithm and / or a histogram equalization filter algorithm in a non-uniform image area; and completing the gray scale normalization processing by linearly mapping the Hausdorff unit of the filtered CT image into a standard gray scale range to obtain the preprocessed CT image.

[0012] According to the embodiment of the present application, the marker points of the near-infrared two-region fluorescence image and the marker points of the magnetic particle image obtained by the key point detection operation include: performing noise reduction, image enhancement and pixel normalization processing on the near-infrared two-region fluorescence image to obtain a preprocessed near-infrared two-region fluorescence image; performing multi-scale Gaussian blur processing on the preprocessed near-infrared two-region fluorescence image and calculating the Gaussian difference between adjacent scales to obtain a Gaussian difference pyramid; and performing extreme point detection operation and key point positioning operation on the Gaussian difference pyramid to obtain the marker points of the preprocessed near-infrared two-region fluorescence image.

[0013] According to the embodiment of the present application, the marker points of the near-infrared two-region fluorescence image and the marker points of the magnetic particle image obtained by the key point detection operation further include: performing three-dimensional Gaussian convolution preprocessing on the magnetic particle image to obtain a three-dimensional scale space of the magnetic particle image; performing three-dimensional connected domain extraction processing on the three-dimensional scale space and screening processing based on the area threshold value and the position to obtain the marker points of the magnetic particle image.

[0014] According to the embodiment of the present application, the registration operation of the corresponding marking points on the CT image, the near-infrared two-region fluorescence image and the magnetic particle image comprises: projecting the marking points of the CT image to the imaging plane of the near-infrared two-region fluorescence image, and estimating an initial rotation matrix and a translation vector by principal component analysis; performing the registration operation of the corresponding marking points on the near-infrared two-region fluorescence image and the CT image based on the initial rotation matrix and the translation vector by using an iterative closest point search algorithm; performing the registration operation of the corresponding marking points on the CT image and the magnetic particle image after the marking points are screened by using the iterative closest point search algorithm by dynamically adjusting the iteration step; taking the CT image as a reference, calculating a first registration transformation matrix between the CT image and the magnetic particle image and a second registration transformation matrix between the CT image and the near-infrared two-region fluorescence image; and performing the registration operation of the corresponding marking points on the magnetic particle image and the near-infrared two-region fluorescence image by using the first registration transformation matrix and the second registration transformation matrix.

[0015] The second aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0016] The third aspect of the present application also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0017] The near-infrared two-region fluorescence-magnetic particle-CT three-modality fusion imaging method provided by the present application realizes the synchronous acquisition of three-modality medical images through a multi-modality image acquisition device, and realizes the accurate fusion of three-modality images through registration. Meanwhile, the method provided by the present application combines the advantages of high resolution and high sensitivity of near-infrared two-region fluorescence molecular imaging, the advantages of no imaging depth limitation and linear quantification of magnetic particle imaging, and the advantage of providing high-resolution anatomical structure of CT, realizes the accurate visualization and quantitative analysis of three-dimensional biological tissue structure and function with high resolution, high sensitivity and no imaging depth limitation, and provides more accurate intermediate information for the diagnosis of the lesion area. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 is a flowchart of the near-infrared two-region fluorescence-magnetic particle-CT three-modality fusion imaging method according to the embodiment of the present application.

[0020] Figure 2 is a structural schematic diagram of a multi-modal image acquisition device according to an embodiment of the present application.

[0021] Fig. 3(a) is a structural schematic diagram of functional units in a magnetic particle imaging module according to an embodiment of the present application.

[0022] Fig. 3(b) is a structural schematic diagram of a receiving coil according to an embodiment of the present application.

[0023] Figure 4 is a structural schematic diagram of a near-infrared two-region fluorescence-CT rotating acquisition control system according to an embodiment of the present application.

[0024] Figure 5 is an effect schematic diagram of near-infrared two-region-magnetic particle-CT three-modal fusion imaging according to an embodiment of the present application.

[0025] Figure 6 is a block diagram of an electronic device suitable for implementing a near-infrared two-region-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0027] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present application. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0028] All terms used herein (including technical and scientific terms) have meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0029] In the case of using expressions like "at least one of A, B, and C", it will be understood that such phrases are meant to encompass the selections of: A alone, B alone, C alone, combinations with two of A, B, and C, such as AT and B together, A and C together, B and C together, combinations of all three, such as A and B and C together, and the like.

[0030] In view of the problems of FMI, MPI and CT imaging, the present application proposes a three-modal fusion imaging system and method, which realizes the seamless combination of high-resolution, high-sensitivity molecular functional imaging, quantitative tracing and anatomical structure by the fusion of three different modalities, combining the advantages of high resolution and high sensitivity of NIR-II FMI, the advantage of no imaging depth limitation of MPI, the advantage of linear quantification and the advantage of providing high-resolution anatomical structure of CT.

[0031] Figure 1 is a flowchart of a near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present application.

[0032] As shown in Figure 1 The above near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method includes operations S110-S160.

[0033] In operation S110, a multi-modal image acquisition device is used to perform a multi-modal imaging operation on a target biological body injected with a fluorescence-magnetic particle dual-mode tracer, to obtain original near-infrared two-zone fluorescence images, magnetic particle images and CT images of the target biological body.

[0034] The above multi-modal image acquisition device includes a fluorescence imaging module, a magnetic particle imaging module and a CT imaging module, and can perform synchronous multi-modal medical image acquisition on the target biological body, thereby realizing more accurate positioning of the lesion area of the target biological body.

[0035] In operation S120, an automatic threshold segmentation operation and a connected component filtering operation are performed on the CT images to obtain marker points representing the contour edges of the target biological body in the CT images.

[0036] In the above operation S120, the maximum inter-class variance algorithm can be selected to segment and filter the connected components of the CT images.

[0037] In operation S130, an iterative deconvolution-based defocus blur removal operation is performed on the original near-infrared two-zone fluorescence images to obtain near-infrared two-zone fluorescence images, and key point detection operations are performed to obtain marker points of the near-infrared two-zone fluorescence images and marker points of the magnetic particle images.

[0038] The near-infrared two-region fluorescence image is subjected to defocus blur removal processing by using a Richardson-Lucy deconvolution algorithm.

[0039] The labeled points of the near-infrared two-region fluorescence image and the labeled points of the magnetic particle image correspond to the labeled points of the CT image, that is, the positions of the target organisms represented by the labeled points of the CT image correspond to the labeled points of the near-infrared two-region fluorescence image and the labeled points of the magnetic particle image.

[0040] In operation S140, registration of corresponding labeled points is performed on the labeled points of the CT image, the labeled points of the near-infrared two-region fluorescence image and the labeled points of the magnetic particle image.

[0041] The present application uses an iterative closest point search algorithm (ICP) to register the above-mentioned three kinds of images.

[0042] In operation S150, a region of interest segmentation operation is performed on the registered near-infrared two-region fluorescence image, the registered magnetic particle image and the registered CT image, respectively.

[0043] In operation S160, multi-modal fusion processing is performed on the region of interest of the near-infrared two-region fluorescence image, the region of interest of the magnetic particle image and the region of interest of the CT image, to obtain a lesion region of the target organism.

[0044] The registered above-mentioned three kinds of medical images are fused by using the trained multi-modal fusion model or deep learning algorithm.

[0045] The above-mentioned operations S110 to S160 are generally related to experimental animals such as mice in the embodiments, specific embodiments, specific implementation manners or experiments of the present application. However, the method provided by the present application is also applicable to humans. In the case of humans as target organisms, the permission of the target organism is required, and multi-modal imaging is performed on the target organism.

[0046] It should be particularly pointed out that in the case of the target organism being a human, the multi-modal medical image or other information and data related to the privacy of the target organism involved in the present application are all information and data authorized by the target organism or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose authorization or refusal.

[0047] Meanwhile, in the case of the target organism being a human, the above operation S110 to operation S160 aims to provide more accurate intermediate information for the diagnosis of the lesion area, rather than directly obtaining the diagnosis information or health status of the target organism, and the above operation S110 to operation S160 and other operations of the embodiments of the present application all belong to information processing operations implemented by a computer or the like.

[0048] The near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method provided by the present application realizes the synchronous acquisition of three-modal medical images through a multi-modal image acquisition device, and realizes the accurate fusion of three-modal images through registration; meanwhile, the method provided by the present application combines the advantages of high resolution and high sensitivity of near-infrared two-zone fluorescence molecular imaging, the advantages of no imaging depth limitation and linear quantification of magnetic particle imaging, and the advantages of providing high-resolution anatomical structure of CT, realizes the accurate visualization and quantitative analysis of three-dimensional biological tissue structure and function with high resolution, high sensitivity and no imaging depth limitation, and provides more accurate intermediate information for the diagnosis of the lesion area.

[0049] According to the embodiments of the present application, the above multi-modal image acquisition device includes a near-infrared two-zone fluorescence imaging module, a magnetic particle imaging module and a CT imaging module; wherein the near-infrared two-zone fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit and a signal processing unit; wherein the magnetic particle imaging module includes a gradient magnetic field generating unit, a driving unit and an excitation and receiving unit; wherein the CT imaging module includes an X-ray generating unit, a detector unit and a three-dimensional spiral scanning mechanism.

[0050] According to the embodiments of the present application, the above multi-modal imaging operation on the target organism injected with the fluorescence-magnetic particle dual-mode tracer by using the multi-modal image acquisition device to obtain the original near-infrared two-zone fluorescence image, the magnetic particle image and the CT image of the target organism includes: in the magnetic particle image imaging process, a magnetic particle imaging module with a two-stage feed-through compensation circuit is obtained by setting a gradient magnetic field generating unit, a driving unit and an excitation and receiving unit, and the target organism is scanned by using the magnetic particle imaging module with the two-stage feed-through compensation circuit to obtain the magnetic particle image.

[0051] The application will be described in detail below with specific examples and in conjunction with the accompanying drawings Figures 2-4 The multi-modal image acquisition device provided by the application will be described in detail.

[0052] Figure 2 Fig. 1 is a structural schematic diagram of a multi-modal image acquisition device according to an embodiment of the application.

[0053] Fig. 3(a) is a structural schematic diagram of each functional unit in a magnetic particle imaging module according to an embodiment of the application.

[0054] Fig. 3(b) is a structural schematic diagram of a receiving coil according to an embodiment of the application.

[0055] Figure 4 Fig. 4 is a structural schematic diagram of a near-infrared two-region fluorescence-CT rotating acquisition control system according to an embodiment of the application.

[0056] As shown in Fig. 1, the multi-modal image acquisition device provided by the application has a near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging module, and each module works cooperatively through a mechanical interface and a control system. Figure 2 As shown in Fig. 1, the device includes an optical imaging region: an NIR-II fluorescence module and a CT module are integrated, and synchronous rotating acquisition is realized through a planar turntable, and the optical imaging region mainly includes an X-ray ball tube, an NIR-II CMOS camera, a laser, and an X-ray detector; the optical imaging region is controlled to image through an acquisition control system and a motion control system; and a magnetic particle imaging region: independent of the optical imaging region, sample transfer is realized through a high-precision translation stage (or a high-precision translation stage and an animal bed), and the magnetic particle imaging region includes a permanent magnet and a driving coil; the permanent magnet is connected with a power module, and the driving coil is connected with a high-sensitivity signal processing module, wherein the high-sensitivity signal processing module includes an amplifier, a wave trap, and a signal processing and visualization module. Figure 2 As shown in Fig. 1, the control system and a data processing region: contain a PLC controller (Programmable Logic Controller), a host computer (for example, a development environment programmed by a graphical editing language G language: LabVIEW software), and a multi-modal data fusion processing unit. Figure 2 As shown in Fig. 1, the image processing workstation is used to receive image signals and process and display them, and the impedance matching circuit is used between the high-sensitivity signal processing module and the power module to maximize power transmission and reduce power reflection to the amplifier, thereby improving energy transmission efficiency.

[0057] The near-infrared two-region (NIR-II) fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit, and a signal processing unit.

[0058] The laser excitation unit: a pulsed laser with a wavelength of 1000 nm, a repetition frequency adjustable in the range of 10-100 kHz, and an output power density monitored in real time by a power meter and controlled at ≤100 mW / cm². The laser is coupled to a ring-shaped illuminator through an optical fiber, and the internal diffuse reflection element ensures that the excitation light intensity uniformity error is ≤5%, ensuring uniform excitation of the sample surface.

[0059] The fluorescence acquisition unit: a back-illuminated sCOMS camera equipped with a deep refrigeration system (temperature reduced to -40°C), a quantum efficiency of more than 90%, a pixel size of 6.5 μm×6.5 μm, and a 1000-1700 nm long-pass filter to filter excitation light stray light. A high numerical aperture objective lens (NA=0.8, silicon-based material, working distance 10-20 mm) is used to collect signals from tissues with a depth of 1-2 cm.

[0060] The signal processing unit: the original image is deblurred by the Richardson-Lucy deconvolution algorithm, and the region of interest (ROI) is segmented by the Otsu threshold algorithm (Otsu threshold algorithm: an automatic threshold selection algorithm for image segmentation).

[0061] During the near-infrared two-zone fluorescence image imaging process, the laser power density is 80 mW / cm², which is uniformly output to the imaging area after fiber coupling. The sCOMS camera acquires fluorescence images with an exposure time of 100 ms, and a total of 180 fluorescence spot images are obtained by rotating the turntable every 2°.

[0062] The MPI module includes a gradient magnetic field generating unit, a driving unit, and an excitation and reception unit. The specific structure of the gradient magnetic field generating unit, the driving unit, and the excitation and reception unit is shown in Figure 3(a).

[0063] As shown in Figure 3(a), the gradient magnetic field generating unit: a pair of NdFeBN52 permanent magnets are symmetrically distributed with a spacing of 45 mm, generating a gradient magnetic field of 0.3-2.0 T / m / µ0 in the z-axis direction, and the FFP (Field-Free Point) position is calibrated by a gauss meter.

[0064] As shown in Figure 3(a), the driving unit: y-axis and z-axis driving coils (using a Helmholtz coil structure (spacing equal to radius 40 mm), when a 20 Hz sinusoidal current is passed, the FFP translation speed is 1 mm / s, the scanning step is 100 μm, and three-dimensional space (50 mm×50 mm×50 mm) scanning is realized.

[0065] Wherein, as shown in Figure 3(a), the excitation and receiving unit: the excitation coil is a hollow solenoid (Litz wire 0.1mmx300strands, 111 turns, wound in two sections, inductance 150μH), 25kHz, 27mT / µ0sinusoidal current, water cooling system control coil temperature rise≤5℃.

[0066] Wherein, Figure 3(a) also shows the position relationship between the high-precision translation stage and the receiving coil, the high-precision translation stage is used to place the target organism to be imaged (such as experimental animals).

[0067] The receiving coil is a three-stage gradient structure (as shown in Figure 3(b)):

[0068] Signal detection coil: single layer Litz wire (0.1mmx35strands, 50 turns, inductance 80μH), SPIONs signal sensitivity 10nV / (mg / mLFe), wherein the voltage preamplifier is used to amplify the received voltage signal, and the amplified signal is processed on the computer / DAQ (Data Acquisition), and the processed result is fed back to the voltage preamplifier with a 25kHZ modulation signal.

[0069] Compensation coil: 25 turns at both ends, reverse winding, mutual inductance coefficient with signal detection coil-0.95, first stage suppresses 85% feedthrough signal, residual signal peak-to-peak value≤4V.

[0070] Two-stage feedthrough compensation circuit:

[0071] First stage: by adjusting the number of turns of the compensation coil (±5 turns), the voltage difference u pc =u d −u c Peak-to-peak value satisfies V pp (u pc )≤4V, suppresses 10-50kHz feedthrough signal.

[0072] Second stage: NI DAQ (National Instruments Data Acquisition, hardware devices and supporting software for data acquisition and control) generates a 25kHz modulation signal (phase difference 180°±5° with residual feedthrough signal), after processing by a differential amplifier circuit with a gain of 1000 times, the feedthrough signal is suppressed to≤10mV, and the base frequency signal retention rate>95%.

[0073] In the process of magnetic particle imaging, Figure 2The high-precision translation table shown pushes the fixed bed to the magnetic particle imaging module (MPI module), after the initial position calibration of the FFP, the drive coil is connected with 20Hz current, the FFP is controlled to carry out three-dimensional scanning with 100mu m step length and 1mm / s speed.The excitation coil is connected with 25kHz current to excite the probe magnetization, the receiving coil synchronously collects signals, after two-stage feedthrough compensation, the X-space algorithm is used to reconstruct the probe concentration distribution.

[0074] The CT imaging module comprises an X-ray generating unit, a detector unit and a three-dimensional spiral scanning mechanism.

[0075] The X-ray generating unit is a micro-focus X-ray ball with adjustable voltage of 80-120kV and adjustable current of 50-200mA, is matched with a 0.5mm beryllium window and a lead collimator, generates an X-ray beam with a fan angle of 45°, and irradiates a field of view covering a region with a diameter of 50mm.

[0076] The detector unit is a cadmium telluride semiconductor detector array (pixel spacing 100mu m, 1024*1024 channels, dynamic range 16 bits), the single-frame acquisition time is 50ms, the quantum detection efficiency (QDE) reaches 80% at 80keV, and motion artifacts are effectively reduced.

[0077] The three-dimensional spiral scanning mechanism comprises a plane turntable (diameter 300mm) for synchronously rotating the X-ray ball tube and the detector at a step length of 2° to carry out 360° scanning, 180 projection data are collected, and the layer thickness is adjustable in the range of 0.1-1.0mm.

[0078] In the CT imaging process, Figure 2 The high-precision translation table shown pushes the fixed bed to the plane turntable, the X-ray ball tube voltage is 100kV, the current is 150mA, the spiral scanning is carried out at a layer thickness of 0.5mm and a speed of 1turn / s, 180 projection data are collected, and a three-dimensional anatomical image is reconstructed by filtered back projection (FBP).

[0079] The motion control system of the multi-modal image acquisition device provided by the application mainly refers to a near-infrared two-zone fluorescence-CT rotating acquisition control system, such as Figure 4As shown, it comprises a near-infrared camera, a laser, an X-ray detector, an X-ray ball tube, a translation turntable, a motion control system and an acquisition control system; wherein the motion control system and the acquisition control system are connected with a computer, the acquisition control system is connected with the X-ray detector and the near-infrared camera, the motion control system is connected with a high-precision translation stage and an animal bed, and the near-infrared two-zone fluorescence-CT rotary acquisition control system adopts a plane turntable (vertically arranged, with a diameter of 300 mm) integrated with a near-infrared fluorescence imaging module and a CT module, and can rotate around the central axis by 360°. Positioning grooves are arranged at the edge of the turntable, and are accurately connected with the positioning points of the fixed bed. The rotation speed of the turntable is 10° / s, and the fluorescence camera exposure (exposure time: 100 ms) and CT projection acquisition are triggered once every 2° of rotation. The high-precision translation stage is connected with the animal bed (i.e. the high-precision translation stage) through a clamping groove type interface, and after the fluorescence / CT acquisition is completed, it is horizontally translated to the center of the MPI module at a speed of 50 mm / s.

[0080] The multi-modal image acquisition device provided by the application has a compact structure, each imaging module can work independently or jointly image, is suitable for different experimental requirements and sample types, and provides a multifunctional platform for biomedical research and clinical diagnosis. Resetting and data storage of the multi-modal image acquisition device: the laser, the X-ray ball tube and the magnetic field generating unit and the driving unit are turned off, the high-precision translation stage returns the fixed bed to the initial position, and the mechanical arm is homed. The original data and the fusion result are stored in a server in a DICOM (Digital Imaging and Communications in Medicine) format, and a metadata file containing imaging parameters and a registration matrix is generated, so that subsequent tracing and analysis are facilitated.

[0081] According to the embodiment of the application, the automatic threshold segmentation operation and the connected domain screening operation are performed on the CT image to obtain the marker points representing the contour edges of the target biological body in the CT image, which includes: performing noise reduction processing and gray scale normalization processing on the CT image to obtain a preprocessed CT image, and calculating a gray scale histogram of the preprocessed CT image; performing traversal operation of candidate threshold value iteration by calculating the inter-class variance of the gray scale histogram to obtain a maximum threshold value, and generating a binary image of the preprocessed CT image based on the maximum threshold value; performing connectedness analysis processing on the binary image and calculating the pixel area of each connected domain in the binary image, and performing segmentation processing and screening processing on the connected domains based on the calculated pixel area; and performing edge detection processing and marker point sampling processing on the connected domain screening result by using a preset edge detection algorithm to obtain the marker points of the preprocessed CT image.

[0082] According to the embodiment of the present application, the above-mentioned denoising and gray scale normalization of the CT image to obtain the preprocessed CT image comprises: in the image region with uniform imaging, the CT image is denoised by using the Gaussian filtering algorithm and / or the median filtering algorithm to obtain the CT image with preserved edge structure; in the image region with non-uniform imaging, the gray scale distribution of the CT image is corrected by using the homomorphic filtering algorithm and / or the histogram equalization filtering algorithm; the gray scale normalization is completed by linearly mapping the Hausfield unit of the filtered CT image into the standard gray scale range to obtain the preprocessed CT image.

[0083] The above-mentioned embodiment relates to the key point detection and connected domain screening of the CT image based on the Ostu algorithm, and through the above-mentioned embodiment, the contour of the target organism can be obtained, and the spatial positioning of the specific region (for example, the lesion region) of the target organism can be realized in combination with the marker point (or the metal marker point).

[0084] In the above-mentioned embodiment, the combination strategy of the Gaussian filtering and the median filtering is adopted to effectively suppress the noise while preserving the edge structure. For the non-uniform imaging region, the homomorphic filtering (eliminating uneven illumination) and the histogram equalization (enhancing contrast) are introduced for local correction.

[0085] In the above-mentioned embodiment, the Hounsfield Unit (Hounsfield Unit, or HU value) is linearly mapped to the standard gray scale range (for example, 0-255) to solve the gray scale difference problem between different scanning devices.

[0086] In the above-mentioned embodiment, the optimal threshold is automatically determined by iteratively calculating the maximum inter-class variance, which is especially suitable for the CT image with a bimodal histogram. The above-mentioned algorithm has low time complexity and is suitable for real-time imaging.

[0087] In the above-mentioned embodiment, the pixel area of the connected domain is calculated and the threshold is set to effectively filter the artifacts caused by the noise. The edge detection algorithm (for example, the Canny algorithm) is used for sub-pixel level edge extraction, and the fine structure is preserved by non-maximum suppression.

[0088] According to the embodiment of the present application, the above-mentioned marker points of the near-infrared two-zone fluorescence image and the magnetic particle image obtained through the key point detection operation comprise: denoising, image enhancement and pixel normalization processing are performed on the near-infrared two-zone fluorescence image to obtain a preprocessed near-infrared two-zone fluorescence image; multi-scale Gaussian blur processing is performed on the preprocessed near-infrared two-zone fluorescence image, and Gaussian difference between adjacent scales is calculated to obtain a Gaussian difference pyramid; extreme point detection operation and key point positioning operation are performed on the Gaussian difference pyramid to obtain the marker points of the preprocessed near-infrared two-zone fluorescence image.

[0089] The near-infrared two-region fluorescence image needs to be denoised (such as non-local mean denoising), contrast enhanced (CLAHE algorithm: Contrast Limited Adaptive Histogram Equalization, contrast limited adaptive histogram equalization) and pixel normalized. These operations can eliminate environmental noise interference, improve the signal-to-noise ratio of the weak signal area, and provide standardized input data for subsequent multi-scale analysis.

[0090] An image pyramid is generated by multi-scale Gaussian blur (value gradient increment), and a difference pyramid (DoG, Difference of Gaussian) is obtained by subtracting adjacent scales. This process simulates the visual characteristics of the human eye and can effectively capture features of different sizes of marker points.

[0091] Three-dimensional extreme value search (space + scale dimension) is used to locate candidate points, and Taylor expansion is used to correct coordinate offset. Unstable points are removed by contrast threshold, and Hessian (Hessian) matrix is used to eliminate edge response interference, and finally the marker point coordinates with sub-pixel level accuracy are obtained.

[0092] The SIFT algorithm used in the application is scale invariant, which is suitable for both near-infrared fluorescence (high dynamic range) and magnetic particle image (low signal-to-noise ratio) marker point detection without adjusting core parameters.

[0093] According to the embodiment of the application, the marker points of the near-infrared two-region fluorescence image and the marker points of the magnetic particle image obtained by the key point detection further include: performing three-dimensional Gaussian convolution preprocessing on the magnetic particle image to obtain a three-dimensional scale space of the magnetic particle image; performing three-dimensional connected domain extraction processing on the three-dimensional scale space and performing screening processing based on an area threshold and a position to obtain the marker points of the magnetic particle image.

[0094] The above embodiment extends the two-dimensional Gaussian kernel to a three-dimensional convolution kernel to process the MPI image. Different The three-dimensional Gaussian filter with different values is used to perform multi-scale smoothing on the original image to form a three-dimensional scale space pyramid, effectively preserving the magnetic particle distribution characteristics at different resolutions.

[0095] The above embodiment realizes feature point detection of MPI three-dimensional body data through three-dimensional Gaussian convolution and connected domain analysis, solving the problem of three-dimensional marker point positioning in medical images.

[0096] By adjusting the Gaussian kernel parameters, both near-infrared two-region fluorescence images (two-dimensional) and MPI images (three-dimensional) can be processed simultaneously to realize a unified detection framework for multi-modal image marker points.

[0097] ​According to the embodiment of the present application, the registration operation of the corresponding marking points on the CT image, the near-infrared two-region fluorescent image and the magnetic particle image comprises: projecting the marking points of the CT image to the imaging plane of the near-infrared two-region fluorescent image, and estimating an initial rotation matrix and a translation vector by principal component analysis; based on the initial rotation matrix and the translation vector, performing the registration operation of the corresponding marking points on the near-infrared two-region fluorescent image and the CT image by using an iterative closest point search algorithm; performing screening processing on the marking points of the CT image and the magnetic particle image, and performing the registration operation of the corresponding marking points on the CT image and the magnetic particle image after the marking point screening by using the iterative closest point search algorithm through dynamic adjustment of the iteration step length; taking the CT image as a reference, calculating a first registration transformation matrix between the CT image and the magnetic particle image and a second registration transformation matrix between the CT image and the near-infrared two-region fluorescent image; and performing the registration operation of the corresponding marking points on the magnetic particle image and the near-infrared two-region fluorescent image by using the first registration transformation matrix and the second registration transformation matrix.

[0098] The above embodiment mainly relates to initial registration, ICP optimization and transformation transmission. The initial registration is projecting the CT marking points to the NIR-II FMI imaging plane and estimating an initial pose matrix by using PCA (Principal Component Analysis). The ICP optimization is respectively completing the dual-modality registration of CT and NIR-II FMI and CT and MPI by using a dynamic step length adjustment strategy. The transformation transmission is deducing a third transformation matrix between NIR-II FMI and MPI by matrix operation of the first registration matrix (CT→MPI) and the second registration matrix (CT→NIR-II FMI).

[0099] The above embodiment adopts dynamic step length ICP when registering the magnetic particle image, adaptively adjusts the search step length according to the point cloud density, avoids the local optimal problem, filters abnormal points by curvature characteristics, adopts marking point screening, and improves the registration robustness and multi-level precision control.

[0100] The above embodiment first realizes the unified coordinate system mapping of the X-ray (CT image), the optics (near-infrared two-region fluorescent image) and the magnetism (magnetic particle image).

[0101] Figure 5 It is an effect schematic diagram of near-infrared two-region-magnetic particle-CT three-modality fusion imaging according to the embodiment of the present application.

[0102] The near-infrared two-region-magnetic particle-CT three-modality fusion imaging method provided by the present application is based on the deep learning algorithm proposed, and the registered images are deeply fused. The anatomical structure provided by the CT image is displayed by a gray image, and the three-dimensional spatial distribution of the probe after fusion is displayed by a pseudo-color, as shown in Figure 5 . Figure 5 (a) in (c) represents a CT image, Figure 5 (b) in (c) represents a magnetic particle image, Figure 5 (c) in (c) represents a CT image, Figure 5 (d) in (c) represents a three-modality fusion result.

[0103] Figure 6 is a block diagram of an electronic device suitable for implementing the near-infrared two-region-magnetic particle-CT three-modality fusion imaging method according to an embodiment of the present application.

[0104] As shown in Figure 6 , the electronic device 600 according to an embodiment of the present application includes a processor 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), and the like. The processor 601 can also include an on-board memory for cache use. The processor 601 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to embodiments of the present application.

[0105] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The processor 601 performs various operations of the method processes according to embodiments of the present application by executing programs in the ROM 602 and / or the RAM 603. It should be noted that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method processes according to embodiments of the present application by executing programs stored in the one or more memories.

[0106] According to an embodiment of the present application, the electronic device 600 can further include an input / output (I / O) interface 605 that is also connected to the bus 604. The electronic device 600 can further include one or more of the following components connected to the input / output (I / O) interface 605: an input part 606 including a keyboard, a mouse, etc.; an output part 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 608 including a hard disk, etc.; and a communication part 609 including a network interface card such as a LAN card, a modem, etc. The communication part 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage part 608 as necessary.

[0107] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist separately without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the near-infrared two-region fluorescence-magnetic particle-CT three-modality fusion imaging method according to the embodiments of the present application.

[0108] According to an embodiment of the present application, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories of the above-described ROM 602 and / or RAM 603 and / or one or more memories other than the ROM 602 and the RAM 603.

[0109] The computer program product of the present application can be a computer program product comprising a computer readable storage medium and a computer program mechanism embedded in the computer readable storage medium. Such computer program product can further include a computer readable storage medium and program means for causing a processor or other programmable processing apparatus to function in a particular manner, such that the computer program mechanism embedded in the computer readable storage medium can be used to actually effect the apparatus functions.

[0110] Those skilled in the art will appreciate that the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly contemplated in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways without departing from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0111] The embodiments of the present application have been described above. However, these embodiments are merely intended for illustration, and are not intended to limit the scope of the present application. Although the embodiments are described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various alternatives and modifications can be made to the embodiments of the present application without departing from the scope of the present application, and such alternatives and modifications are intended to fall within the scope of the present application.

Claims

1. A near-infrared two-region fluorescence-magnetic particle-CT three-modality fusion imaging method, characterized in that, The method comprises: performing a multi-modal imaging operation on a target organism injected with a fluorescent-magnetic particle dual-mode tracer by using a multi-modal image acquisition device to obtain an original near-infrared two-zone fluorescent image, a magnetic particle image and a CT image of the target organism; performing an automatic threshold segmentation operation and a connected domain screening operation on the CT image to obtain marker points representing the contour edges of the target organism in the CT image; performing an iterative deconvolution-based defocusing blur removal operation on the original near-infrared two-zone fluorescent image by using a Richardson-Lucy deconvolution algorithm to obtain a near-infrared two-zone fluorescent image, and obtaining marker points of the near-infrared two-zone fluorescent image and the magnetic particle image through a key point detection operation; performing a registration operation of corresponding marker points on the marker points of the CT image, the marker points of the near-infrared two-zone fluorescent image and the marker points of the magnetic particle image; performing a region of interest segmentation operation on the registered near-infrared two-zone fluorescent image, the registered magnetic particle image and the registered CT image respectively; performing a multi-modal fusion processing on the region of interest of the near-infrared two-zone fluorescent image, the region of interest of the magnetic particle image and the region of interest of the CT image to obtain a lesion region of the target organism; The registration operation of corresponding marker points on the marker points of the CT image, the marker points of the near-infrared two-zone fluorescent image and the marker points of the magnetic particle image comprises: projecting the marker points of the CT image to the imaging plane of the near-infrared two-zone fluorescent image, and estimating an initial rotation matrix and a translation vector through principal component analysis; based on the initial rotation matrix and the translation vector, performing a registration operation of corresponding marker points on the near-infrared two-zone fluorescent image and the CT image by using an iterative closest point search algorithm; performing a registration operation of corresponding marker points on the CT image and the magnetic particle image after marker point screening by using the iterative closest point search algorithm through adaptive dynamic adjustment of the iteration step length based on the point cloud density; calculating a first registration transformation matrix between the CT image and the magnetic particle image and a second registration transformation matrix between the CT image and the near-infrared two-zone fluorescent image; performing a registration operation of corresponding marker points on the magnetic particle image and the near-infrared two-zone fluorescent image by using the first registration transformation matrix and the second registration transformation matrix.

2. The method of claim 1, wherein, The multi-modal image acquisition device comprises a near-infrared two-zone fluorescent imaging module, a magnetic particle imaging module and a CT imaging module; wherein the near-infrared two-zone fluorescent imaging module comprises a laser excitation unit, a fluorescent acquisition unit and a signal processing unit; wherein the magnetic particle imaging module comprises a gradient magnetic field generating unit, a driving unit, an excitation and receiving unit; wherein the CT imaging module comprises an X-ray generating unit, a detector unit and a three-dimensional spiral scanning mechanism.

3. The method of claim 2, wherein, The multi-modal image acquisition device is used to perform a multi-modal imaging operation on a target organism injected with a fluorescent-magnetic particle dual-mode tracer to obtain an original near-infrared two-zone fluorescent image, a magnetic particle image, and a CT image of the target organism, including: During the imaging of the magnetic particle image, the gradient magnetic field generating unit, the driving unit, and the excitation and receiving unit are set to obtain a magnetic particle imaging module with a two-stage feedthrough compensation circuit, and the target organism is scanned using the magnetic particle imaging module with the two-stage feedthrough compensation circuit to obtain the magnetic particle image.

4. The method of claim 1, wherein, An automatic threshold segmentation operation and a connected domain screening operation are performed on the CT image to obtain labeled points in the CT image representing the contour edges of the target organism, including: The CT image is subjected to noise reduction processing and gray scale normalization processing to obtain a pre-processed CT image, and a gray scale histogram of the pre-processed CT image is calculated; The inter-class variance of the gray scale histogram is calculated to iteratively perform a traversal operation of candidate threshold values, obtain a maximum threshold value, and generate a binary image of the pre-processed CT image based on the maximum threshold value; The binary image is subjected to connectivity analysis processing and the pixel area of each connected domain in the binary image is calculated, and the connected domains are segmented and screened based on the calculated pixel area; A pre-set edge detection algorithm is used to perform edge detection processing and labeled point sampling processing on the connected domain screening results to obtain labeled points of the pre-processed CT image.

5. The method of claim 4, wherein, The CT image is subjected to noise reduction processing and gray scale normalization processing to obtain a pre-processed CT image, including: In the imaging uniform image area, the CT image is subjected to noise reduction processing using a Gaussian filter algorithm and / or a median filter algorithm to obtain a CT image that retains edge structures; In the imaging non-uniform image area, the gray scale distribution of the CT image is corrected using a homomorphic filter algorithm and / or a histogram equalization filter algorithm; The gray scale normalization processing is completed by linearly mapping the Hausdorff unit of the filtered CT image to a standard gray scale range to obtain the pre-processed CT image.

6. The method of claim 1, wherein, The labeled points of the near-infrared two-zone fluorescent image and the labeled points of the magnetic particle image are obtained through a key point detection operation, including: The near-infrared two-zone fluorescent image is subjected to denoising, image enhancement, and pixel normalization processing to obtain a pre-processed near-infrared two-zone fluorescent image; The pre-processed near-infrared two-zone fluorescent image is subjected to multi-scale Gaussian blur processing and the Gaussian difference between adjacent scales is calculated to obtain a Gaussian difference pyramid; The Gaussian difference pyramid is subjected to extreme point detection operation and key point positioning operation to obtain the labeled points of the pre-processed near-infrared two-zone fluorescent image.

7. The method of claim 6, wherein, Also includes: The magnetic particle image is subjected to three-dimensional Gaussian convolution preprocessing to obtain a three-dimensional scale space of the magnetic particle image; The three-dimensional scale space is subjected to three-dimensional connected domain extraction processing and screening processing based on area threshold and position to obtain labeled points of the magnetic particle image.

8. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, the computer program or instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-7.

Images