Near-infrared two-region fluorescence-magnetic particle-CT (Computed Tomography) three-mode fusion imaging method

Abstract

The invention provides a near-infrared two-region fluorescence-magnetic particle-CT (Computed Tomography) three-mode fusion imaging method, which can be applied to the technical field of biomedical imaging. The method comprises the following steps: carrying out threshold segmentation and connected domain screening on a CT image to obtain mark points of the CT image; carrying out defocusing blur removal operation on the original near-infrared two-region fluorescence image to obtain the near-infrared two-region fluorescence image, and carrying out key point detection to obtain a mark point of the near-infrared two-region fluorescence image and a mark point of the magnetic particle image; registration of corresponding mark points is carried out on the mark points of the CT image, the mark points of the near-infrared two-region fluorescence image and the mark points of the magnetic particle image; respectively carrying out region-of-interest segmentation on the registered near-infrared two-region fluorescence image, the registered magnetic particle image and the registered CT image; and performing multi-modal fusion on the region of interest of the near-infrared two-region fluorescence image, the region of interest of the magnetic particle image and the region of interest of the CT image to obtain a focus region of the target organism.

Description

Technical Field

[0001] This invention relates to the field of biomedical imaging technology, specifically to a near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method. Background Technology

[0002] Near-infrared region-II fluorescence molecular imaging (NIR-II FMI) is a high-resolution, highly sensitive, and clinically translated advanced molecular imaging technique capable of accurately detecting early-stage tumors and lesion metastases. However, NIR-II FMI can only provide information on the distribution of fluorescent photons on the surface of biological tissues, and cannot provide information on the three-dimensional spatial distribution of tumors. Furthermore, the imaging depth is limited due to the absorption and scattering of photons within the tissue. These limitations restrict the further development and widespread application of NIR-II FMI in the field of precision oncology diagnosis and treatment.

[0003] Magnetic particle imaging (MPI) is an emerging molecular imaging technique with advantages such as no imaging depth limitations, linear quantization, high sensitivity, no background signal interference, and no ionizing radiation hazards, showing great promise for biomedical applications. However, MPI has relatively low spatial resolution and suffers from resolution anisotropy, hindering its clinical application.

[0004] Computed tomography (CT) uses X-ray computed tomography to provide high-resolution images of anatomical structures, with spatial resolution down to the micrometer level. It can clearly show the morphology of biological tissues and organs, making it an important imaging tool for disease diagnosis. However, CT can only provide structural information and lacks the acquisition of molecular functional information. Summary of the Invention

[0005] In view of the above problems, the present invention provides a near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method to at least solve one of the problems of the prior art.

[0006] According to a first aspect of the present invention, a near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method is provided, comprising: performing multimodal imaging on a target organism injected with a fluorescence-magnetic particle dual-modal tracer using a multimodal image acquisition device to obtain the original near-infrared two-region fluorescence image, magnetic particle image, and CT image of the target organism; performing automatic thresholding and connected component filtering on the CT image to obtain marker points in the CT image that characterize the contour edges of the target organism; iteratively performing a deconvolution-based defocus blur removal operation on the original near-infrared two-region fluorescence image to obtain a near-infrared two-region fluorescence image, and obtaining the marker points of the near-infrared two-region fluorescence image and the marker points of the magnetic particle image through key point detection;

[0007] Registration operations are performed on the marker points of the CT image, the marker points of the near-infrared II fluorescence image, and the marker points of the magnetic particle image. Region of interest segmentation operations are performed on the registered near-infrared II fluorescence image, the registered magnetic particle image, and the registered CT image, respectively. Multimodal fusion processing is performed on the regions of interest of the near-infrared II fluorescence image, the regions of interest of the magnetic particle image, and the regions of interest of the CT image to obtain the lesion region of the target organism.

[0008] According to an embodiment of the present invention, the above-mentioned multimodal image acquisition device includes a near-infrared II fluorescence imaging module, a magnetic particle imaging module, and a CT imaging module; wherein, the near-infrared II fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit, and a signal processing unit; wherein, the magnetic particle imaging module includes a gradient magnetic field generating unit, a driving unit, and an excitation and receiving unit; wherein, the CT imaging module includes an X-ray generating unit, a detector unit, and a three-dimensional spiral scanning mechanism.

[0009] According to an embodiment of the present invention, the above-mentioned multimodal imaging operation performed on a target organism injected with a fluorescent-magnetic particle dual-mode tracer using a multimodal image acquisition device to obtain the original near-infrared II fluorescence image, magnetic particle image and CT image of the target organism includes: in the magnetic particle image imaging process, by setting a gradient magnetic field generating unit, a driving unit and an excitation and receiving unit, a magnetic particle imaging module with a dual-stage feedthrough compensation circuit is obtained, and the target organism is scanned using the magnetic particle imaging module with the dual-stage feedthrough compensation circuit to obtain the magnetic particle image.

[0010] According to an embodiment of the present invention, the above-described automatic thresholding and connected component filtering operation on the CT image to obtain marker points representing the contour edges of the target organism in the CT image includes: performing noise reduction and grayscale normalization on the CT image to obtain a preprocessed CT image, and calculating the grayscale histogram of the preprocessed CT image; iteratively performing a traversal operation of candidate thresholds by calculating the inter-class variance of the grayscale histogram to obtain the maximum threshold, and generating a binary image of the preprocessed CT image based on the maximum threshold; performing connectivity analysis on the binary image and calculating the pixel area of ​​each connected component in the binary image, and performing segmentation and filtering on the connected components based on the calculated pixel area; and performing edge detection and marker point sampling on the connected component filtering results using a preset edge detection algorithm to obtain the marker points of the preprocessed CT image.

[0011] According to an embodiment of the present invention, the above-mentioned denoising and gray-level normalization processing of CT images to obtain preprocessed CT images includes: in image regions with uniform imaging, denoising processing of the CT images is performed using a Gaussian filtering algorithm and / or a median filtering algorithm to obtain CT images that preserve edge structures; in image regions with non-uniform imaging, the gray-level distribution of the CT images is corrected using a homomorphic filtering algorithm and / or a histogram equalization filtering algorithm; and gray-level normalization processing is completed by linearly mapping the Hausfield cells of the filtered CT images to a standard gray-level range to obtain preprocessed CT images.

[0012] According to an embodiment of the present invention, the above-mentioned method of obtaining the marker points of the near-infrared II fluorescence image and the marker points of the magnetic particle image through key point detection includes: performing denoising, image enhancement, and pixel normalization processing on the near-infrared II fluorescence image to obtain a preprocessed near-infrared II fluorescence image; performing multi-scale Gaussian blur processing on the preprocessed near-infrared II fluorescence image and calculating the Gaussian difference between adjacent scales to obtain a Gaussian difference pyramid; and performing extreme point detection and key point localization operations on the Gaussian difference pyramid to obtain the marker points of the preprocessed near-infrared II fluorescence image.

[0013] According to an embodiment of the present invention, the above-mentioned marker points for obtaining near-infrared II fluorescence images and magnetic particle images by key point detection further include: performing three-dimensional Gaussian convolution preprocessing on the magnetic particle image to obtain the three-dimensional scale space of the magnetic particle image; performing three-dimensional connected component extraction processing on the three-dimensional scale space and filtering processing based on area threshold and position to obtain the marker points of the magnetic particle image.

[0014] According to an embodiment of the present invention, the above-described registration operation of the marker points of the CT image, the marker points of the near-infrared II fluorescence image, and the marker points of the magnetic particle image includes: projecting the marker points of the CT image onto the imaging plane of the near-infrared II fluorescence image, and estimating the initial rotation matrix and translation vector through principal component analysis; based on the initial rotation matrix and translation vector, performing the registration operation of the corresponding marker points of the near-infrared II fluorescence image and the CT image using an iterative nearest-point search algorithm; filtering the marker points of the CT image and the magnetic particle image, and performing the registration operation of the corresponding marker points of the CT image and the magnetic particle image after marker point filtering by dynamically adjusting the iteration step size using an iterative nearest-point search algorithm; using the CT image as a reference, calculating the first registration transformation matrix between the CT image and the magnetic particle image and the second registration transformation matrix between the CT image and the near-infrared II fluorescence image; and performing the registration operation of the corresponding marker points of the magnetic particle image and the near-infrared II fluorescence image using the first registration transformation matrix and the second registration transformation matrix.

[0015] A second aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] A third aspect of the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0017] The near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method provided by this invention achieves simultaneous acquisition of three-modal medical images through multimodal image acquisition equipment, and achieves precise fusion of the three-modal images through registration. At the same time, the method provided by this invention combines the advantages of high resolution and high sensitivity of near-infrared two-zone fluorescence molecular imaging, the advantages of magnetic particle imaging without imaging depth limitations and linear quantification, and the advantages of CT in providing high-resolution anatomical structures. It realizes high-resolution, high-sensitivity, and depth-limit-free three-dimensional precise visualization and quantitative analysis of biological tissue structure and function, providing more accurate intermediate information for the diagnosis of lesion areas. Attached Figure Description

[0018] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0019] Figure 1 This is a flowchart of a near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of a multimodal image acquisition device according to an embodiment of the present invention.

[0021] Figure 3(a) is a schematic diagram of the structure of each functional unit in the magnetic particle imaging module according to an embodiment of the present invention.

[0022] Figure 3(b) is a schematic diagram of the structure of the receiving coil according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the near-infrared two-zone fluorescence-CT rotation acquisition control system according to an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram illustrating the effect of near-infrared two-zone-magnetic particle-CT three-modal fusion imaging according to an embodiment of the present invention.

[0025] Figure 6 This is a block diagram of an electronic device suitable for implementing a near-infrared two-zone-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present invention. Detailed Implementation

[0026] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] In view of the problems existing in FMI, MPI and CT imaging, this invention proposes a three-modal fusion imaging system and method. By fusing three different modalities, combining the high resolution and high sensitivity advantages of NIR-II FMI, the no imaging depth limitation and linear quantization advantage of MPI, and the high resolution anatomical structure advantage of CT, a seamless combination of high resolution, high sensitivity molecular functional imaging, quantitative tracing and anatomical structure is achieved.

[0031] Figure 1 This is a flowchart of a near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present invention.

[0032] like Figure 1 As shown, the above-mentioned near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method includes operations S110 to S160.

[0033] In operation S110, a multimodal imaging operation is performed on a target organism injected with a fluorescent-magnetic particle dual-mode tracer using a multimodal image acquisition device to obtain the original near-infrared II fluorescence image, magnetic particle image and CT image of the target organism.

[0034] The aforementioned multimodal image acquisition device includes a fluorescence imaging module, a magnetic particle imaging module, and a CT imaging module, which can simultaneously acquire multimodal medical images of the target organism, thereby achieving more accurate localization of the lesion area of ​​the target organism.

[0035] In operation S120, automatic thresholding and connected component filtering are performed on the CT image to obtain marker points in the CT image that represent the contour edges of the target organism.

[0036] In the above operation S120, the maximum inter-class variance algorithm can be selected to segment and filter connected components of the CT image.

[0037] In operation S130, a defocus blur removal operation based on deconvolution is iteratively performed on the original near-infrared II fluorescence image to obtain a near-infrared II fluorescence image. The marker points of the near-infrared II fluorescence image and the marker points of the magnetic particle image are obtained through key point detection operation.

[0038] The Richardson-Lucy deconvolution algorithm was used to remove defocus blur from the near-infrared II fluorescence image. Scale-Invariant Feature Transform (SIFT) was used to detect marker points in both the near-infrared II fluorescence image and the magnetic particle image, respectively, to obtain the marker points in the near-infrared II fluorescence image and the magnetic particle image.

[0039] The markers in the near-infrared II fluorescence image and the magnetic particle image correspond to the markers in the CT image. That is, the location of the target organism represented by the markers in the CT image corresponds to the markers in the near-infrared II fluorescence image and the magnetic particle image.

[0040] In operation S140, registration of the corresponding marker points is performed on the marker points of the CT image, the marker points of the near-infrared II fluorescence image, and the marker points of the magnetic particle image.

[0041] This invention employs the Iterative Closest Point (ICP) algorithm to register images of the above three modalities.

[0042] In operation S150, region of interest segmentation is performed on the registered near-infrared II fluorescence image, the registered magnetic particle image, and the registered CT image, respectively.

[0043] In operation S160, multimodal fusion processing is performed on the region of interest (ROI) of the near-infrared II fluorescence image, the region of interest (ROI) of the magnetic particle image, and the region of interest (ROI) of the CT image to obtain the lesion region of the target organism.

[0044] The three registered medical images are fused using a trained multimodal fusion model or a deep learning algorithm.

[0045] In the above-described operations S110 to S160, the target organism involved in the embodiments, specific embodiments, specific implementation methods, or experiments of the present invention is generally a laboratory animal such as a mouse. However, the method provided by the present invention is also applicable to humans. When the target organism is a human, permission from the target organism is required to perform multimodal imaging on the target organism.

[0046] It should be specifically noted that, when the target organism is a human, the multimodal medical images or other information and data that may involve the privacy of the target organism involved in this invention are all information and data authorized by the target organism or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation interfaces for users to choose to authorize or refuse.

[0047] Meanwhile, when the target organism is a human, the purpose of the above operations S110 to S160 is to provide more accurate intermediate information for the diagnosis of the lesion area, rather than to directly obtain the diagnostic information or health status of the target organism; at the same time, the above operations S110 to S160 and other operations in the embodiments of the present invention are all information processing operations performed by a computer or other device.

[0048] The near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method provided by this invention achieves simultaneous acquisition of three-modal medical images through multimodal image acquisition equipment, and achieves precise fusion of the three-modal images through registration. At the same time, the method provided by this invention combines the advantages of high resolution and high sensitivity of near-infrared two-zone fluorescence molecular imaging, the advantages of magnetic particle imaging without imaging depth limitations and linear quantification, and the advantages of CT in providing high-resolution anatomical structures. It realizes high-resolution, high-sensitivity, and depth-limit-free three-dimensional precise visualization and quantitative analysis of biological tissue structure and function, providing more accurate intermediate information for the diagnosis of lesion areas.

[0049] According to an embodiment of the present invention, the above-mentioned multimodal image acquisition device includes a near-infrared II fluorescence imaging module, a magnetic particle imaging module, and a CT imaging module; wherein, the near-infrared II fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit, and a signal processing unit; wherein, the magnetic particle imaging module includes a gradient magnetic field generating unit, a driving unit, and an excitation and receiving unit; wherein, the CT imaging module includes an X-ray generating unit, a detector unit, and a three-dimensional spiral scanning mechanism.

[0050] According to an embodiment of the present invention, the above-mentioned multimodal imaging operation performed on a target organism injected with a fluorescent-magnetic particle dual-mode tracer using a multimodal image acquisition device to obtain the original near-infrared II fluorescence image, magnetic particle image and CT image of the target organism includes: in the magnetic particle image imaging process, by setting a gradient magnetic field generating unit, a driving unit and an excitation and receiving unit, a magnetic particle imaging module with a dual-stage feedthrough compensation circuit is obtained, and the target organism is scanned using the magnetic particle imaging module with the dual-stage feedthrough compensation circuit to obtain the magnetic particle image.

[0051] The following specific embodiments, in conjunction with the appendix, demonstrate this process. Figures 2-4 The multimodal image acquisition device provided by the present invention will be described in further detail.

[0052] Figure 2 This is a schematic diagram of the structure of a multimodal image acquisition device according to an embodiment of the present invention.

[0053] Figure 3(a) is a schematic diagram of the structure of each functional unit in the magnetic particle imaging module according to an embodiment of the present invention.

[0054] Figure 3(b) is a schematic diagram of the structure of the receiving coil according to an embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of the near-infrared two-zone fluorescence-CT rotation acquisition control system according to an embodiment of the present invention.

[0056] like Figure 2 As shown, the multimodal image acquisition device provided by this invention has a near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging module, and each module works collaboratively with the control system through a mechanical interface. Among them, as... Figure 2 As shown, the aforementioned equipment includes an optical imaging area: integrating a NIR-II fluorescence module and a CT module, achieving synchronous rotational acquisition via a planar turntable. The optical imaging area mainly includes an X-ray tube, an NIR-II CMOS camera, a laser, and an X-ray detector. Imaging control in the optical imaging area is achieved through an acquisition control system and a motion control system. A magnetic particle imaging area, independent of the optical imaging area, achieves sample transfer via a high-precision translation stage (or high-precision translation stage and animal bed), including a permanent magnet and a drive coil. The permanent magnet is connected to the power supply module, and the drive coil is connected to a high-sensitivity signal processing module, which includes an amplifier, a notch filter, and a signal processing and visualization module. The control system and data processing area includes a PLC (Programmable Logic Controller), a host computer (e.g., a development environment programmed using the graphical programming language G: LabVIEW software), and a multimodal data fusion processing unit. Figure 2 The image processing workstation shown is used to receive, process, and display image signals. The impedance matching circuit is used between the high-sensitivity signal processing module and the power supply module to maximize power transfer and reduce power reflection to the amplifier, thereby improving energy transfer efficiency.

[0057] The near-infrared II (NIR-II) fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit, and a signal processing unit.

[0058] The laser excitation unit employs a 1000nm pulsed laser with an adjustable repetition frequency ranging from 10 to 100kHz. The output power density is monitored and controlled in real-time by a power meter to be ≤100mW / cm². The laser is coupled to a ring illuminator via optical fiber, and internal diffuse reflection elements ensure that the uniformity error of the excitation light intensity is ≤5%, guaranteeing uniform excitation on the sample surface.

[0059] The fluorescence acquisition unit features a back-illuminated sCOMS camera equipped with a deep cooling system (temperature reduced to -40℃), achieving a quantum efficiency of over 90%, a pixel size of 6.5μm × 6.5μm, and a 1000-1700nm long-pass filter to filter excitation stray light. A high numerical aperture objective lens (NA=0.8, silicon-based material, working distance 10-20mm) acquires signals from tissues at a depth of 1-2cm.

[0060] The signal processing unit removes defocus blur from the original image using the Richardson-Lucy deconvolution algorithm and segments the region of interest (ROI) using the Otsu thresholding algorithm (an automatic threshold selection algorithm for image segmentation).

[0061] In the near-infrared II region fluorescence imaging process, the laser power density is 80mW / cm², and after being coupled by optical fiber, it is uniformly output to the imaging area. The sCOMS camera acquires fluorescence images with an exposure time of 100ms. One image is acquired every 2° rotation of the turntable, resulting in a total of 180 fluorescence spot images.

[0062] The magnetic particle imaging (MPI) module includes a gradient magnetic field generating unit, a driving unit, and an excitation and receiving unit. Figure 3(a) schematically illustrates the specific structure of the gradient magnetic field generating unit, driving unit, and excitation and receiving unit.

[0063] As shown in Figure 3(a), the gradient magnetic field generating unit consists of a pair of NdFeBN52 permanent magnets symmetrically distributed with a spacing of 45 mm, generating a gradient magnetic field of 0.3-2.0 T / m / µ0 in the z-axis direction. The FFP (Field-Free Point) position is calibrated by a gaussmeter.

[0064] As shown in Figure 3(a), the driving unit consists of y-axis and z-axis driving coils (using Helmholtz coil structure with a spacing equal to a radius of 40mm). When a 20Hz sinusoidal current is applied, the FFP translation speed is 1mm / s and the scanning step size is 100μm, enabling three-dimensional (50mm×50mm×50mm) scanning.

[0065] As shown in Figure 3(a), the excitation and receiving unit is as follows: the excitation coil is a hollow solenoid (0.1mm×300 strands of Litz wire, 111 turns, wound in two sections, with an inductance of 150μH), and a 25kHz, 27mT / µ0 sinusoidal current is applied. The water cooling system controls the coil temperature rise to ≤5℃.

[0066] Figure 3(a) also schematically illustrates the positional relationship between the high-precision translation stage and the receiving coil, with the high-precision translation stage used to place the target organism (e.g., laboratory animal) to be imaged.

[0067] The receiving coil has a three-stage gradient structure (as shown in Figure 3(b)):

[0068] Signal detection coil: Single-layer Litz wire (0.1mm×35 strands, 50 turns, inductance 80μH), sensitivity to SPIONs signal 10nV / (mg / mLFe), wherein the voltage preamplifier is used to amplify the received voltage signal, and the amplified signal is processed on the computer / DAQ (Data Acquisition), and the processed result is fed back to the voltage preamplifier as a 25kHz modulated signal.

[0069] Compensation coil: 25 turns at each end, wound in reverse, with a mutual inductance coefficient of -0.95 with the signal detection coil. It suppresses 85% of the feedthrough signal in the first stage, and the residual signal peak-to-peak value is ≤4V.

[0070] Two-stage feedthrough compensation circuit:

[0071] Phase 1: Adjust the number of turns in the compensation coil (±5 turns) to make the voltage difference u between the signal detection coil and the compensation coil equal. pc =u d -u c Peak-to-peak value satisfies V pp (u pc ≤4V, suppressing 10-50kHz feedthrough signals.

[0072] The second stage: NI DAQ (National Instruments Data Acquisition, hardware equipment and supporting software for data acquisition and control) generates a 25kHz modulated signal (phase difference of 180°±5° from the residual feedthrough signal). After being processed by a differential amplifier circuit with a gain of 1000 times, the feedthrough signal is suppressed to ≤10mV and the fundamental frequency signal retention rate is >95%.

[0073] In the process of magnetic particle imaging Figure 2The high-precision translation stage shown transfers the fixed bed to the magnetic particle imaging module (MPI module). After initial FFP position calibration, a 20Hz current is applied to the drive coil, controlling the FFP to perform three-dimensional scanning at a step size of 100μm and a speed of 1mm / s. A 25kHz current is applied to the excitation coil to magnetize the probe, and the receiving coil synchronously acquires signals. After two-stage feedthrough compensation, the X-space algorithm is used to reconstruct the probe concentration distribution.

[0074] The CT imaging module includes an X-ray generating unit, a detector unit, and a three-dimensional spiral scanning mechanism.

[0075] The X-ray generating unit features a microfocus X-ray sphere with adjustable voltage (80-120kV) and adjustable current (50-200mA). It is equipped with a 0.5mm beryllium window and a lead collimator to generate an X-ray beam with a fan-shaped angle of 45°, covering an area with a diameter of 50mm.

[0076] The detector unit is a cadmium telluride semiconductor detector array (100μm pixel pitch, 1024×1024 channels, 16-bit dynamic range), with a single frame acquisition time of 50ms and a quantum detection efficiency (QDE) of 80% at 80keV, effectively reducing motion artifacts.

[0077] Among them, the three-dimensional spiral scanning mechanism: a planar turntable (300mm in diameter) drives the X-ray tube and detector to rotate synchronously, performing a 360° scan at a step size of 2°, acquiring 180 projection data images, with a layer thickness adjustable from 0.1 to 1.0mm.

[0078] During CT imaging, Figure 2 The high-precision translation stage shown pushes the fixed bed below the planar turntable. The X-ray tube voltage is 100kV and the current is 150mA. It performs spiral scanning at a slice thickness of 0.5mm and a speed of 1 revolution / second, acquiring 180 projection data. The three-dimensional anatomical image is reconstructed by filtered back projection (FBP).

[0079] The motion control system of the multimodal image acquisition device provided by this invention mainly refers to the near-infrared two-zone fluorescence-CT rotation acquisition control system, such as... Figure 4As shown, the system includes a near-infrared camera, laser, X-ray detector, X-ray tube, translation turntable, motion control system, and acquisition control system. The motion control system and acquisition control system are connected to a computer; the acquisition control system is connected to the X-ray detector and near-infrared camera; and the motion control system, high-precision translation stage, and animal bed. The near-infrared II-zone fluorescence-CT rotation acquisition control system uses a planar turntable (vertically set, 300mm in diameter) integrating a near-infrared fluorescence imaging module and a CT module, capable of 360° rotation scanning around a central axis. Positioning slots are provided on the turntable's edge for precise docking with the positioning points on the fixed bed. The turntable rotates at 10° / s, triggering a fluorescence camera exposure (exposure time 100ms) and CT projection acquisition every 2° rotation. The high-precision translation stage (i.e., the high-precision translation stage): The air-bearing guide rail translation stage connects to the fixed bed via a slot-type interface. After fluorescence / CT acquisition is completed, it is laterally translated to the center of the MPI module at a speed of 50mm / s.

[0080] The multimodal image acquisition device provided by this invention has a compact structure, with each imaging module capable of independent or combined imaging, making it suitable for different experimental needs and sample types. It provides a multifunctional platform for biomedical research and clinical diagnosis. The device's reset and data storage are as follows: the laser, X-ray tube, magnetic field generating unit, and drive unit are shut down; the high-precision translation stage returns the fixed bed to its initial position; and the robotic arm returns to its original position. Raw data and fusion results are stored on the server in DICOM (Digital Imaging and Communications in Medicine) format, while a metadata file containing imaging parameters and registration matrices is generated for subsequent traceability and analysis.

[0081] According to an embodiment of the present invention, the above-described automatic thresholding and connected component filtering operation on the CT image to obtain marker points representing the contour edges of the target organism in the CT image includes: performing noise reduction and grayscale normalization on the CT image to obtain a preprocessed CT image, and calculating the grayscale histogram of the preprocessed CT image; iteratively performing a traversal operation of candidate thresholds by calculating the inter-class variance of the grayscale histogram to obtain the maximum threshold, and generating a binary image of the preprocessed CT image based on the maximum threshold; performing connectivity analysis on the binary image and calculating the pixel area of ​​each connected component in the binary image, and performing segmentation and filtering on the connected components based on the calculated pixel area; and performing edge detection and marker point sampling on the connected component filtering results using a preset edge detection algorithm to obtain the marker points of the preprocessed CT image.

[0082] According to an embodiment of the present invention, the above-mentioned denoising and gray-level normalization processing of CT images to obtain preprocessed CT images includes: in image regions with uniform imaging, denoising processing of the CT images is performed using a Gaussian filtering algorithm and / or a median filtering algorithm to obtain CT images that preserve edge structures; in image regions with non-uniform imaging, the gray-level distribution of the CT images is corrected using a homomorphic filtering algorithm and / or a histogram equalization filtering algorithm; and gray-level normalization processing is completed by linearly mapping the Hausfield cells of the filtered CT images to a standard gray-level range to obtain preprocessed CT images.

[0083] The above embodiments involve key point detection and connected component filtering of CT images based on the Ostu algorithm. Through the above embodiments, the outline of the target organism can be obtained, and spatial localization of specific regions (e.g., lesion regions) of the target organism can be achieved by combining marker points (or metal marker points).

[0084] The study employs a combination of Gaussian filtering and median filtering to effectively suppress noise while preserving edge structure. For regions with non-uniform imaging, homomorphic filtering (to eliminate illumination inhomogeneities) and histogram equalization (to enhance contrast) are introduced for local correction.

[0085] Among them, the Hounsfield Unit (or HU value) is linearly mapped to a standard grayscale range (e.g., 0-255) to solve the problem of grayscale differences between different scanning devices.

[0086] The optimal threshold is automatically determined by iteratively calculating and maximizing the inter-class variance, which is particularly suitable for CT images with bimodal histograms. This algorithm has low time complexity and is suitable for real-time imaging.

[0087] This process involves calculating the pixel area of ​​connected components and setting a threshold to effectively filter artifacts caused by noise. An edge detection algorithm (such as the Canny algorithm) is used for sub-pixel-level edge extraction, and non-maximum suppression is employed to preserve fine structure.

[0088] According to an embodiment of the present invention, the above-mentioned method of obtaining the marker points of the near-infrared II fluorescence image and the marker points of the magnetic particle image through key point detection includes: performing denoising, image enhancement, and pixel normalization processing on the near-infrared II fluorescence image to obtain a preprocessed near-infrared II fluorescence image; performing multi-scale Gaussian blur processing on the preprocessed near-infrared II fluorescence image and calculating the Gaussian difference between adjacent scales to obtain a Gaussian difference pyramid; and performing extreme point detection and key point localization operations on the Gaussian difference pyramid to obtain the marker points of the preprocessed near-infrared II fluorescence image.

[0089] Near-infrared II fluorescence images require denoising (e.g., non-local mean denoising), contrast enhancement (CLAHE algorithm: Contrast Limited Adaptive Histogram Equalization), and pixel normalization. These operations eliminate environmental noise interference, improve the signal-to-noise ratio in weak signal regions, and provide standardized input data for subsequent multi-scale analysis.

[0090] Through multi-scale Gaussian blur ( An image pyramid is generated by increasing the value gradient, and the difference pyramid (DoG) is obtained by subtracting adjacent scales. This process simulates the characteristics of human vision and can effectively capture the features of marker points of different sizes.

[0091] Candidate points are located using a three-dimensional extremum search (spatial + scale dimension), and coordinate offset is corrected using Taylor expansion. Unstable points are eliminated by contrast thresholding, and edge response interference is eliminated using the Hessian matrix, ultimately obtaining the coordinates of the marker points with sub-pixel accuracy.

[0092] The SIFT algorithm used in this invention, through its scale invariance, is applicable to marker detection in both near-infrared fluorescence (high dynamic range) and magnetic particle images (low signal-to-noise ratio) without the need to adjust core parameters.

[0093] According to an embodiment of the present invention, the above-mentioned marker points for obtaining near-infrared II fluorescence images and magnetic particle images by key point detection further include: performing three-dimensional Gaussian convolution preprocessing on the magnetic particle image to obtain the three-dimensional scale space of the magnetic particle image; performing three-dimensional connected component extraction processing on the three-dimensional scale space and filtering processing based on area threshold and position to obtain the marker points of the magnetic particle image.

[0094] The above embodiments expand the two-dimensional Gaussian kernel into a three-dimensional convolution kernel to process MPI images. Through different... The three-dimensional Gaussian filter is used to smooth the original image at multiple scales, forming a three-dimensional scale space pyramid, which effectively preserves the distribution characteristics of magnetic particles at different resolutions.

[0095] The above embodiments, through three-dimensional Gaussian convolution and connected component analysis, have for the first time achieved feature point detection in three-dimensional volume data such as MPI, solving the problem of locating stereoscopic marker points in medical images.

[0096] By adjusting the Gaussian kernel parameters, near-infrared II fluorescence images (two-dimensional) and MPI images (three-dimensional) can be processed simultaneously, realizing a unified detection framework for multimodal image markers.

[0097] According to an embodiment of the present invention, the above-described registration operation of the marker points of the CT image, the marker points of the near-infrared II fluorescence image, and the marker points of the magnetic particle image includes: projecting the marker points of the CT image onto the imaging plane of the near-infrared II fluorescence image, and estimating the initial rotation matrix and translation vector through principal component analysis; based on the initial rotation matrix and translation vector, performing the registration operation of the corresponding marker points of the near-infrared II fluorescence image and the CT image using an iterative nearest-point search algorithm; filtering the marker points of the CT image and the magnetic particle image, and performing the registration operation of the corresponding marker points of the CT image and the magnetic particle image after marker point filtering by dynamically adjusting the iteration step size using an iterative nearest-point search algorithm; using the CT image as a reference, calculating the first registration transformation matrix between the CT image and the magnetic particle image and the second registration transformation matrix between the CT image and the near-infrared II fluorescence image; and performing the registration operation of the corresponding marker points of the magnetic particle image and the near-infrared II fluorescence image using the first registration transformation matrix and the second registration transformation matrix.

[0098] The above embodiments mainly involve initial registration, ICP optimization, and transformation transfer. Initial registration involves projecting CT markers onto the NIR-II FMI imaging plane and estimating the initial pose matrix using PCA (Principal Component Analysis). ICP optimization employs a dynamic step size adjustment strategy to complete dual-modal registration of CT and NIR-II FMI, and CT and MPI, respectively. Transform transfer derives the third transformation matrix between NIR-II FMI and MPI through matrix operations between the first registration matrix (CT→MPI) and the second registration matrix (CT→NIR-II FMI).

[0099] In the above embodiments, dynamic step size ICP is used for magnetic particle image registration. The search step size is adaptively adjusted according to the point cloud density to avoid local optima. At the same time, outlier points are filtered by curvature features and marker points are used for screening to improve the robustness of registration and multi-level accuracy control.

[0100] The above embodiments are the first to achieve a unified coordinate system mapping of three physical quantity images: X-ray (CT image), optical (near-infrared II fluorescence image), and magnetic (magnetic particle image).

[0101] Figure 5 This is a schematic diagram illustrating the effect of near-infrared two-zone-magnetic particle-CT three-modal fusion imaging according to an embodiment of the present invention.

[0102] The near-infrared two-region-magnetic particle-CT three-modal fusion imaging method provided by this invention is based on a proposed deep learning algorithm. It performs deep fusion on registered images, displaying anatomical structures from the CT images in grayscale, and showing the three-dimensional spatial distribution of the fused probes in pseudo-color. Figure 5 As shown, where, Figure 5 (a) in the image represents the near-infrared II fluorescence image. Figure 5 (b) in the image represents a magnetic particle image. Figure 5 (c) in the image represents a CT image. Figure 5 In the diagram, (d) represents the result of the three-modal fusion.

[0103] Figure 6 This is a block diagram of an electronic device suitable for implementing a near-infrared two-zone-magnetic particle-CT three-modal fusion imaging method according to an embodiment of the present invention.

[0104] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0105] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0106] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0107] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the near-infrared two-zone fluorescence-magnetic particle-CT three-modal fusion imaging method according to embodiments of the present invention.

[0108] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0110] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0111] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A near-infrared two-region fluorescence-magnetic particle-CT three-modal fusion imaging method, characterized in that, The method includes: Multimodal imaging was performed on a target organism injected with a fluorescent-magnetic particle dual-mode tracer using a multimodal image acquisition device to obtain the original near-infrared II fluorescence image, magnetic particle image and CT image of the target organism; Automatic thresholding and connected component filtering operations are performed on the CT image to obtain marker points in the CT image that characterize the contour edge of the target organism; The original near-infrared II fluorescence image is iteratively subjected to a deconvolution-based defocus blur removal operation to obtain a near-infrared II fluorescence image. The marker points of the near-infrared II fluorescence image and the marker points of the magnetic particle image are obtained through a key point detection operation. The registration operation is performed on the marker points of the CT image, the marker points of the near-infrared II fluorescence image, and the marker points of the magnetic particle image. Region of interest segmentation was performed on the registered near-infrared II fluorescence image, the registered magnetic particle image, and the registered CT image, respectively. Multimodal fusion processing is performed on the region of interest of the near-infrared fluorescence image, the region of interest of the magnetic particle image, and the region of interest of the CT image to obtain the lesion region of the target organism.

2. The method according to claim 1, characterized in that, The multimodal image acquisition device includes a near-infrared two-zone fluorescence imaging module, a magnetic particle imaging module, and a CT imaging module; The near-infrared second-zone fluorescence imaging module includes a laser excitation unit, a fluorescence acquisition unit, and a signal processing unit. The magnetic particle imaging module includes a gradient magnetic field generating unit, a driving unit, and an excitation and receiving unit. The CT imaging module includes an X-ray generating unit, a detector unit, and a three-dimensional spiral scanning mechanism.

3. The method according to claim 2, characterized in that, Multimodal imaging was performed on a target organism injected with a fluorescent-magnetic particle dual-mode tracer using a multimodal image acquisition device, resulting in the original near-infrared II fluorescence image, magnetic particle image, and CT image of the target organism, including: In the magnetic particle image imaging process, by setting the gradient magnetic field generating unit, the driving unit, and the excitation and receiving unit, a magnetic particle imaging module with a dual-stage feedthrough compensation circuit is obtained, and the target organism is scanned using the magnetic particle imaging module with the dual-stage feedthrough compensation circuit to obtain the magnetic particle image.

4. The method according to claim 1, characterized in that, Automatic thresholding and connected component filtering are performed on the CT image to obtain marker points in the CT image that characterize the contour edge of the target organism, including: The CT image is subjected to noise reduction and grayscale normalization to obtain a preprocessed CT image, and the grayscale histogram of the preprocessed CT image is calculated. The candidate threshold is iteratively traversed by calculating the inter-class variance of the grayscale histogram to obtain the maximum threshold, and a binary image of the preprocessed CT image is generated based on the maximum threshold. The binary graph is subjected to connectivity analysis and the pixel area of ​​each connected component in the binary graph is calculated. Based on the calculated pixel area, the connected components are segmented and filtered. The connected component filtering results are processed by edge detection and marker point sampling using a preset edge detection algorithm to obtain the marker points of the preprocessed CT image.

5. The method according to claim 4, characterized in that, The CT images are subjected to noise reduction and grayscale normalization to obtain preprocessed CT images, including: In a uniformly imaged region, the CT image is denoised using a Gaussian filtering algorithm and / or a median filtering algorithm to obtain a CT image that preserves the edge structure. In image regions with uneven imaging, the grayscale distribution of the CT image is corrected using homomorphic filtering and / or histogram equalization filtering algorithms; The gray-level normalization process is completed by linearly mapping the Hausfield cells of the filtered CT image to the standard gray-level range, thus obtaining the preprocessed CT image.

6. The method according to claim 1, characterized in that, The marker points obtained through key point detection operations for the near-infrared II fluorescence image and the magnetic particle image include: The near-infrared II fluorescence image is subjected to denoising, image enhancement and pixel normalization to obtain a preprocessed near-infrared II fluorescence image. The preprocessed near-infrared II fluorescence image is subjected to multi-scale Gaussian blurring and the Gaussian difference between adjacent scales is calculated to obtain the Gaussian difference pyramid. The extreme point detection operation and key point localization operation are performed on the Gaussian difference pyramid to obtain the marked points of the preprocessed near-infrared II fluorescence image.

7. The method according to claim 6, characterized in that, Also includes: The magnetic particle image is preprocessed by three-dimensional Gaussian convolution to obtain the three-dimensional scale space of the magnetic particle image; The three-dimensional scale space is processed by three-dimensional connected component extraction and filtered based on area threshold and position to obtain the marker points of the magnetic particle image.

8. The method according to claim 1, characterized in that, The registration operation for the marker points in the CT image, the near-infrared II fluorescence image, and the magnetic particle image includes: The marker points of the CT image are projected onto the imaging plane of the near-infrared II fluorescence image, and the initial rotation matrix and translation vector are estimated by principal component analysis. Based on the initial rotation matrix and the translation vector, the near-infrared II fluorescence image and the CT image are registered using the iterative nearest point search algorithm to perform the registration operation of the corresponding marker points; The marker points of the CT image and the magnetic particle image are filtered, and the iterative nearest point search algorithm is used to perform registration operation on the corresponding marker points of the CT image and the magnetic particle image after the marker point filtering is dynamically adjusted. Using the CT image as a reference, calculate the first registration transformation matrix between the CT image and the magnetic particle image, and the second registration transformation matrix between the CT image and the near-infrared II fluorescence image; Using the first registration transformation matrix and the second registration transformation matrix, a registration operation is performed on the magnetic particle image and the near-infrared II fluorescence image for corresponding marker points.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images