Three-dimensional magnetic particle imaging enhancement method and system based on distillation structure contrast learning

Abstract

The invention belongs to the field of magnetic particle imaging, particularly relates to a three-dimensional magnetic particle imaging enhancement method and system based on distillation structure contrast learning, and aims to solve the problems of anisotropic resolution and background noise interference caused by sparse scanning tracks in three-dimensional magnetic particle imaging. The method comprises the following steps: constructing a data set; building a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; training a two-dimensional isotropic teacher network model and storing model parameters; the three-dimensional resolution improvement student network model obtains model parameters and then carries out training, an obtained three-dimensional training result is converted into two-dimensional training input data, and the two-dimensional training input data is input into the two-dimensional isotropic teacher network model to generate a supervision benchmark which is used for updating and optimizing the three-dimensional resolution improvement student network model; and after training, carrying out resolution improvement processing on the input three-dimensional magnetic particle imaging image. According to the method, the isotropic resolution of the three-dimensional magnetic particle image is improved, and background noise is effectively suppressed.

Description

Background Art

[0002] In clinical diagnosis and detection, how to accurately and objectively locate tumors and other lesions has always been a research hotspot and challenging problem internationally. Existing medical imaging technologies such as CT, MRI, SPECT, etc. all have problems such as great harm, poor localization, and low accuracy. In recent years, a brand-new tracer-based imaging method - magnetic particle imaging technology (MPI) has been proposed. Using tomographic imaging technology, MPI can accurately locate tumors or targets by detecting the spatial concentration distribution of superparamagnetic iron oxide nanoparticles (SPIONs) that are harmless to the human body, and has the characteristics of three-dimensional imaging, high spatio-temporal resolution, and high sensitivity. In addition, MPI does not display anatomical structures and is not interfered by background signals. Therefore, the intensity of the signal is directly proportional to the concentration of the tracer, which is a new method with great potential for medical applications.

[0003] In MPI, x-space is a commonly used reconstruction algorithm. The x-space algorithm reconstructs by projecting the received magnetic nanoparticle (MNPs) response signal from the time domain to the spatial domain. The reconstructed image is the convolution of the MNPs concentration distribution map and the point spread function (PSF). In two-dimensional scanning, various factors such as low signal-to-noise ratio, weak gradient field strength, and small MNPs particle size will affect the quality and accuracy of the PSF, resulting in blurred reconstructed images. And because the scanning frequencies in the two directions are different, it often leads to the appearance of anisotropic resolution. Especially for Cartesian trajectories, the scanning frequencies in the two directions have an order-of-magnitude difference, resulting in obvious anisotropic resolution. When extended to three dimensions, due to the differences in gradient field strengths in each direction and the non-uniformity of the scanning trajectory, anisotropy in resolution in the three directions will also occur. In addition, when tomographic scanning is performed through mechanical displacement, the sparsity of the mechanical displacement will further cause a reduction in the z-axis resolution. In addition to anisotropic resolution, the magnetic particle scanning system will be affected by various background noises, further affecting the imaging resolution.

[0004] Based on this, the present invention proposes a three-dimensional magnetic particle imaging enhancement method and system based on contrast learning of a distillation structure. Summary of the Invention

[0005] To solve the above problems in the prior art, namely, the existing medical imaging technologies (such as CT, MRI, SPECT) have problems of great harm, inaccurate positioning, and low imaging accuracy, and in the x-space reconstruction algorithm of magnetic particle imaging technology (MPI), due to low signal-to-noise ratio, strong and weak gradient fields, and small particle sizes of MNPs during two-dimensional / three-dimensional scanning, the quality of the point spread function (PSF) is poor and the image is blurred, the anisotropic resolution is caused by the difference in scanning frequencies in each direction and the uneven gradient field strength, and the mechanical displacement sparsity reduces the z-axis resolution. At the same time, the background noise further affects the imaging quality. The present invention provides a three-dimensional magnetic particle imaging enhancement method and system based on contrastive learning of a distillation structure.

[0006] In the first aspect of the present invention, a three-dimensional magnetic particle imaging enhancement method based on contrastive learning of a distillation structure is proposed. The method includes: Construct a data set, which includes a two-dimensional isotropic teacher network image data set and a three-dimensional resolution improvement student network data set; Build a three-dimensional isotropic resolution improvement network based on contrastive learning of a distillation structure, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module. The energy focusing module is used to activate image features from the spatial dimension; Pre-train the two-dimensional isotropic teacher network model through the two-dimensional isotropic teacher network image data set and save the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network data set. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which are input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; Use the trained three-dimensional resolution improvement student network model to perform resolution improvement processing on the input real MPI image, and output an isotropic high-resolution three-dimensional image.

[0007] Further, the specific steps of constructing the two-dimensional isotropic teacher network image data set include: Step A1, use the three-dimensional blood vessel data set in MedMNIST as the original three-dimensional magnetic particle concentration distribution map; Step A2, obtain a plurality of magnetic particle imaging response signals by adjusting the driving field frequency and performing FFP layer scanning on the three-dimensional magnetic particle concentration distribution map; Step A3: Perform x-space reconstruction on multiple magnetic particle imaging response signals to obtain a two-dimensional magnetic particle imaging image in the x-y plane. Stack the two-dimensional magnetic particle imaging images of multiple x-y planes in sequence to obtain a three-dimensional magnetic particle imaging image. Slice the three-dimensional magnetic particle imaging image in the x-z plane to obtain a low-resolution two-dimensional magnetic particle imaging image in the x-z plane, which serves as the two-dimensional noise-free input data. Use the high-resolution two-dimensional magnetic particle imaging image in the x-z plane at the corresponding position in the three-dimensional magnetic particle concentration distribution map in Step A1 as the two-dimensional noise-free label data. Step A4: Add Gaussian noise with different noise levels to the low-resolution two-dimensional magnetic particle imaging image in the x-z plane and the high-resolution two-dimensional magnetic particle imaging image in the x-z plane respectively, serving as the two-dimensional noisy input data and the two-dimensional noisy label data. Step A5: Use the two-dimensional noise-free input data and the two-dimensional noise-free label data obtained in Steps A3 and A4 as the two-dimensional noise-free dataset. Divide the two-dimensional noisy input data and the two-dimensional noisy label data into multiple groups of two-dimensional noisy datasets according to different noises. Combine the two-dimensional noise-free dataset and the multiple groups of two-dimensional noisy datasets as the two-dimensional isotropic teacher network image dataset, and conduct separate training and verification on the three-dimensional isotropic resolution improvement network based on distilled structure contrast learning.

[0008] Further, the specific steps for constructing the three-dimensional resolution improvement student network dataset include: Step B1: Use the MedMNIST three-dimensional vascular dataset as the original magnetic particle concentration distribution map, and obtain a three-dimensional magnetic particle concentration distribution map with the same field of view size through resampling. Step B2: Obtain multiple magnetic particle imaging response signals by adjusting the driving field frequency and performing FFP layer scanning on the three-dimensional magnetic particle concentration distribution map. Step B3: Reconstruct the multiple magnetic particle imaging response signals using the x-space algorithm to generate two-dimensional magnetic particle imaging images. After stacking, obtain a three-dimensional magnetic particle imaging image under a low gradient field strength, which serves as the three-dimensional noise-free input data. Use the three-dimensional magnetic particle concentration distribution map in Step B1 as the three-dimensional noise-free label data after adjusting the resolution through interpolation processing. Step B4: Add Gaussian noise with different intensities to the magnetic particle imaging response signals obtained by layer scanning, and reconstruct and stack them using the x-space algorithm to obtain a noisy three-dimensional magnetic particle imaging image, which serves as the three-dimensional noisy input data. Add Gaussian noise with different intensities to the three-dimensional magnetic particle concentration distribution map in Step B1 after adjusting the resolution through interpolation processing, serving as the three-dimensional noisy label data. Step B5: Use the three-dimensional noise-free input data and three-dimensional noise-free label data obtained in Steps B3 and B4 as the three-dimensional noise-free dataset. Divide the three-dimensional noisy input data and three-dimensional noisy label data into multiple groups of three-dimensional noisy datasets according to different noises. Use the three-dimensional noise-free dataset and multiple groups of three-dimensional noisy datasets together as the three-dimensional resolution improvement student network dataset, and separately train and validate the three-dimensional isotropic resolution improvement network based on the distillation structure contrast learning.

[0009] Furthermore, the structure of the two-dimensional isotropic teacher network model includes: A dense downsampling feature compression module, which consists of n sequentially connected feature compression modules. Each of the feature compression modules consists of a feature extraction layer and a max pooling layer; A dense upsampling image reconstruction module, which consists of n energy focusing modules and n image reconstruction modules connected alternately in sequence. The image reconstruction module consists of a feature extraction layer and an upsampling layer; Among them, the i-th feature compression module establishes a residual connection with the (n - i + 1)-th image reconstruction module, where i = 1, 2, ……, n.

[0010] Furthermore, the energy focusing module includes a feature input layer, a first convolutional layer, a first normalization and activation layer, a second convolutional layer, a second normalization and activation layer, and a feature output layer connected in sequence; After receiving the input data, the feature input layer inputs it to the first convolutional layer for the first convolution process, performs the first normalization and activation on the output of the first convolution process, and performs the second convolution process through the second convolutional layer. Perform the second normalization and activation on the output of the second convolution process to obtain the attention map. After expanding the attention map to the number of channels of the input data in the channel dimension, multiply it element-wise with the input data to obtain the activated attention map. Add the activated attention map to the input data residually to obtain the output data, and output it through the feature output layer.

[0011] Furthermore, the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, specifically: Obtain the model parameters of the set modules in the two-dimensional isotropic teacher network model as the dense downsampling feature compression module and the energy focusing module, and randomly initialize the model parameters of the dense upsampling reconstruction module in the three-dimensional resolution improvement student network model.

[0012] Furthermore, input it into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model, specifically: Slice the three-dimensional training result into two-dimensional training input data, input the two-dimensional training input data into the pre-trained two-dimensional isotropic teacher network model to obtain two-dimensional training output data, calculate the contrast loss between the two-dimensional training input data and the two-dimensional training output data, and update and optimize the three-dimensional resolution improvement student network model.

[0013] Further, for the two-dimensional isotropic teacher network, Smooth loss is used to optimize it, and the specific loss function is: ; where is the high-resolution magnetic particle image output by the two-dimensional isotropic teacher network, with dimensions H×W; is the two-dimensional noise-free label data or two-dimensional noisy label data.

[0014] Further, for the three-dimensional resolution improvement student network, Smooth and InfoNCE losses are used for constraint, and the specific loss function L is: ; ; where is the high-resolution three-dimensional magnetic particle image output by the three-dimensional resolution improvement student network, with dimensions D × H × W ; is the three-dimensional noise-free label data or three-dimensional noisy label data; is the cosine similarity calculation; is the temperature coefficient; is the total number of two-dimensional tomographic images of positive and negative samples. During training, select the tomographic images at the same position as the two-dimensional image reconstruction result as positive examples, and other tomographic images as negative examples, and supervise the model training through the comparison between the selected positive and negative examples; Perform a comprehensive evaluation of Smooth and InfoNCE to obtain the loss function : L : ; where is the hyperparameter specified during training.

[0015] Further, the trained three-dimensional resolution improvement student network model is used to perform resolution improvement processing on the input three-dimensional magnetic particle imaging image. The method for obtaining the three-dimensional magnetic particle imaging image is as follows: Any collected real MPI image is subjected to layer scanning and drive field frequency adjustment processing to obtain a real magnetic particle imaging response signal, which is superimposed to form a three-dimensional magnetic particle imaging image.

[0016] On the other hand, the present invention proposes a three-dimensional magnetic particle imaging enhancement system based on contrast learning with a distillation structure, based on a three-dimensional magnetic particle imaging enhancement method with a distillation structure. The system includes: A dataset construction module configured to construct a dataset, which includes a two-dimensional isotropic teacher network image dataset and a three-dimensional resolution improvement student network dataset; A model building module configured to build a three-dimensional isotropic resolution improvement network based on contrast learning with a distillation structure, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module. The energy focusing module is used to activate image features from the spatial dimension; Among them, the parameters of the dense downsampling feature compression module and the energy focusing module in the three-dimensional resolution improvement student network model are loaded from the two-dimensional isotropic teacher network model, and the parameters of the dense upsampling image reconstruction module in the three-dimensional resolution improvement student network model are randomly initialized; A training module configured to pre-train the two-dimensional isotropic teacher network model through the two-dimensional isotropic teacher network image dataset and save the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network dataset. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which are input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; A resolution improvement module configured to use the trained three-dimensional resolution improvement student network model to perform resolution improvement processing on the input real MPI image and output an isotropic high-resolution three-dimensional image.

[0017] Advantages of the present invention: Significantly improve imaging resolution: By adopting the "teacher-student" knowledge distillation architecture, the present invention uses a two-dimensional isotropic teacher network model to guide the learning process of a three-dimensional resolution improvement student network model. This method can not only effectively improve the resolution of the image along the focusing field direction (z-axis) and achieve isotropy in the z-axis direction, but also obtain higher resolution in the xy plane through the activation and optimization of spatial dimension features, thereby comprehensively improving the overall quality of three-dimensional magnetic particle imaging.

[0018] Enhance noise suppression ability: The spatial attention module introduced in the two-dimensional isotropic teacher network can adaptively adjust the network's attention to different spatial positions and is particularly good at extracting feature information in sparse trajectory regions. This mechanism helps to effectively suppress background noise, making the finally generated image clearer, reducing artifacts or blurring caused by noise, and improving the accuracy of medical diagnosis.

[0019] Improve image reconstruction efficiency and quality: The design of the dense downsampling feature compression module and the dense upsampling image reconstruction module proposed in the present invention can not only efficiently process input data, but also ensure the quality of the output image while guaranteeing the image reconstruction speed. Especially for objects with complex structures and details, such as tumors, this method can more accurately capture their morphological features and provide more reliable data support for clinical diagnosis. Brief Description of the Drawings

[0020] Other features, purposes, and advantages of the present application will become more obvious by reading the detailed description of the non-limiting embodiments with reference to the following drawings: Figure 1 is a schematic flowchart of a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention; Figure 2 is a schematic flowchart of the MPI three-dimensional image enhancement process of a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention; Figure 3 is a schematic diagram of the structures of the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model of a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention; Figure 4 is a schematic diagram of the structure of the energy focusing module of a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention; Figure 5 is a flowchart of the training process of the network model in a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention. Detailed Embodiments

[0021] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. Additionally, it should be noted that for the convenience of description, only the parts related to the relevant invention are shown in the drawings.

[0022] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0023] The present invention provides a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning. The method includes: Constructing a data set, the data set including a two-dimensional isotropic teacher network image data set and a three-dimensional resolution improvement student network data set; Building a three-dimensional isotropic resolution improvement network based on distillation structure contrast learning, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module. The energy focusing module is used to activate image features from the spatial dimension; Pre-training the two-dimensional isotropic teacher network model through the two-dimensional isotropic teacher network image data set and saving the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network data set. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which are input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; The three-dimensional resolution improvement student network model inherits the model parameters of the dense downsampling feature compression module and the energy focusing module of the two-dimensional isotropic teacher network model, and randomly initializes the model parameters of the dense upsampling reconstruction module; Using the trained three-dimensional resolution improvement student network model to perform resolution improvement processing on the input real MPI image, and outputting an isotropic high-resolution three-dimensional image.

[0024] To more clearly illustrate a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning of the present invention, the following is combined with Figure 1 and Figure 2 to elaborate on the embodiments of the present invention. Each step is described in detail as follows: Constructing a data set, the data set including a two-dimensional isotropic teacher network image data set and a three-dimensional resolution improvement student network data set; The specific steps of constructing the two-dimensional isotropic teacher network image dataset in step S1 include: Step A1, using the three-dimensional vascular dataset in MedMNIST as the original three-dimensional magnetic particle concentration distribution map; Step A2, obtaining multiple magnetic particle imaging response signals by adjusting the driving field frequency and performing FFP layer scanning on the three-dimensional magnetic particle concentration distribution map; Specifically, the FFP scanning imaging method is adopted to scan and image along the Cartesian trajectory. For the Cartesian trajectory in three-dimensional magnetic particle imaging, it is simulated and realized by layer scanning and changing the driving field frequency. First, the three-dimensional phantom map is evenly layered, and then each layer of MNPs concentration distribution map is scanned along the Cartesian scanning trajectory by adjusting the three-dimensional driving field frequency. According to the two-dimensional magnetic particle imaging forward mathematical model, magnetic particle imaging response signals are generated. In this embodiment, the Langevin model is used to simulate and generate magnetic particle response signals under low gradient field strength; Among them, according to the generation process of magnetic particle response signals, simulated time-domain signals are generated. The Langevin model is used as the forward model to simulate the nonlinear magnetization response process of MNPs and generate magnetic particle signals. The magnetization intensity of MNPs under the action of an external magnetic field is described by the Langevin equation as follows: ; Among them, represents the magnetic moment of MNPs, and each MNPs generates a magnetic moment. represents the saturation magnetization intensity of MNPs, represents the concentration of MNPs, is related to the characteristics of MNPs. represents the magnetic field strength at the position at time represents the position of the magnetic field free point, is the magnitude of the magnetic field gradient. Among them, the assumption for the Langevin equation to hold is that there is no relaxation effect of particles, that is, the magnetic domains can quickly adjust to be consistent with the magnetic field direction under the action of the magnetic field, and there is no interaction between MNPs in the system. Within the imaging field of view, the magnetization intensity of MNPs is converted into flux: ; Among them, represents the convolution operation. It can be obtained from the above that the total magnetization intensity in the system is the convolution of the MNPs concentration and the Langevin function kernel. According to Faraday's law of electromagnetic induction, the magnetization response signal received by the detection coil from MNPs is obtained : wherein, represents the sensitivity of the receiving coil, represents the derivative of the Langevin function.

[0025] The original magnetic particle signal scanned under a low gradient magnetic field strength (3 T / m) is generated through a simulation program.

[0026] Step A3: Reconstruct the multiple magnetic particle imaging response signals in x-space to obtain a two-dimensional magnetic particle imaging image in the x-y plane. Stack the two-dimensional magnetic particle imaging images of multiple x-y planes in sequence to obtain a three-dimensional magnetic particle imaging image. Slice the three-dimensional magnetic particle imaging image in the x-z plane to obtain a low-resolution two-dimensional magnetic particle imaging image in the x-z plane, which is used as two-dimensional noise-free input data; Use the high-resolution two-dimensional magnetic particle imaging image in the x-z plane at the corresponding position in the three-dimensional magnetic particle concentration distribution map in Step A1 as two-dimensional noise-free label data; Specifically, according to the magnetization response signal of the MNPs in Step A2 , project it onto the image domain, and the concentration distribution image of the MNPs is: ; Based on the process of x-space reconstruction, the simulation generation of the two-dimensional magnetic particle imaging image is realized. The original magnetic particle signal scanned under a low gradient magnetic field strength (3 T / m) corresponds to the original magnetic particle imaging image, which is used as the original image domain; Step A4: Add Gaussian noise with different noise levels to the low-resolution two-dimensional magnetic particle imaging image in the x-z plane and the high-resolution two-dimensional magnetic particle imaging image in the x-z plane respectively, as two-dimensional noisy input data and two-dimensional noisy label data; In order to simulate real acquisition signals, the present invention adds Gaussian noise with different noise levels to generate images with signal-to-noise ratios of 20 dB, 30 dB, and 40 dB, and constructs a data set to complete the training of the anti-noise model.

[0027] Step A5: Use the two-dimensional noise-free input data and two-dimensional noise-free label data obtained in Step A3 and Step A4 as a two-dimensional noise-free data set. Divide the two-dimensional noisy input data and two-dimensional noisy label data into multiple groups of two-dimensional noisy data sets according to different noises. Use the two-dimensional noise-free data set and multiple groups of two-dimensional noisy data sets together as a two-dimensional isotropic teacher network image data set, and conduct separate training and verification on the three-dimensional isotropic resolution improvement network based on distillation structure contrast learning.

[0028] The specific steps for constructing the three-dimensional resolution improvement student network data set include: Step B1: Use the MedMNIST three-dimensional vascular dataset as the original magnetic particle concentration distribution map, and through resampling, obtain a three-dimensional magnetic particle concentration distribution map with the same size as the field of view. In the present invention, the three-dimensional vascular dataset in MedMNIST is selected as the original MNPs concentration distribution map. The original three-dimensional image is resampled to obtain an image with the same size as the field of view (20×20×20mm 3 ), and this image is used as the MNPs concentration distribution map.

[0029] Step B2: Through drive field frequency adjustment and performing FFP layer-by-layer scanning on the three-dimensional magnetic particle concentration distribution map, obtain multiple magnetic particle imaging response signals. The present invention adopts the FFP scanning imaging method to scan and image along the Cartesian trajectory. For the Cartesian trajectory in the aforementioned three-dimensional magnetic particle imaging, it is simulated and realized by means of layer-by-layer scanning and changing the drive field frequency, specifically as follows: Step B21: First, evenly divide the three-dimensional phantom map into layers (slice 1, 2,..., m), and then by adjusting the three-dimensional drive field frequency (H DX , H DY , H DZ ), use the Cartesian scanning trajectory to scan each layer of the MNPs concentration distribution map. According to the Langevin function magnetic moment of the magnetic particles in the two-dimensional magnetic particle imaging described in Step A2, generate magnetic particle imaging response signals. Generate the original magnetic particle signals scanned under a low gradient field strength (3T / m) through a simulation program.

[0030] Step B22: Use the x-space algorithm to process the generated magnetic particle signals to complete the reconstruction of the magnetic particle imaging image, and obtain a two-dimensional magnetic particle imaging image.

[0031] Step B3: Reconstruct the multiple magnetic particle imaging response signals using the x-space algorithm, generate a two-dimensional magnetic particle imaging image, and after superposition, obtain a three-dimensional magnetic particle imaging image under a low gradient field strength as the three-dimensional noise-free input data. Adjust the resolution of the three-dimensional magnetic particle concentration distribution map in Step B1 through interpolation processing as the three-dimensional noise-free label data. The present invention generates a three-dimensional magnetic particle imaging image by stacking. Among them, the original magnetic particle signals scanned under a low gradient field strength (3T / m) correspond to the original magnetic particle imaging image, which is used as the original image domain to simulate the result of magnetic particle imaging. At the same time, use the original three-dimensional phantom map as the target domain training set to represent the MNPs concentration distribution under ideal conditions. The resolution of the images in the original dataset is low along the focusing field (z-axis) direction (32×32×16, with few pixels in the focusing field direction), so the Bicubic interpolation method is used to adjust the size of the three-dimensional phantom to 32×32×32.

[0032] Step B4: Add Gaussian noise with different intensities to the magnetic particle imaging response signals obtained by layer-by-layer scanning, and reconstruct and superimpose them through the x-space algorithm to obtain three-dimensional magnetic particle imaging images with noise, which are used as three-dimensional noisy input data. Adjust the resolution of the three-dimensional magnetic particle concentration distribution map in Step B1 through interpolation processing and then add Gaussian noise with different intensities as three-dimensional noisy label data; In the present invention, Gaussian noise at different levels is added to the response signals to generate MNPs response signals with signal-to-noise ratios of 20 dB, 30 dB, and 40 dB, and finally magnetic particle imaging images are reconstructed.

[0033] Step B5: Use the three-dimensional noiseless input data and three-dimensional noiseless label data obtained in Step B3 and Step B4 as a three-dimensional noiseless dataset. Divide the three-dimensional noisy input data and three-dimensional noisy label data into multiple groups of three-dimensional noisy datasets according to different noises. Use the three-dimensional noiseless dataset and multiple groups of three-dimensional noisy datasets together as a three-dimensional resolution improvement student network dataset, and conduct separate training and verification on the three-dimensional isotropic resolution improvement network based on contrastive learning with a distillation structure.

[0034] Build a three-dimensional isotropic resolution improvement network based on contrastive learning with a distillation structure, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module, and the energy focusing module is used to activate image features in the spatial dimension; Among them, as Figure 2 shown, the structure of the two-dimensional isotropic teacher network model includes: A dense downsampling feature compression module, which is composed of n sequentially connected feature compression modules, and each feature compression module is composed of a feature extraction layer and a max pooling layer; A dense upsampling image reconstruction module, which is composed of n energy focusing modules and n image reconstruction modules connected alternately in sequence, and the image reconstruction module is composed of a feature extraction layer and an upsampling layer; Among them, the i-th feature compression module and the (n - i + 1)-th image reconstruction module establish a residual connection, where i = 1, 2, ……, n.

[0035] In this embodiment, n is preferably 4. That is to say, a residual connection is established between the first feature compression module and the fourth image reconstruction module, a residual connection is established between the second feature compression module and the third image reconstruction module, a residual connection is established between the third feature compression module and the second image reconstruction module, and a residual connection is established between the fourth feature compression module and the first image reconstruction module.

[0036] Among them, the feature extraction layer includes two feature extractors. The number of channels of the feature map is doubled through the first feature extractor (consisting of a convolutional kernel of size 3×3 for convolution operation), followed by Batch Normalization (BN) and ReLU activation function processing. Then, through the second feature extractor (consisting of a convolutional kernel of size 3×3 for convolution operation), the number of channels remains unchanged, and then BN and ReLU activation function processing are performed. The maximum pooling is set to a 2×2 pooling kernel with a stride of 2 for maximum pooling.

[0037] Each upsampling image reconstruction module includes three parts, namely energy focus calculation, feature extraction layer, and upsampling. Finally, a residual connection is made with the corresponding feature map in the downsampling feature compression process to obtain the image output of the upsampling module.

[0038] As Figure 3 shown, the energy focus module includes a feature input layer, a first convolutional layer, a first normalization and activation layer, a second convolutional layer, a second normalization and activation layer, and a feature output layer connected in sequence; After receiving the input data, the feature input layer inputs it to the first convolutional layer for the first convolution process, performs the first normalization and activation on the output of the first convolution process, and then performs the second convolution process through the second convolutional layer. The second normalization and activation are performed on the output of the second convolution process to obtain an attention map. After expanding the attention map to the number of channels of the input data in the channel dimension, it is multiplied element-wise with the input data to obtain an activated attention map. The activated attention map is added to the input data residually to obtain the output data, which is output through the feature output layer.

[0039] Among them, ReLU is used for activation in the first normalization and activation layer, and Sigmoid is used for activation in the second normalization and activation layer.

[0040] In this embodiment, specifically, it first performs a convolution operation using a convolution kernel of size 1×1 to keep the number of channels and the size of the feature map unchanged. Then, it performs a 3×3 convolution operation to halve the number of channels while keeping the size of the feature map unchanged. Through normalization and the ReLU activation function, it then performs a 3×3 convolution operation to make the number of channels of the feature map 1. It applies normalization and the Sigmoid activation again to limit the feature map within the range [0, 1] to obtain the attention map. It expands the attention map back to the number of channels of the original input in the channel dimension, multiplies it element-wise with the original input, and finally adds the result to the original feature residually to obtain the image features after attention activation.

[0041] Pre-train the two-dimensional isotropic teacher network model through the two-dimensional isotropic teacher network image dataset and save the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network dataset. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which is input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; In the present invention, the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, specifically: Obtain the model parameters of the set modules in the two-dimensional isotropic teacher network model as the dense downsampling feature compression module and the energy focusing module. Randomly initialize the model parameters of the dense upsampling reconstruction module in the three-dimensional resolution improvement student network model, perform gradient calculation during training, and iteratively update the parameters.

[0042] In the present invention, inputting into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model, specifically: Slice the three-dimensional training results into two-dimensional training input data, input the two-dimensional training input data into the pre-trained two-dimensional isotropic teacher network model to obtain two-dimensional training output data, calculate the contrast loss between the two-dimensional training input data and the two-dimensional training output data, and update and optimize the three-dimensional resolution improvement student network model.

[0043] Among them, for the two-dimensional isotropic teacher network, for the two-dimensional isotropic teacher network, Smooth loss is used to optimize it, and the specific loss function is: ; Among them, is the high-resolution magnetic particle image output by the two-dimensional isotropic teacher network, with the dimension of H×W; It is two-dimensional noise-free label data or two-dimensional noisy label data.

[0044] For the three-dimensional resolution improvement student network, for the three-dimensional resolution improvement student network, Smooth and InfoNCE losses are used for constraint, and the specific loss function L is: ; ; where The high-resolution three-dimensional magnetic particle image output by the three-dimensional resolution improvement student network, with dimensions of D × H × W ; is three-dimensional noise-free label data or three-dimensional noisy label data; is the cosine similarity calculation; is the temperature coefficient; is the total number of two-dimensional tomographic images of positive and negative samples. During training, select the tomographic images at the same position as the two-dimensional image reconstruction result in as positive examples, and other tomographic images as negative examples, and supervise the model training through the comparison between the selected positive and negative examples; Comprehensively evaluate Smooth and InfoNCE to obtain the loss function L : ; where is a hyperparameter specified during training. After experiments, = 0.1 is adopted.

[0045] In order to improve the reconstruction effect of the model along the focusing field direction, the present invention designs a structural distillation loss to supervise the three-dimensional network Fs. Transfer the spatial knowledge along the focusing field direction modeled in the two-dimensional teacher network Ft to the three-dimensional student network Fs to help model comprehensive three-dimensional spatial information and improve the overall image quality.

[0046] Specifically, for the pre-trained and fixed two-dimensional teacher network Ft, intercept the j-th tomographic image of the three-dimensional input image along the focusing field direction, input it into Ft for reconstruction to obtain the high-resolution two-dimensional tomographic image .

[0047] At the same time, for the output , select a tomographic image at the same position as the aforementioned two-dimensional image , and use the output of the two-dimensional teacher network Ft to perform structural distillation supervision on it, so as to effectively achieve the transfer of spatial knowledge along the direction of the focusing field.

[0048] Such as Figure 5 shown, train the network model, and its specific steps are as follows: Step D1, first train the two-dimensional isotropic teacher network. Input the training set in the two-dimensional isotropic teacher network image dataset into the two-dimensional isotropic teacher network model, and set the training batch size to 32. Subsequently, the network outputs the enhanced result, and adopt the loss function , calculate the loss function according to the output result of the network model and the label image. Then, update the network model parameters through backpropagation and save the network model. Repeat the above steps, iterate and train for 200 epochs, and finally save the optimal model as the two-dimensional isotropic teacher network model.

[0049] Step D2, load the parameters of the dense downsampling feature compression module and the energy focusing module in the two-dimensional isotropic teacher network into the three-dimensional resolution improvement student network model, and randomly initialize the remaining parameters.

[0050] Step D3, train the three-dimensional resolution improvement student network model. Input the training set in the three-dimensional resolution improvement student network dataset into the three-dimensional resolution improvement student network model. The network outputs the enhanced result, adopt the loss function L, and calculate the loss function according to the output result of the network model and the label image. Update the network model parameters through backpropagation and save the network model. Repeat the above steps, iterate and train for 200 epochs, and finally save the optimal model as the three-dimensional image anisotropic resolution improvement student network model.

[0051] In the above embodiments, although each step is described in the above order, those skilled in the art can understand that in order to achieve the effects of this embodiment, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are all within the protection scope of the present invention.

[0052] A three-dimensional magnetic particle imaging enhancement system based on distillation structure contrast learning according to the second embodiment of the present invention, based on a three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning according to the first embodiment, the system includes: A dataset construction module configured to construct a dataset, the dataset including a two-dimensional isotropic teacher network image dataset and a three-dimensional resolution improvement student network dataset; A model construction module configured to construct a three-dimensional isotropic resolution enhancement network based on distillation structure contrast learning, including a two-dimensional isotropic teacher network model and a three-dimensional resolution enhancement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution enhancement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module, and the energy focusing module is used to activate image features from the spatial dimension; A training module configured to train the two-dimensional isotropic teacher network model and the three-dimensional resolution enhancement student network model in stages and optimize model parameters based on a loss function; A resolution enhancement module configured to use the trained three-dimensional resolution enhancement student network model to perform resolution enhancement processing on the input real MPI image and output an isotropic high-resolution three-dimensional image.

[0053] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related descriptions of the above-described system can refer to the corresponding process in the foregoing method embodiments, and will not be elaborated herein.

[0054] It should be noted that the above-described three-dimensional magnetic particle imaging enhancement system based on distillation structure contrast learning is only illustrated by the above division of each functional module. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only used to distinguish each module or step, and are not regarded as an improper limitation of the present invention.

[0055] An electronic device according to a third embodiment of the present invention includes: At least one processor; and A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-described three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning.

[0056] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-described three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning.

[0057] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes and related descriptions of the above-described storage device and processing device can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here.

[0058] Those skilled in the art should be able to realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0059] Terms such as "first", "second", etc. are used to distinguish similar objects, rather than to describe or represent a specific order or sequence.

[0060] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, so that a process, method, article, or device / equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes the elements inherent in these processes, methods, articles, or devices / equipment.

[0061] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A three-dimensional magnetic particle imaging enhancement method based on distillation structure contrast learning, characterized in that The method includes: Constructing a dataset, which includes a two-dimensional isotropic teacher network image dataset and a three-dimensional resolution improvement student network dataset; Building a three-dimensional isotropic resolution improvement network based on contrastive learning of the distillation structure, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module, and the energy focusing module is used to activate image features from the spatial dimension; Pre-training the two-dimensional isotropic teacher network model with the two-dimensional isotropic teacher network image dataset and saving the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network dataset. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which are input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; Using the trained three-dimensional resolution improvement student network model to perform resolution improvement processing on the input real MPI image and output an isotropic high-resolution three-dimensional image.

2. The three-dimensional magnetic particle imaging enhancement method based on contrastive learning of distillation structures according to claim 1, wherein The specific steps for constructing the two-dimensional isotropic teacher network image dataset include: Step A1, using the three-dimensional vascular dataset in MedMNIST as the original three-dimensional magnetic particle concentration distribution map; Step A2, obtaining multiple magnetic particle imaging response signals by adjusting the driving field frequency and performing FFP layer scanning on the three-dimensional magnetic particle concentration distribution map; Step A3, performing x-space reconstruction on the multiple magnetic particle imaging response signals to obtain two-dimensional magnetic particle imaging images in the x-y plane. Stack the two-dimensional magnetic particle imaging images of multiple x-y planes in sequence to obtain a three-dimensional magnetic particle imaging image. Slice the three-dimensional magnetic particle imaging image in the x-z plane to obtain a low-resolution two-dimensional magnetic particle imaging image in the x-z plane as two-dimensional noise-free input data; use the high-resolution two-dimensional magnetic particle imaging image in the x-z plane at the corresponding position in the three-dimensional magnetic particle concentration distribution map in Step A1 as two-dimensional noise-free label data; Step A4, adding Gaussian noise with different noise levels to the low-resolution two-dimensional magnetic particle imaging image in the x-z plane and the high-resolution two-dimensional magnetic particle imaging image in the x-z plane respectively as two-dimensional noise-containing input data and two-dimensional noise-containing label data; Step A5, using the two-dimensional noise-free input data and two-dimensional noise-free label data obtained in Steps A3 and A4 as a two-dimensional noise-free dataset, dividing the two-dimensional noise-containing input data and two-dimensional noise-containing label data into multiple groups of two-dimensional noise-containing datasets according to different noises. The two-dimensional noise-free dataset and multiple groups of two-dimensional noise-containing datasets are jointly used as the two-dimensional isotropic teacher network image dataset, and the three-dimensional isotropic resolution improvement network based on contrastive learning of the distillation structure is trained and verified separately.

3. A three-dimensional magnetic particle imaging enhancement method based on contrastive learning of distillation structures according to claim 1, characterized in that, The specific steps for constructing a 3D resolution-enhanced student network dataset are as follows: Step B1: Use the MedMNIST 3D vascular dataset as the original magnetic particle concentration distribution map, and obtain a 3D magnetic particle concentration distribution map with the same field of view size through resampling; Step B2: Obtain multiple magnetic particle imaging response signals by adjusting the driving field frequency and performing FFP layer scanning on the 3D magnetic particle concentration distribution map; Step B3: Reconstruct the multiple magnetic particle imaging response signals using the x-space algorithm to generate 2D magnetic particle imaging images, and stack them to obtain a 3D magnetic particle imaging image under a low gradient field strength as the 3D noise-free input data. Adjust the resolution of the 3D magnetic particle concentration distribution map in Step B1 through interpolation processing as the 3D noise-free label data; Step B4: Add Gaussian noise with different intensities to the magnetic particle imaging response signals obtained by layer scanning, and reconstruct and stack them using the x-space algorithm to obtain a 3D magnetic particle imaging image with noise as the 3D noisy input data. Adjust the resolution of the 3D magnetic particle concentration distribution map in Step B1 through interpolation processing and then add Gaussian noise with different intensities as the 3D noisy label data; Step B5: Use the 3D noise-free input data and 3D noise-free label data obtained in Step B3 and Step B4 as the 3D noise-free dataset. Divide the 3D noisy input data and 3D noisy label data into multiple groups of 3D noisy datasets according to different noises. Combine the 3D noise-free dataset and multiple groups of 3D noisy datasets as the 3D resolution-enhanced student network dataset, and perform separate training and validation on the 3D isotropic resolution-enhanced network based on the contrastive learning of the distillation structure.

4. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of distillation structures according to claim 1, characterized in that The structure of the 2D isotropic teacher network model includes: A dense downsampling feature compression module, which consists of n sequentially connected feature compression modules. Each of the feature compression modules consists of a feature extraction layer and a max pooling layer; A dense upsampling image reconstruction module, which consists of n energy focusing modules and n image reconstruction modules connected alternately in sequence. The image reconstruction module consists of a feature extraction layer and an upsampling layer; Among them, the i-th feature compression module establishes a residual connection with the (n - i + 1)-th image reconstruction module, where i = 1, 2, ……, n.

5. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of a distillation structure according to claim 1, characterized in that The energy focusing module includes a feature input layer, a first convolutional layer, a first normalization and activation layer, a second convolutional layer, a second normalization and activation layer, and a feature output layer connected in sequence; After receiving the input data, the feature input layer inputs it to the first convolutional layer for the first convolutional processing, performs the first normalization and activation on the output of the first convolutional processing, and then performs the second convolutional processing through the second convolutional layer. Perform the second normalization and activation on the output of the second convolutional processing to obtain an attention map. After expanding the attention map to the number of channels of the input data in the channel dimension, multiply it element-wise with the input data to obtain an activated attention map. Add the activated attention map to the input data residually to obtain the output data, and output it through the feature output layer.

6. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of distillation structures according to claim 1, characterized in that The three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, specifically: Obtain the model parameters of the dense downsampling feature compression module and the energy focusing module in the two-dimensional isotropic teacher network model, and randomly initialize the model parameters of the dense upsampling reconstruction module in the three-dimensional resolution improvement student network model.

7. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of distillation structures according to claim 1, characterized in that Input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model, specifically: Slice the three-dimensional training results into two-dimensional training input data, input the two-dimensional training input data into the pre-trained two-dimensional isotropic teacher network model to obtain two-dimensional training output data, calculate the contrast loss between the two-dimensional training input data and the two-dimensional training output data, and update and optimize the three-dimensional resolution improvement student network model.

8. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of distillation structures according to claim 1, characterized in that For the two-dimensional isotropic teacher network, Smooth loss is used to optimize it, and the specific loss function is as follows: ； Among them, is the high-resolution magnetic particle image output by the two-dimensional isotropic teacher network, with dimensions of H×W; is two-dimensional label data without noise or two-dimensional label data with noise.

9. A three-dimensional magnetic particle imaging enhancement method based on contrast learning of distillation structures according to claim 8, characterized in that, For the three-dimensional resolution-enhanced student network, Smooth and InfoNCE losses are used for constraint, and the specific loss function L is as follows: ； ； Among them, The three-dimensional resolution enhances the high-resolution three-dimensional magnetic particle image output by the student network, with dimensions of D × H × W ; is three-dimensional noise-free label data or three-dimensional noisy label data; is cosine similarity calculation; is the temperature coefficient; is the total number of two-dimensional tomographic images of positive and negative samples. During training, select in the same two-dimensional image reconstruction result The tomographic images at the same position are used as positive examples, and other tomographic images are used as negative examples. The model training is supervised by the comparison between the selected positive and negative examples; Perform a comprehensive evaluation of Smooth and InfoNCE to obtain the loss function L : ； Among them, is a hyperparameter specified during training.

10. A three-dimensional magnetic particle imaging enhancement system based on distillation structure contrast learning, based on the three-dimensional magnetic particle imaging enhancement method according to any one of claims 1-9, characterized in that, The system includes: A dataset construction module configured to construct a dataset, the dataset including a two-dimensional isotropic teacher network image dataset and a three-dimensional resolution improvement student network dataset; A model building module configured to build a three-dimensional isotropic resolution improvement network based on contrastive learning of a distillation structure, including a two-dimensional isotropic teacher network model and a three-dimensional resolution improvement student network model; Both the two-dimensional isotropic teacher network model and the three-dimensional resolution improvement student network model include a dense downsampling feature compression module, a dense upsampling image reconstruction module, and an energy focusing module, and the energy focusing module is used to activate image features from the spatial dimension; A training module configured to pre-train the two-dimensional isotropic teacher network model through the two-dimensional isotropic teacher network image dataset and save the model parameters; the three-dimensional resolution improvement student network model obtains the model parameters of the pre-trained two-dimensional isotropic teacher network model, and is trained based on the three-dimensional resolution improvement student network dataset. The obtained three-dimensional training results are sliced and converted into two-dimensional training input data, which are input into the pre-trained two-dimensional isotropic teacher network model to generate a supervision benchmark for updating and optimizing the three-dimensional resolution improvement student network model; A resolution improvement module configured to use the trained three-dimensional resolution improvement student network model to perform resolution improvement processing on the input real MPI image and output an isotropic high-resolution three-dimensional image.

Images