Magnetic particle imaging reconstruction method based on attention mechanism
By using a neural network model based on a self-attention mechanism for end-to-end reconstruction, the problems of large data volume and information loss in magnetic particle imaging are solved, achieving efficient and high-precision magnetic particle imaging reconstruction, and improving image quality and generalization ability.
Patent Information
- Application Number
- CN202111198399.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-10-14
AI Technical Summary
Existing magnetic particle imaging technology suffers from problems such as large data volume leading to high reconstruction complexity, information loss due to preprocessing, inconsistent signal length under different conditions, and lack of a universal signal information extraction method, resulting in poor reconstructed image quality.
A neural network model based on a self-attention mechanism is adopted to reconstruct a two-dimensional image from a one-dimensional frequency domain signal through end-to-end reconstruction. The computational power of the neural network is used to extract effective information, reduce information loss and improve reconstruction accuracy.
It achieves efficient computation and high-precision magnetic particle imaging reconstruction, reduces the impact of noise, improves the resolution and accuracy of reconstructed images, and has generalization performance for different signal lengths.
Smart Images

Figure CN113850883B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of magnetic particle imaging, and in particular relates to a magnetic particle imaging reconstruction method from one-dimensional frequency domain signals to two-dimensional images based on a deep learning self-attention mechanism. Background Art
[0002] In clinical diagnosis and detection, how to accurately and objectively locate tumors and other lesions has always been a hot topic and challenging issue in the world. Traditional medical imaging technologies such as CT, MRI, SPECT and other methods have problems such as great harm, poor positioning and low precision. In recent years, a new imaging method based on tracers, magnetic particle imaging (MPI), has been proposed. This technology uses tomography technology to accurately locate the target by detecting the spatial concentration distribution of superparamagnetic iron oxide nanoparticles (SPIOs) that are harmless to the human body. It has the characteristics of three-dimensional imaging, high spatiotemporal resolution and high sensitivity. In addition, MPI does not display anatomical structures and has no background signal interference, so the signal intensity is directly proportional to the concentration of the tracer, which is a detection method with great potential for medical application.
[0003] At present, most MPI uses the original signals of magnetic particle imaging, including one-dimensional time domain signals and one-dimensional frequency domain signals, to reconstruct images through traditional algorithms, such as system matrix-based or X-space-based reproduction methods. However, magnetic particle imaging reconstruction methods based on deep learning are mainly based on multi-layer perceptron structures.
[0004] However, both traditional methods and deep learning methods have obvious shortcomings, which are mainly reflected in the following aspects: (1) Due to the large amount of original acquired data, existing methods, especially traditional methods, need to preprocess the data, which increases the complexity of reconstruction; (2) Existing preprocessing methods include downsampling or truncation, which are all artificial preprocessing methods, which will lead to the loss of sample information in the data and ultimately cause the loss of reconstructed image quality; (3) Due to the different lengths of acquired signals under different conditions, there is a lack of a universal processing method for extracting key information from different signals. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention proposes a magnetic particle imaging reconstruction method based on an attention mechanism. The method utilizes the powerful computing power of a neural network model to match the collected large amount of data, and learns to extract effective information from the signal by integrating a self-attention mechanism, thereby reducing the signal information loss caused by downsampling or truncation, and realizing end-to-end reconstruction from one-dimensional frequency domain signals to two-dimensional images.
[0006] The technical solution of the present invention is as follows:
[0007] A magnetic particle imaging reconstruction method based on an attention mechanism comprises the following steps:
[0008] S1 acquires a data set and obtains a simulated image and a corresponding one-dimensional frequency domain signal based on real magnetic particle imaging;
[0009] Construction of S2 neural network model, construction of neural network model integrating self-attention mechanism;
[0010] S3 training of the neural network model, dividing the data generated in step S1 into a training set and a validation set, training the neural network model, updating the neural network model parameters and saving the optimal neural network model;
[0011] S4 reconstructs the image and inputs any real magnetic particle imaging one-dimensional frequency domain signal into the optimal neural network model obtained by S3, and the reconstructed image result can be obtained end-to-end.
[0012] Preferably, in S1, the simulated image is a black and white binary image, white represents the signal, and black represents the background. The simulated image is used as the actual magnetic particle distribution input using a reconstruction algorithm, and the FFP scanning process is simulated by a simulation code to obtain a one-dimensional time domain signal and a one-dimensional frequency domain signal corresponding to the simulated image.
[0013] Preferably, in S1, the one-dimensional frequency domain signal is preprocessed to obtain an array with a uniform length of 1*10000, and each value is a complex number.
[0014] Preferably, in S1, the preprocessing method is: for a signal that is too long, a truncated signal segment at the rear is adopted, and for a signal that is too short, a zero-filling operation is adopted.
[0015] Preferably, in S2, the structure of the neural network model is: the signal generated by the magnetic particles passes through a fully connected layer, an activation function, a fully connected layer, an activation function in sequence, and then is input into an energy attention module, and then passes through a reshape layer to convert the one-dimensional signal into a two-dimensional image, and then passes through a convolution layer, an activation function, a convolution layer, and an activation function in sequence to obtain an output reconstructed image.
[0016] Preferably, the signal input into the energy attention module first passes through a one-dimensional convolution layer to obtain a multi-channel signal, and then a global average pooling layer and a global maximum pooling layer are used to aggregate information from different channels. The two pooled feature maps are passed through a multilayer perceptron consisting of three fully connected layers and three activation functions, and the two corresponding output feature maps obtained by the multilayer perceptron are connected and the corresponding energy attention result is calculated through the Sigmoid activation function.
[0017] Preferably, S3 also includes, before training the neural network model, processing the input data as follows: extracting the real part and imaginary part of each complex number in the original one-dimensional frequency domain signal of magnetic particle imaging 1*10000, obtaining two real number arrays with a length of 1*10000, the real part and imaginary part of the magnetic particle imaging are respectively symmetrical structures about the center, taking the first half of the real part and the imaginary part to obtain two 1*5000 arrays, and connecting the normalized real signal and imaginary signal to obtain an array with a length of 1*10000.
[0018] Preferably, in S3, the processed data is used as input, the simulated image is used as the label image, the neural network model is trained, the mean square error is used as the loss function, the loss function is calculated according to the output result and the label image, the neural network model parameters are updated, the loss function is calculated on the validation set, the network model parameters that minimize the loss function are updated and saved, and after the iterative training is completed, the optimal neural network model is saved.
[0019] Preferably, the loss function is as follows:
[0020]
[0021] Where I is the image output result, is the label image, H, W, and C represent the height, width, and number of channels of the feature map, respectively.
[0022] Preferably, in S4, any collected real magnetic particle imaging one-dimensional frequency domain signal is preprocessed and converted into an array of length 1*10000, and then input into the optimal neural network model obtained in S3 to obtain an accurate distribution image of the magnetic particles.
[0023] Compared with the prior art, the present invention has the following beneficial effects:
[0024] 1. The magnetic particle imaging reconstruction method based on the attention mechanism of the present invention adopts a deep learning method to complete the magnetic particle imaging reconstruction process, making full use of the efficient computing power of the neural network model and matching the computing requirements of a large amount of data in the reconstruction process;
[0025] 2. The convolutional neural network model constructed by the present invention includes an energy attention module. The neural network model integrated with the self-attention mechanism helps to extract effective information from the signal, reduce the impact of noise signals, reduce signal information loss caused by downsampling or truncation, and improve the resolution and accuracy of the reconstructed image;
[0026] 3. The magnetic particle imaging reconstruction method based on the attention mechanism of the present invention has certain generalization performance for one-dimensional frequency domain signals of magnetic particle imaging of different lengths. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The present invention may be better understood by reference to the accompanying drawings, wherein the components in the figures are not to be considered to scale, emphasis instead being placed upon illustrating the principles of the invention.
[0028] Attached Figure 1 It is a flow chart of the magnetic particle imaging reconstruction method based on the attention mechanism of the present invention;
[0029] Attached Figure 2 This is a schematic diagram of the real part of the one-dimensional frequency domain signal of magnetic particle imaging;
[0030] Attached Figure 3 This is a schematic diagram of the imaginary part of the one-dimensional frequency domain signal of magnetic particle imaging;
[0031] Attached Figure 4 Schematic diagram of the neural network model structure constructed by the present invention. DETAILED DESCRIPTION
[0032] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:
[0033] The technical solution of the magnetic particle imaging reconstruction method based on the attention mechanism proposed in the present invention is as follows: Figure 1 As shown in the figure, it mainly includes the acquisition of data sets, the construction of neural network models, the training of models, and the testing of one-dimensional frequency domain signals of real magnetic particles. The specific implementation plan is as follows:
[0034] Acquisition of S1 data set: Generate simulated images to simulate the results of magnetic particle imaging in actual situations. The simulated images are black and white binary images, where white represents signals and black represents backgrounds. Some simple shapes such as ellipses and rectangles are used to simulate the shapes of samples collected in actual situations. Using the existing reconstruction algorithm, the generated simulated images are used as the input of the actual magnetic particle distribution, and the FFP scanning process is simulated through the simulation code, and finally the one-dimensional time domain signal and one-dimensional frequency domain signal corresponding to the simulated image are obtained. After simple preprocessing, the one-dimensional frequency domain signal is obtained to obtain an array with a uniform length of 1*10000. The specific method is: for signals that are too long, the signal-free segment is truncated at the back, and for signals that are too short, the zero-filling operation is performed, and each value of the signal is a complex number. A total of 10,000 sets of simulated images and corresponding one-dimensional frequency domain signals are obtained as the training set and validation set data of the neural network.
[0035] Construction of S2 neural network model: Build a neural network model that integrates the self-attention mechanism. The structure of the model is: the signal generated by the magnetic particles first passes through the fully connected layer, activation function, fully connected layer, and activation function in sequence. The number of neurons in the two fully connected layers is 20,000 and 10,000 respectively, and both activation functions are ReLU activation functions; the output from the second activation function is a 1*10,000 vector, which is the input of the subsequent energy attention module. The energy attention module suppresses noise signals by learning to focus on the energy concentrated part (effective information part) of the signal. Specifically, the module first obtains a multi-channel signal through a one-dimensional convolution layer. The convolution layer contains 32 1*5 convolution kernels, and after processing, a 1*10000*32 multi-channel feature vector is obtained. Then, the global average pooling layer and the global maximum pooling layer are used to aggregate the information of different channels to obtain two 1*10000 feature vectors. These two feature vectors are passed through a multi-layer perceptron. The specific structure of the multi-layer perceptron is a fully connected layer, an activation function, a fully connected layer, an activation function, a fully connected layer, and an activation function. The number of neurons in the three fully connected layers is 10000, 1250, and 10000 respectively. The activation function is ReLU. The output of the multi-layer perceptron, that is, the two 1*10000 feature maps are added and the output result of the corresponding energy attention module is calculated through a Sigmoid activation function. The output of the energy attention module is multiplied element-wise with the input of the energy attention module to obtain a feature vector of length 1*10000 processed by the attention module, and then a reshape layer is used to convert the one-dimensional feature vector of length 1*10000 into a two-dimensional feature map of size 100*100, and then a convolution layer is passed, which contains 16 convolution kernels of size 3*3, and then a ReLU activation function is passed. Then a convolution layer is input, which contains a convolution kernel of size 1*1, and finally a ReLU activation function is used to obtain the output two-dimensional reconstructed image.
[0036] Training of the S3 neural network model: The data generated in S1 is divided into a training set and a validation set at a ratio of 9:1, with the one-dimensional frequency domain signal as the input and the simulated image as the label image. Since the one-dimensional frequency domain signal is a complex number, a simple preprocessing is performed on it. The specific method is: extract the real part and imaginary part of each complex number in the original one-dimensional frequency domain signal of magnetic particle imaging 1*10000 to obtain two real number arrays with a length of 1*10000. Since the real and imaginary parts of magnetic particle imaging are respectively symmetrical structures about the center, in order to reduce data redundancy and for more efficient training, only the first half of the real and imaginary parts are taken to obtain two 1*5000 arrays. The normalized real and imaginary signals are connected to obtain an array with a length of 1*10000, which is the final input vector;
[0037] The mean square error is used as the loss function:
[0038]
[0039] Where I is the image output result, is the label image, H, W and C represent the height, width and number of channels of the feature map respectively.
[0040] The loss function is calculated based on the output results of the neural network model and the label image, the neural network model parameters are updated and the network model is saved. The iterative training reduces the loss function. The iterative training is repeated for 300 epochs, and the optimal model is finally saved.
[0041] S4 Test of the one-dimensional frequency domain signal of real magnetic particles: First, a phantom is manufactured, and then the image of the phantom is collected using an existing commercial magnetic particle imaging instrument. The original one-dimensional frequency domain signal collected from the instrument is used as the test input signal, and the real image is used as the true image. The input signal is simply preprocessed to convert it into an array of length 1*10000, and the final result is obtained by testing the saved optimal model. After comparing with the true image, the reconstruction result is evaluated. After the model is verified, the collected real magnetic particle imaging one-dimensional frequency domain signal is simply preprocessed, and the accurate distribution image of the magnetic particles can be reconstructed through the saved optimal model.
[0042] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0043] In the present invention, unless otherwise clearly stipulated and limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features not being in direct contact but being in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes the first feature being directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes the first feature being directly below and obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.
[0044] In the present invention, the terms "first", "second", "third", and "fourth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise clearly defined.
[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A magnetic particle imaging reconstruction method based on attention mechanism, It is characterized in that The following steps are involved: S1 acquires a data set and obtains a simulated image and a corresponding one-dimensional frequency domain signal based on real magnetic particle imaging; Construction of S2 neural network model, construction of neural network model integrating self-attention mechanism; S3 training of the neural network model, dividing the data generated in step S1 into a training set and a validation set, training the neural network model, updating the neural network model parameters and saving the optimal neural network model; S4 reconstructs the image, inputs any real magnetic particle imaging one-dimensional frequency domain signal into the optimal neural network model obtained by S3, and can obtain the reconstructed image result end-to-end; In S1, the simulated image is a black-and-white binary image, where white represents a signal and black represents a background. The simulated image is used as an input of the actual magnetic particle distribution by a reconstruction algorithm, and the FFP scanning process is simulated by a simulation code to obtain a one-dimensional time domain signal and a one-dimensional frequency domain signal corresponding to the simulated image; In S2, the structure of the neural network model is: the signal generated by the magnetic particle passes through a fully connected layer, an activation function, a fully connected layer, an activation function, and then is input into an energy attention module, and then passes through a reshape layer to convert the one-dimensional signal into a two-dimensional image, and then passes through a convolution layer, an activation function, a convolution layer, and an activation function to obtain an output reconstructed image; The signal input into the energy attention module first passes through a one-dimensional convolution layer to obtain a multi-channel signal, and then uses a global average pooling layer and a global maximum pooling layer to aggregate information from different channels. The two pooled feature maps are passed through a multi-layer perceptron consisting of three fully connected layers and three activation functions. The two corresponding output feature maps obtained by the multi-layer perceptron are connected and the corresponding energy attention results are calculated through the Sigmoid activation function.
2. The magnetic particle imaging reconstruction method according to claim 1, It is characterized in that In S1, the one-dimensional frequency domain signal is preprocessed to obtain an array with a uniform length of 1*10000, and each value is a complex number.
3. The magnetic particle imaging reconstruction method according to claim 2, It is characterized in that In S1, the preprocessing method is: for a signal that is too long, a truncated signal segment at the rear is adopted, and for a signal that is too short, a zero padding operation is adopted.
4. The magnetic particle imaging reconstruction method according to claim 1, It is characterized in that The S3 also includes that before training the neural network model, the input data is processed as follows: the real part and imaginary part of each complex number in the original one-dimensional frequency domain signal of magnetic particle imaging 1*10000 are extracted to obtain two real number arrays with a length of 1*10000, the real part and imaginary part of the magnetic particle imaging are respectively symmetrical structures about the center, the first half of the real part and the imaginary part are taken to obtain two 1*5000 arrays, and the normalized real signal and imaginary signal are connected to obtain an array with a length of 1*10000.
5. The magnetic particle imaging reconstruction method according to claim 4, It is characterized in that In S3, the processed data is used as input, the simulated image is used as the label image, the neural network model is trained, the mean square error is used as the loss function, the loss function is calculated according to the output result and the label image, the neural network model parameters are updated, the loss function is calculated on the validation set, the network model parameters that minimize the loss function are updated and saved, and after the iterative training is completed, the optimal neural network model is saved.
6. The magnetic particle imaging reconstruction method according to claim 5, It is characterized in that The loss function is as follows: Where I is the image output result, is the label image, H, W and C represent the height, width and number of channels of the feature map respectively.
7. The magnetic particle imaging reconstruction method according to claim 1, It is characterized in that In S4, any collected real magnetic particle imaging one-dimensional frequency domain signal is preprocessed and converted into an array of length 1*10000, and then input into the optimal neural network model obtained in S3 to obtain an accurate distribution image of the magnetic particles.
