Neural network magnetic particle imaging reconstruction method and system with forward model constraints

Abstract

The forward model constrained neural network magnetic particle imaging reconstruction method disclosed by the application obtains a system matrix through calibration, measures voltage data generated by a sample; Fourier transforms the collected data into a frequency domain, uses a signal-to-noise ratio threshold to screen frequency characteristics of the data; uses a Pytorch reconstruction network to realize mapping from one-dimensional voltage data to multi-dimensional magnetic particle concentration distribution; uses the system matrix as a forward model of magnetic particle imaging, generates voltage simulation data from the reconstructed magnetic particle concentration distribution, calculates a difference between the voltage simulation data and input voltage data as a loss function to update network parameters; adds a regularization term in the loss function, and adjusts training parameters and regularization parameters to obtain optimal reconstruction effect. The application uses a forward model constrained neural network to perform magnetic particle imaging reconstruction, and further improves the reconstruction effect by adding a total variation regularization term in the loss function of the network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image reconstruction, and particularly relates to a forward model constrained neural network magnetic particle imaging reconstruction method and a forward model constrained neural network magnetic particle imaging reconstruction system. BACKGROUND

[0002] Magnetic particle imaging (MPI) is a new quantitative functional imaging technology, which uses magnetic nanoparticles as tracers and finally images the concentration distribution of the magnetic nanoparticles. Compared with CT, PET and other imaging technologies, magnetic particle imaging does not contain any ionizing radiation, and at the same time has high spatial resolution, high temporal resolution and high sensitivity. It has broad application prospects and good application potential.

[0003] At present, there are two main reconstruction methods for magnetic particle imaging. One is the X-Space reconstruction method, which directly reconstructs the voltage signal in the time domain, has a faster reconstruction speed, but the frequent deconvolution process in the reconstruction process will amplify the noise and cause artifacts. The other method is the system matrix method. Compared with the X-Space method, the system matrix contains complex magnetic field characteristics and particle characteristics, and can consider the system defects, so it has a more accurate reconstruction result. However, since the system matrix method needs to solve the non-unique equation, there are still many artifacts in the reconstruction result.

[0004] Through the above analysis, the problems and defects of the prior art are:

[0005] (1) The current magnetic particle imaging reconstruction method, whether it is the system matrix method or the X-Space method, has obvious noise interference in the reconstruction result, and the reconstruction effect needs to be improved.

[0006] (2) Traditional neural network training requires a large amount of training data, but the current magnetic particle imaging equipment is expensive and has not been widely used. It is difficult to obtain measurement data, and there are only dozens of groups of publicly available magnetic particle imaging data on the Internet, which is not enough to train the neural network.

[0007] The difficulty of solving the above problems and defects is: the current magnetic particle imaging reconstruction quality needs to be further improved, so the problem of poor reconstruction effect needs to be solved; the training of the neural network requires a large amount of measured data, but the magnetic particle imaging technology has not been widely used, and there are few magnetic particle imaging devices. The magnetic particle imaging data is scarce, so the problem of updating the neural network parameters with a small amount or a single sample needs to be solved.

[0008] The significance of solving the above problems and defects is that the forward model constrained neural network is realized by Pytorch, the reconstruction of single data is realized, the suitable loss function is designed, the total variation regularization term is added, and the regularization parameter is adjusted to improve the reconstruction quality of magnetic particle imaging. SUMMARY

[0009] The purpose of the present application is to provide a forward model constrained neural network magnetic particle imaging reconstruction method, realize the reconstruction of single data, design a suitable loss function, add a total variation regularization term, and adjust the regularization parameter to improve the reconstruction quality of magnetic particle imaging.

[0010] The technical scheme adopted by the present application is a forward model constrained neural network magnetic particle imaging reconstruction method, and the specific operation steps are as follows:

[0011] (1) A standard concentration of magnetic particle sample is used to calibrate the magnetic particle imaging system to obtain the system matrix and the system's empty data as the background signal;

[0012] (2) The magnetic particle imaging system is used to scan the sample to be measured to obtain voltage data;

[0013] (3) Fourier transform is performed on the obtained system matrix and the voltage data obtained by scanning to obtain the corresponding frequency domain distribution;

[0014] (4) The signal-to-noise ratio of different frequency components is calculated;

[0015] (5) The signal-to-noise ratio threshold is set for frequency feature screening to reduce the influence of noise signals;

[0016] (6) Pytorch is used to realize the reconstruction network;

[0017] (7) The system matrix is used as the forward model of magnetic particle imaging;

[0018] (8) A loss function is proposed, and a total variation regularization term is added;

[0019] (9) The measured voltage data and system matrix data are input into the network, and the Adam optimization algorithm is used for parameter updating;

[0020] (10) The regularization parameter is adjusted to obtain the optimal reconstruction effect.

[0021] In the present application, the network framework is realized by Pytorch programming, which includes:

[0022] 1) The reconstruction network is implemented using Pytorch. The reconstruction network part is composed of an architecture similar to the DCGAN generator. Five layers of complex deconvolution networks are used in the network to realize the mapping of one-dimensional voltage frequency domain data to multi-dimensional magnetic particle concentration distribution. After the first four layers of deconvolution network, a complex batch normalization layer and a complex ReLU activation layer are added respectively.

[0023] The forward model proposed in the application is as follows:

[0024] 1) The magnetic particle imaging system is a linear system. The voltage signals generated by all magnetic particles in the imaging space can be regarded as the weighted sum of the voltage signals of unit concentration magnetic particles at each position. The system matrix reconstruction method utilizes this principle and is usually performed in the frequency domain. The received voltage signal u(t) is represented as u after Fourier transform. The relationship between u, the system matrix S and the concentration distribution c of the magnetic particles can be represented as:

[0025] u=Sc

[0026] The system matrix reconstruction method is a process of solving the magnetic particle concentration c under the condition that the voltage data u and the system matrix S are known. The forward propagation model is to obtain the generated voltage signal u under the premise that the concentration distribution c and the system matrix S are known.

[0027] 2) The encoder part of the second network uses a hollow convolution to increase the receptive field. In order to prevent the problem of gradient disappearance caused by too deep network, the decoder uses a residual network.

[0028] The loss function proposed in the application is as follows:

[0029] 1) The mean square error function is used to optimize from the pixel level:

[0030]

[0031] In the formula, u represents the received voltage signal input into the network, S represents the system matrix, The network output, i.e. the magnetic particle concentration distribution, is represented by M. The MSE represents the mean square error calculation formula, and the specific formula is as follows:

[0032]

[0033] In the formula, The network output is represented by M. The voltage signal obtained after the forward model calculation, i∈1,2..., B represents the frequency component corresponding to the voltage, and N is the number of frequency components.

[0034] 2) Total variation regularization is used for denoising angle optimization:

[0035]

[0036] In the formula, V is the calculation formula of total variation, which is as follows:

[0037]

[0038] In the formula, represents the magnetic particle concentration at the spatial position (j, k).

[0039] 3) The final loss function is:

[0040] L = L MSE + lambda L tv

[0041] In the formula, the total variation coefficient lambda is set to 10 -6 .

[0042] The network is optimized by the Adam optimization algorithm, which includes:

[0043] 1) Using a complex deconvolution network to reconstruct the spatial distribution of the magnetic particle concentration;

[0044] 2) The learning rate is set to 10 -4 .

[0045] The application also has the characteristics of:

[0046] A forward model of magnetic particle imaging is added to the neural network as a constraint, so that the parameters in the neural network are trained only using single measurement data; further, by adding a total variation regularization term in the loss function of the network, the influence of noise in the reconstruction result is weakened by adjusting the regularization parameter.

[0047] The application has the beneficial effects that the application obtains measurement data and system matrix data of a sample; Fourier transforms the obtained system matrix and the voltage data obtained by scanning to obtain corresponding frequency domain distributions; respectively calculates the signal-to-noise ratio of different frequency components through the system matrix and the empty measurement data, and eliminates the frequency components with lower signal-to-noise ratio by setting a threshold; builds a reconstruction network based on Pytorch 3.8; uses the system matrix as a forward model of magnetic particle imaging; proposes a loss function and adds a total variation regularization term; inputs the measurement obtained voltage data and the system matrix data into the network, and updates the parameters using the Adam optimization algorithm; adjusts the regularization parameter to obtain the optimal reconstruction effect. The application can be used for magnetic particle imaging reconstruction of the system matrix method, and realizes the magnetic particle imaging reconstruction of the forward model constrained neural network by using the forward model. The application can realize magnetic particle imaging reconstruction through the forward model constrained neural network, and improve the image reconstruction quality by adjusting the regularization parameter in the loss function. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The method is a forward model constrained neural network magnetic particle imaging reconstruction method provided by an embodiment of the application.

[0049] Figure 2 The system structure schematic diagram is a forward model constrained neural network magnetic particle imaging reconstruction system structure schematic diagram provided by an embodiment of the application.

[0050] Figure 2 In the figure, 1 is a magnetic particle imaging data acquisition module, 2 is a magnetic particle imaging data processing module, 3 is a reconstruction network module, and 4 is a forward transmission module.

[0051] Figure 3 The neural network structure schematic diagram is a neural network structure schematic diagram provided by an embodiment of the application.

[0052] Fig. 4(a) is an actual magnetic particle concentration distribution diagram, and Fig. 4(b) is a magnetic particle concentration distribution reconstruction result diagram.

[0053] Fig. 5(a) is an actual magnetic particle concentration distribution diagram, and Fig. 5(b) is a magnetic particle concentration distribution reconstruction result diagram. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the application clearer and more apparent, the application will be further described in detail below in combination with embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0055] In view of the problems in the prior art, the application provides a forward model constrained neural network magnetic particle imaging reconstruction method and system, which will be described in detail below in combination with the drawings.

[0056] As shown in Fig. 1, the forward model constrained neural network magnetic particle imaging reconstruction method provided by the application includes the following steps: Figure 1 S101: Obtain the measurement data of magnetic particle imaging and the system matrix data obtained by calibration, obtain frequency domain data by Fourier transform, and use a signal-to-noise ratio threshold to screen frequency characteristics;

[0057] S102: Implement the reconstruction network using Pytorch to realize the mapping from one-dimensional voltage data to multi-dimensional magnetic particle concentration distribution;

[0058] S103: Use the system matrix as the forward model of magnetic particle imaging, and generate simulation data of voltage using the reconstructed magnetic particle concentration distribution;

[0059]

[0060] ​S104: update the network parameters using the difference between the voltage simulation data and the input voltage data as a loss function and adding the total variation norm of the concentration.

[0061] The forward model constrained neural network magnetic particle imaging reconstruction method provided by the present application can also be implemented by other steps, Figure 1 The forward model constrained neural network magnetic particle imaging reconstruction method provided by the present application is only one specific embodiment.

[0062] As Figure 2 The forward model constrained neural network magnetic particle imaging reconstruction system provided by the present application comprises:

[0063] A magnetic particle imaging data acquisition module 1 is configured to acquire magnetic particle scanning data and calibrate the obtained system matrix data.

[0064] A magnetic particle imaging data processing module 2 is configured to perform Fourier transform on the acquired magnetic particle imaging data and perform frequency feature screening based on a signal-to-noise ratio threshold.

[0065] A reconstruction network module 3 is configured to implement a reconstruction network using Pytorch.

[0066] A forward transmission module 4 is configured to obtain frequency domain data corresponding to the voltage of the magnetic particle distribution.

[0067] The technical solutions of the present application will be further described below with reference to the accompanying drawings.

[0068] The forward model constrained neural network magnetic particle imaging reconstruction method provided by the present application specifically comprises the following steps:

[0069] (1) calibrate the magnetic particle imaging system using a standard concentration of magnetic particle samples to obtain a system matrix and system empty data as background signals;

[0070] (2) use the magnetic particle imaging system to scan the sample to be measured to obtain voltage data;

[0071] (3) perform Fourier transform on the obtained system matrix and the scanned voltage data to obtain the corresponding frequency domain distribution;

[0072] (4) calculate the signal-to-noise ratio of different frequency components;

[0073] (5) set a signal-to-noise ratio threshold to screen frequency features and reduce the influence of noise signals;

[0074] (6) implement a reconstruction network using Pytorch.

[0075] (7): Using system matrix as the forward model of magnetic particle imaging;

[0076] (8): Proposing loss function and adding total variation regularization term;

[0077] (9): Inputting measured voltage data and system matrix data into the network and updating parameters using Adam optimization algorithm;

[0078] (10) Adjusting the regularization parameter to obtain the optimal reconstruction effect.

[0079] In the preferred embodiment of the application (6) Pytorch programming realizes the network framework, including:

[0080] 1) Using Pytorch to realize the reconstruction network, and the reconstruction framework is as shown in Figure 3 The reconstruction network part is composed of an architecture similar to the DCGAN generator, and five layers of complex deconvolution networks are used in the network to realize the mapping of one-dimensional voltage frequency domain data to multi-dimensional magnetic particle concentration distribution. A complex batch normalization layer and a complex ReLU activation layer are added after the first four layers of deconvolution network.

[0081] In the preferred embodiment of the application (7), the forward model is as follows:

[0082] 1) The magnetic particle imaging system is a linear system, and the voltage signals generated by all magnetic particles in the imaging space can be regarded as the weighted sum of the voltage signals of unit concentration magnetic particles at each position. The system matrix reconstruction method utilizes this principle, and is usually performed in the frequency domain. The received voltage signal u(t) is expressed as u after Fourier transform, and the relationship between u, the system matrix S and the concentration distribution c of the magnetic particles can be expressed as:

[0083] u=Sc

[0084] The system matrix reconstruction method is a process of solving the magnetic particle concentration c under the condition that the voltage data u and the system matrix S are known. The forward propagation model is to obtain the generated voltage signal u under the premise that the concentration distribution c and the system matrix S are known.

[0085] 2) The encoder part of the second network uses a dilated convolution to increase the receptive field. In order to prevent the problem of gradient disappearance caused by too deep network, the decoder uses a residual network.

[0086] In the preferred embodiment of the application (8), the loss function is as follows:

[0087] 1) Using mean square error function to optimize from the pixel level:

[0088]

[0089] where u represents the received voltage signal of the input network, S represents the system matrix, represents the network output, i.e. the magnetic particle concentration distribution. MSE represents the mean square error calculation formula, and the specific formula is as follows:

[0090]

[0091] where, represents the network output The voltage signal obtained after forward model calculation, i e 1, 2..., N represents the frequency component corresponding to the voltage.

[0092] 2) Use total variation regularization to optimize the denoising angle:

[0093]

[0094] where V is the calculation formula of total variation, and the specific formula is as follows:

[0095]

[0096] where, represents the magnetic particle concentration at the spatial position (j, k).

[0097] 3) The final loss function is:

[0098] L = L MSE + λL tv

[0099] where the total variation coefficient λ is set to 5e-6.

[0100] In a preferred embodiment of the present application (9) the network is optimized by the Adam optimization algorithm, which includes:

[0101] 1) Use the complex deconvolution network to reconstruct the spatial distribution of the magnetic particle concentration;

[0102] 2) The learning rate is set to 1e -4 .

[0103] The technical effects of the present application will be described in detail below in combination with tests.

[0104] In a preferred embodiment of the present application (9) the reconstruction is performed on simulation data, and the algorithm performance is evaluated according to the following steps:

[0105] 1) The simulation data used in the present application is generated based on the measured system matrix;

[0106] 2) The present application calculates the mean square error and the structural similarity on the simulation data to evaluate the reconstruction effect, and the calculation formula of the mean square error is as follows:

[0107]

[0108] where M and N are the number of row and column sampling points of the reconstructed result and the actual magnetic particle space, and c(i,j) are the concentration values of the reconstructed result and the actual magnetic particle at position (i,j), respectively.

[0109] The present application uses structural similarity to evaluate the reconstruction effect of the proposed method, and the calculation formula of the structural similarity is as follows:

[0110]

[0111] where:

[0112]

[0113]

[0114]

[0115] where, and u c represent the reconstructed result of the magnetic particle concentration distribution and the mean of the actual distribution c of the magnetic particle concentration, and σ c are the standard deviations of the two concentration distributions, respectively; represents the covariance of the two concentration distributions. is the brightness comparison function of the two concentration distributions, is the contrast comparison function of the two concentration distributions, is the structure comparison function of the two concentration distributions. Where c1, c2, c3 are parameters introduced to adjust the instability when the denominator approaches zero; SSIM is an index that combines brightness, contrast, and structure. In practical applications, α, β, γ are usually set to 1, and c1=(K1L) 2 , c2=(K2L) 2 , where K1=0.01, K2=0.03, and L is the maximum value of the pixel. At this time, the above formula can be simplified as:

[0116]

[0117] Fig. 4(a)-(b) and Fig. 5(a)-(b) are reconstruction results of actual magnetic particle concentration distribution and simulation data of the present application, wherein Fig. 4(a) and Fig. 5(a) are actual magnetic particle concentration distribution, Fig. 4(b) and Fig. 5(b) are reconstruction results of the present application, it can be seen that the reconstruction results have higher consistency with the actual distribution of magnetic particle concentration, only a small amount of noise. Table 1 is the mean square error and structural similarity index of ART iterative algorithm and reconstruction results of the present application. It can be seen that the forward model constrained neural network magnetic particle imaging reconstruction method of the present application has obvious improvement on the reconstruction effect of magnetic particle imaging.

[0118] Table 1 Comparison of mean square error and structural similarity

[0119] Reconstruction method Mean square error Structural similarity ART iterative method 206.26 0.531 The invention 149.18 0.745

[0120] It should be noted that the embodiments of the present application can be realized by hardware, software or a combination of software and hardware. The hardware part can be realized by special logic; the software part can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or a specially designed hardware. Those skilled in the art can understand that the above devices and methods can be realized by computer executable instructions and / or included in processor control code, such as provided on a carrier medium, such as a disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The devices of the present application and their modules can be realized by hardware circuit, such as ultra-large scale integrated circuit or gate array, semiconductor, such as logic chip, transistor, etc., or programmable hardware device, such as field programmable gate array, programmable logic device, etc., or by software executed by various types of processors, or by a combination of the above hardware circuit and software, such as firmware.

[0121] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any modification, equivalent replacement and improvement within the technical range disclosed by the present application, which is within the spirit and principle of the present application, should be covered within the protection scope of the present application.

Claims

1. A neural network magnetic particle imaging reconstruction method with forward model constraints, characterized in that, The specific steps are as follows: Step 1: Acquire the measurement data of magnetic particle imaging and the system matrix data obtained from calibration, obtain the frequency domain data through Fourier transform, and use the signal-to-noise ratio threshold to filter the frequency features; Step 2: Use PyTorch to reconstruct the network, realizing the mapping from one-dimensional voltage data to multi-dimensional magnetic particle concentration distribution; Step 2, using PyTorch to reconstruct the network framework, includes: The reconstruction network is implemented using PyTorch. The entire reconstruction network consists of a DCGAN generator architecture. Five complex deconvolutional networks are used in the network to map one-dimensional voltage frequency domain data to multi-dimensional magnetic particle concentration distribution. Complex batch normalization layers and complex ReLU activation layers are added after the first four deconvolutional networks. Step 3: Use the system matrix as the forward model for magnetic particle imaging, and generate voltage simulation data using the reconstructed multidimensional magnetic particle concentration distribution; Step 4: Update the network parameters using the difference between the voltage simulation data and the input voltage data as the loss function and adding the total variation norm of the concentration; details are as follows: Step 4.1: Add a total variation regularization term to the loss function; Step 4.2: Input the measured voltage simulation data and system matrix data into the network, and use the Adam optimization algorithm to update the parameters; Step 4.3: Adjust the regularization parameters to obtain the optimal reconstruction effect.

2. The neural network magnetic particle imaging reconstruction method with forward model constraints according to claim 1, characterized in that, Step 1 is as follows: Step 1.1: Use a standard concentration of magnetic particle samples to calibrate the magnetic particle imaging system to obtain the system matrix and the system's spatial measurement data as background signals; Step 1.2: Use a magnetic particle imaging system to scan the sample to be measured to obtain voltage data; Step 1.3: Perform Fourier transform on the obtained system matrix and the voltage data obtained by scanning to obtain the corresponding frequency domain distribution; Step 1.4: Calculate the signal-to-noise ratio of different frequency components; Step 1.5: Set a signal-to-noise ratio threshold to filter frequency features and reduce the impact of noise signals.

3. The neural network magnetic particle imaging reconstruction method with forward model constraints according to claim 2, characterized in that, The forward model described in step 3 is as follows: Magnetic particle imaging systems are linear systems. The voltage signals generated by all magnetic particles within the imaging space can be considered as a weighted sum of the voltage signals of magnetic particles at a unit concentration at each location. System matrix reconstruction is based on this principle and is achieved in the frequency domain. The received voltage signals... After Fourier transform, it is represented as Then voltage data System matrix and magnetic particle concentration distribution The relationship between them can be represented as: The system matrix reconstruction method is based on known voltage data. and system matrix Solving for magnetic particle concentration under certain conditions The process; the forward model, on the other hand, is based on a known concentration distribution. and system matrix To obtain the generated voltage signal under the premise of .

4. The neural network magnetic particle imaging reconstruction method with forward model constraints according to claim 1, characterized in that, The loss function proposed in step 4.1 is as follows: 1) Optimize at the pixel level using the mean squared error function: In the formula, This indicates the received voltage signal at the input network. Represents the system matrix. This represents the network output, i.e., the magnetic particle concentration distribution. MSE The formula for calculating the mean squared error is as follows: In the formula, Represents network output The voltage signal obtained after forward model calculation Represents the frequency component of voltage. Number of frequency components; 2) Optimize the denoising angle using total variational regularization: In the formula, The formula for calculating the total variation is as follows: In the formula, Represents spatial location Concentration of magnetic particles at the location; 3) The final loss function is: Total variation coefficients Set to 10 -6 .

5. The neural network magnetic particle imaging reconstruction method with forward model constraints according to claim 1, characterized in that, Step 4.2 Optimizing the network using the Adam optimization algorithm includes: 1) Reconstruct the spatial distribution of magnetic particle concentration using a complex deconvolution network; 2) The learning rate is set to 10. -4 .

Images