A Magnetic Resonance Reconstruction Method and Device Integrating Signal Model and Deep Learning

Abstract

The present invention relates to a magnetic resonance reconstruction method and device integrating a signal model and deep learning. The method includes: acquiring magnetic resonance sparse sampling data; constructing a probabilistic subspace model; preliminarily reconstructing to obtain a subspace reconstruction image; constructing a first deep learning network, using the subspace reconstruction image as the network input, and using the real image as the output of the model to learn the mapping relationship from the subspace reconstruction image to the real image, so as to obtain an improved global reconstruction image; acquiring residual feature sparse sampling data, and preliminarily reconstructing based on a sparse model to obtain a sparse residual reconstruction image; constructing a second deep learning network, using the sparse residual reconstruction image as the network input, and integrating the stable global reconstruction image as a conditional constraint into the network to obtain an improved residual reconstruction image; and fusing the global reconstruction image and the residual reconstruction image to obtain a magnetic resonance reconstruction image. Compared with the prior art, the present invention has the advantages of high reconstruction accuracy and low requirements for training data, etc.

Description

Technical Field

[0001] The present invention relates to the field of magnetic resonance image reconstruction, and in particular to a magnetic resonance reconstruction method and device that combines a signal model and deep learning. Background Art

[0002] Magnetic resonance image reconstruction is one of the cores of fast magnetic resonance imaging technology. Its mathematical essence is to solve an ill-posed inverse problem. The reconstructed images directly obtained from undersampled data using the classical Fourier transform will have problems such as artifacts or low signal-to-noise ratio. To solve this problem, introducing prior information to compensate for the information loss caused by insufficient sampled data is the key. In recent years, deep learning has shown great potential in fast magnetic resonance imaging. This is mainly due to its powerful ability to learn complex priors from a large amount of existing high-resolution magnetic resonance data.

[0003] Currently, the mainstream deep learning-based reconstruction algorithms are mainly divided into three categories, namely: 1) learning the nonlinear transformation from undersampled data to high-quality images; 2) learning the optimal regularization term; 3) learning the spatial-intensity distribution of training data. The learning mechanisms and network structures of deep learning in these three categories of algorithms all come from the field of computer vision. However, there are the following significant differences between medical images and computer vision tasks: 1) the amount of data is small, resulting in the deep learning model being prone to overfitting; 2) different institutions use different acquisition machines and acquisition sequences, so it is difficult for deep learning algorithms trained on data obtained from a certain institution's sequence to be applicable to other institutions and sequences; 3) existing training data often uses normal images, and disease images are relatively few, resulting in a significant decline in the performance of the trained algorithms on disease data and making it difficult to effectively reconstruct disease features. Specifically, the input of traditional machine learning algorithms is usually the classical Fourier transform. Under undersampling conditions, the reconstruction effect of the Fourier transform is poor, so the mapping relationship that deep learning needs to learn is more complex and requires a larger amount of training data; in addition, when the sampling parameters change, the reconstruction of the Fourier transform algorithm also changes greatly, which leads to a large change in the results of deep learning and makes it difficult to be applicable to different machines and sampling parameters. Summary of the Invention

[0004] The purpose of the present invention is to provide a magnetic resonance reconstruction method and device that combines a signal model and deep learning, which reduces the amount of training data required, reduces the influence of acquisition parameters, more effectively reconstructs disease features, and has high generalization.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A magnetic resonance reconstruction method that combines a signal model and deep learning includes the following steps:

[0007] S1: Obtain magnetic resonance sparse sampling data;

[0008] S2: Construct a high-quality training data set, and construct a probability subspace model based on the high-quality training data set;

[0009] S3: Based on the sparse sampling data and the probability subspace model, perform a preliminary reconstruction of the magnetic resonance image to obtain a subspace reconstruction image;

[0010] S4: Construct a first deep learning network assisted by a subspace model, use the subspace reconstruction image as the input of the network, use the real image as the output of the model, and use the first deep network to learn the mapping relationship from the subspace reconstruction image to the real image to obtain an improved global reconstruction image;

[0011] S5: Obtain residual feature sparse sampling data, and based on the sparse model, perform a preliminary reconstruction of the residual feature image to obtain a sparse residual reconstruction image;

[0012] S6: Construct a second deep learning network assisted by a sparse model, use the sparse residual reconstruction image as the input of the second deep learning network, and incorporate the stable global reconstruction image as a conditional constraint into the network to obtain an improved residual reconstruction image;

[0013] S7: Fuse the global reconstruction image and the residual reconstruction image to obtain the final magnetic resonance reconstruction image.

[0014] In S2, given a high-quality training data set {ρ1, ρ2, …, ρ M}, construct a Casorati matrix and perform PCA analysis on it to obtain the basis {φ r (x)} of the linear subspace and the corresponding coefficients {a r}, then the image to be reconstructed is expressed as:

[0015]

[0016] where R is the number of bases of the subspace.

[0017] The method for performing a preliminary reconstruction of the magnetic resonance image based on the sparse sampling data and the probability subspace model is:

[0018]

[0019] where d is the sparse sampling data, is the variance of the noise, Ω is the sampling pattern, is the Fourier transform, and for any a r , its probability density function Pr(a r ) is estimated by the mixture Gaussian algorithm.

[0020] The method for obtaining the sparse sampling data of residual features is to remove the globally reconstructed image obtained by reconstruction through the first deep learning network from the sparse sampling data, so as to obtain the sparse sampling data of residual features, that is:

[0021]

[0022] where d is the sparse sampling data, Ω is the sampling pattern, is the Fourier transform, is the globally reconstructed image obtained by reconstruction through the first deep learning network, is the sparse sampling data of residual features.

[0023] Specifically, the preliminary reconstruction of the residual feature image based on the sparse model is:

[0024]

[0025] where W is the sparse transform, is the regularization parameter, is the residual feature image to be obtained.

[0026] A magnetic resonance reconstruction device that fuses a signal model and deep learning includes:

[0027] A first data acquisition module, configured to acquire magnetic resonance sparse sampling data and a high-quality training data set;

[0028] A global image preliminary reconstruction module, configured to construct a probability subspace model based on the high-quality training data set, and implement preliminary reconstruction of the magnetic resonance image based on the sparse sampling data and the probability subspace model to obtain a subspace reconstructed image;

[0029] An improved global image reconstruction module, configured to construct a first deep learning network assisted by a subspace model, use the subspace reconstructed image as the input of the network, use the real image as the output of the model, and use the first deep network to learn the mapping relationship from the subspace reconstructed image to the real image to obtain an improved globally reconstructed image;

[0030] A second data acquisition module, configured to acquire sparse sampling data of residual features;

[0031] A residual image preliminary reconstruction module, configured to preliminarily reconstruct a residual feature image based on a sparse model to obtain a sparse residual reconstructed image;

[0032] An improved sparse image reconstruction module, configured to construct a second deep learning network assisted by a sparse model, use the sparse residual reconstructed image as the input of the second deep learning network, and incorporate a stable globally reconstructed image as a conditional constraint into the network to obtain an improved residual reconstructed image;

[0033] A fusion module, which is used to fuse the global reconstructed image and the residual reconstructed image to obtain the final magnetic resonance reconstructed image.

[0034] In the global image preliminary reconstruction module, given a high-quality training data set {ρ1, ρ2, …, ρ M}, a Casorati matrix is constructed and PCA analysis is performed on it to obtain the basis {φ r (x)} of the linear subspace and the corresponding coefficients {a r}. Then the image to be reconstructed is represented as:

[0035]

[0036] where R is the number of bases of the subspace.

[0037] The method for preliminarily reconstructing magnetic resonance images based on sparse sampling data and a probabilistic subspace model is as follows:

[0038]

[0039] where d is the sparse sampling data, is the variance of the noise, Ω is the sampling pattern, is the Fourier transform. For any a r , its probability density function Pr(a r ) is estimated by the mixture Gaussian algorithm.

[0040] The second data acquisition module removes the globally reconstructed image obtained by reconstructing through the first deep learning network from the sparse sampling data to obtain the sparse sampling data of the residual features, that is:

[0041]

[0042] where d is the sparse sampling data, Ω is the sampling pattern, is the Fourier transform, is the globally reconstructed image obtained by reconstructing through the first deep learning network, is the sparse sampling data of the residual features.

[0043] Specifically, the preliminary reconstruction of the residual feature image based on the sparse model is as follows:

[0044]

[0045] where W is the sparse transform, is the regularization parameter, is the residual feature image to be obtained.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] (1) The reconstruction effect of the signal model adopted by the present invention is much better than that of Fourier transform. Therefore, the mapping relationship that needs to be learned by deep learning is simpler, reducing the requirement for the amount of data, enabling the method of the present invention to be not limited by small amounts of original data.

[0048] (2) The results obtained by reconstructing the signal model in the present invention are more robust to different parameters and machines, which is beneficial to improving the compatibility of deep learning with different sampling parameters and machines.

[0049] (3) The present invention proposes an effective image splitting method (i.e., splitting global features and local residual features), and respectively fuses appropriate signal models and deep learning for reconstruction, better restoring image details, especially local features representing diseases such as tumors, with accurate feature capture and high image reconstruction accuracy. Specifically, for global features, a linear subspace model that can capture global features and a deep learning algorithm are fused; for local residual features, based on their sparsity, a sparse model and a deep learning algorithm are fused. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of the method of the present invention;

[0051] Figure 2 is a schematic diagram of the data processing process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0053] This embodiment provides a magnetic resonance reconstruction method that fuses a signal model and deep learning. Its core concept is to organically integrate the respective advantages of the signal model and deep learning (the signal model is more stable and deep learning has stronger expression ability), while avoiding their respective disadvantages. Specifically, the image to be reconstructed is divided into two parts. The first part represents the global feature ρ G (x), and the second part represents the local residual feature ρ L (x) (such as tumor features). For global features, a probabilistic subspace model is first used to capture the global features located in the linear space, and then a deep learning network assisted by the subspace model is used to learn the non-linear mapping relationship from the image reconstructed by the subspace model to the real image to improve the reconstruction effect of the global features; for local residual features, first, using their sparsity, a sparse model is used to reconstruct the residual features, and then a deep learning network assisted by the sparse model is used to learn from the sparse model reconstruction to the real residual features to improve the reconstruction effect of the residual features.

[0054] The method flow is asFigure 1 As shown in the figure, it includes the following steps:

[0055] S1: Obtain the magnetic resonance sparse sampling data d.

[0056] S2: Construct a high-quality training data set {ρ1, ρ2, …, ρ M}, and construct a probability subspace model based on the high-quality training data set.

[0057] Given the high-quality training data set {ρ1, ρ2, …, ρ M}, construct a Casorati matrix and perform PCA analysis on it to obtain the basis {φ r (x)} of the linear subspace and the corresponding coefficients {a r}, then the image to be reconstructed is expressed as:

[0058]

[0059] where R is the number of subspace bases.

[0060] S3: Based on the sparse sampling data and the probability subspace model, perform a preliminary reconstruction of the magnetic resonance image to obtain a subspace reconstruction image:

[0061]

[0062] where d is the sparse sampling data, is the variance of the noise, Ω is the sampling pattern, is the Fourier transform, for any a r , its probability density function Pr(a r ) is estimated by the mixture Gaussian algorithm.

[0063] S4: Construct a first deep learning network assisted by the subspace model, use the subspace reconstruction image as the input of the network, use the real image as the output of the model, and use the first deep network to learn the mapping relationship from the subspace reconstruction image to the real image to obtain an improved global reconstruction image.

[0064] Based on the fact that the subspace model is essentially a linear model, since the features outside the linear space are difficult to be reconstructed, the reconstructed image of the subspace model still has a large error, but its reconstruction result is robust. The deep learning network has a strong expression ability due to its universal approximation ability, but as mentioned above, it has problems such as non-robustness. Therefore, it is proposed here to fuse the subspace model and the deep learning network to make complementary advantages. This embodiment gives a fusion method, that is, using the initial reconstruction of the subspace model as the input of the deep learning network, and using the deep network to learn the mapping relationship from the subspace reconstruction image to the real image, so as to obtain an improved global reconstruction image ρ G(x), that is:

[0065]

[0066] where f GNN is the first deep learning network assisted by the subspace model obtained through training.

[0067] S5: Obtain the sparse sampling data of the residual features, and based on the sparse model, preliminarily reconstruct the residual feature image to obtain a sparse residual reconstruction image.

[0068] In this embodiment, the method for obtaining the sparse sampling data of the residual features is to remove the globally reconstructed image obtained through the reconstruction of the first deep learning network from the sparse sampling data, so as to obtain the sparse sampling data of the residual features, that is:

[0069]

[0070] where d is the sparse sampling data, Ω is the sampling pattern, is the Fourier transform, is the globally reconstructed image obtained through the reconstruction of the first deep learning network, is the sparse sampling data of the residual features.

[0071] Specifically, the preliminary reconstruction of the residual feature image based on the sparse model is:

[0072]

[0073] where W is the sparse transform, is the regularization parameter, is the residual feature image to be solved.

[0074] S6: Construct a second deep learning network assisted by the sparse model, use the sparse residual reconstruction image as the input of the second deep learning network, and incorporate the stable globally reconstructed image as a conditional constraint into the network to obtain an improved residual reconstruction image.

[0075] Similarly, the expressive ability of the sparse model is also limited, but it can use sparsity to give a stable initial reconstruction of the residual features. Using the same concept, we organically integrate the sparse model with deep learning. Here, a fusion method is also given, that is, using the result of the sparse model as the input of the deep network, and at the same time incorporating the stable globally reconstructed features that have been obtained as a conditional constraint into the network to further improve the accuracy and robustness of the residual feature reconstruction result.

[0076]

[0077] where f LNN is the second deep learning network assisted by the sparse model obtained through training.

[0078] S7: Integrate the global reconstructed image and the residual reconstructed image to obtain the final magnetic resonance reconstructed image:

[0079]

[0080] The overall data processing process of the method is as Figure 2 shown. The reconstructed magnetic resonance image reconstructed by the method described in the present invention has high reconstruction accuracy, and the model does not depend on a large amount of training data, has strong model robustness, and improves the compatibility of the method with different sampling parameters and machines.

[0081] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art shall fall within the protection scope determined by the claims.

Claims

1. A magnetic resonance imaging reconstruction method integrating signal model and deep learning, characterized in that: The following steps are involved: S1: Acquire sparsely sampled magnetic resonance data; S2: Build a high-quality training dataset and construct a probabilistic subspace model based on the high-quality training dataset; S3: Preliminary reconstruction of the magnetic resonance image is performed based on sparsely sampled data and the probabilistic subspace model to obtain a subspace reconstructed image; S4: Construct a first deep learning network assisted by the subspace model, take the subspace reconstructed image as the network input, take the real image as the model output, use the first deep network to learn the mapping relationship between the subspace reconstructed image and the real image, and obtain an improved global reconstructed image; S5: Obtain residual feature sparse sampling data, and preliminarily reconstruct the residual feature image based on the sparse model to obtain a sparse residual reconstructed image; S6: Build a second deep learning network assisted by a sparse model, use the sparse residual reconstructed image as the input of the second deep learning network, incorporate the stable global reconstructed image into the network as a conditional constraint, and obtain an improved residual reconstructed image; S7: The global reconstructed image and the residual reconstructed image are fused to obtain the final magnetic resonance reconstructed image.

2. The magnetic resonance reconstruction method integrating signal model and deep learning according to claim 1, characterized in that: In S2, given a high-quality training dataset {ρ1,ρ2,…,ρ M }, construct the Casorati matrix and perform PCA analysis on it to obtain the basis of the linear subspace {φ r (x)} and the corresponding coefficients {a r }, the image to be reconstructed is expressed as: Where R is the number of subspace bases.

3. The magnetic resonance reconstruction method integrating signal model and deep learning according to claim 2, characterized in that: The preliminary reconstruction method of magnetic resonance images based on sparse sampling data and probabilistic subspace model is: Among them, d is sparse sampling data, is the variance of the noise, Ω is the sampling mode, is the Fourier transform, for any a r , its probability density function Pr(a r ) is estimated by the Gaussian mixture algorithm.

4. The magnetic resonance reconstruction method integrating signal model and deep learning according to claim 1, characterized in that: The method for obtaining the residual feature sparse sampling data is to remove the global reconstructed image obtained by the first deep learning network reconstruction from the sparse sampling data to obtain the sparse sampling data of the residual features, that is: Among them, d is the sparse sampling data, Ω is the sampling mode, is the Fourier transform, is the global reconstructed image obtained by the first deep learning network reconstruction, Sparsely sampled data for residual features.

5. The magnetic resonance reconstruction method integrating signal model and deep learning according to claim 4, characterized in that: The preliminary reconstruction of the residual feature image based on the sparse model is specifically as follows: Where W is the sparse transformation, is the regularization parameter, is the residual feature image to be determined.

6. A magnetic resonance reconstruction device integrating signal model and deep learning, characterized in that: include: A first data acquisition module is used to acquire magnetic resonance sparse sampling data and a high-quality training data set; A global image preliminary reconstruction module is used to build a probabilistic subspace model based on a high-quality training data set, and to perform preliminary reconstruction of the magnetic resonance image based on the sparsely sampled data and the probabilistic subspace model to obtain a subspace reconstructed image; An improved global image reconstruction module is used to construct a first deep learning network assisted by a subspace model. The subspace reconstructed image is used as the input of the network, and the real image is used as the output of the model. The mapping relationship between the subspace reconstructed image and the real image is learned by the first deep network to obtain an improved global reconstructed image. A second data acquisition module is used to acquire residual feature sparse sampling data; A residual image preliminary reconstruction module is used to preliminarily reconstruct the residual feature image based on the sparse model to obtain a sparse residual reconstructed image; An improved sparse image reconstruction module is used to construct a second deep learning network assisted by a sparse model. The sparse residual reconstructed image is used as the input of the second deep learning network, and the stable global reconstructed image is incorporated into the network as a conditional constraint to obtain an improved residual reconstructed image. The fusion module is used to fuse the global reconstructed image and the residual reconstructed image to obtain the final magnetic resonance reconstructed image.

7. The magnetic resonance reconstruction device integrating signal model and deep learning according to claim 6, characterized in that: In the global image preliminary reconstruction module, given a high-quality training dataset {ρ1, ρ2, ..., ρ M }, construct the Casorati matrix and perform PCA analysis on it to obtain the basis of the linear subspace {φ r (x)} and the corresponding coefficients {a r }, the image to be reconstructed is expressed as: Where R is the number of subspace bases.

8. The magnetic resonance reconstruction device integrating signal model and deep learning according to claim 7, characterized in that: The preliminary reconstruction method of magnetic resonance images based on sparse sampling data and probabilistic subspace model is: Among them, d is sparse sampling data, is the variance of the noise, Ω is the sampling mode, is the Fourier transform, for any a r , its probability density function Pr(a r ) is estimated by the Gaussian mixture algorithm.

9. The magnetic resonance reconstruction device integrating signal model and deep learning according to claim 6, characterized in that: The second data acquisition module removes the global reconstructed image obtained by the first deep learning network reconstruction from the sparse sampling data to obtain the sparse sampling data of the residual features, that is: Among them, d is the sparse sampling data, Ω is the sampling mode, is the Fourier transform, is the global reconstructed image obtained by the first deep learning network reconstruction, Sparsely sampled data for residual features.

10. The magnetic resonance reconstruction device integrating signal model and deep learning according to claim 9, characterized in that: The preliminary reconstruction of the residual feature image based on the sparse model is specifically as follows: Where W is the sparse transformation, is the regularization parameter, is the residual feature image to be determined.

Images