Under-sampled magnetic resonance image reconstruction method based on reference images and data correction

Abstract

The application provides an undersampling magnetic resonance image reconstruction method based on a reference image and data correction, comprising the following steps: acquiring a full sampling T1 weighted image as a reference image I ref ; acquiring a full sampling T2 weighted image I T2 , and converting it into undersampling k-space data y; initializing an image sequence I s and initializing a mutual information value sequence MI s ; using a convolutional neural network based on a residual module, establishing an undersampling magnetic resonance image reconstruction model based on the reference image I ref and the undersampling k-space data y; training the undersampling magnetic resonance image reconstruction model; recording the reconstructed image in the reconstruction process and the mutual information value thereof with the reference image; selecting the best reconstructed image based on the mutual information value; and performing iterative k-space data correction on the best reconstructed image to obtain a final reconstructed image. The application can improve the accuracy of the reconstructed image.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for realizing unsupervised undersampling magnetic resonance image reconstruction, in particular to improving the quality of reconstruction based on unsupervised learning of reference images, introducing a k-space data correction module to speed up the convergence speed of the convolutional neural network, and improving the reconstruction accuracy by iterative k-space data correction in post-processing. BACKGROUND

[0002] Magnetic Resonance Imaging (MRI) has made a great leap in the field of life science and medicine in the past 30 years, and has become an important tool for clinical diagnosis and basic research [1]. In clinical diagnosis and later image analysis, full-sampling MRI images can provide rich auxiliary information, but full-sampling MRI scanning time is relatively long, and the imaging process takes more than 30 minutes. Therefore, in actual imaging, the resolution of magnetic resonance images is limited by various factors, such as hardware conditions, scanning time, and patient tolerance, etc. Generally, undersampling technology is used, that is, part of the frequency information is sampled to reduce the time of magnetic resonance imaging, and the less the part of the sampling, the faster the imaging time; however, the undersampled magnetic resonance image often has artifacts, blurred details, etc., which leads to its inability to be applied to clinical diagnosis.

[0003] Under the condition of not changing the hardware condition, how to reduce the MRI scanning time and reconstruct the image with the required resolution is a problem to be solved. The commonly used method of accelerating imaging based on software algorithm mainly includes two types:

[0004] (1) Based on supervised machine learning algorithm. This kind of algorithm designs a machine learning model, learns the mapping function between the undersampled image and its corresponding full-sampling image through a large-scale image training set [2]-[7]; so that in the prediction, the learned mapping function can be directly used to reconstruct the full-sampling image from the same type of undersampled image. This kind of algorithm needs to prepare a large number of pairs of undersampling / full-sampling image data in advance, which is difficult to do in a clinical scenario, and at present, simulated data is mostly used for training, which limits the application.

[0005] (2) Based on unsupervised machine learning algorithm. This kind of algorithm only uses one or several reference images and undersampling magnetic resonance data to reconstruct the image, without the need for a large training set [8]-

[11] . The current algorithm has the problems of not fully utilizing the reference image information and not fully utilizing the undersampling original data.

[0006] [1]. Ma D, Gulani V, Seiberlich N, Liu KC, Sunshine JL, Duerk JL, Griswold MA. Magnetic Resonance Fingerprinting. Nature, 495: 187-192 (2013).

[0007] [2]. Yang G, Yu S, Dong H, et al. DAGAN: Deep De-Aliasing generative adversarial networks for fast compressed sensing MRI reconstruction [J]. IEEE Transactions on Medical Imaging, 2017.

[0008] [3]. Lee D, Yoo J, Ye J C. Deep residual learning for compressed sensing MRI [C] / / Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on. IEEE, 2017: 15-18.

[0009] [4]. Li R, Zhang W, Suk H I, et al. Deep learning based imaging data completion for improved brain disease diagnosis [C] / / International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2014: 305-312.

[0010] [5]. Huang Y, Shao L, Frangi A F. Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding [J]. arXiv preprint arXiv: 1705.02596, 2017.

[0011] [6].Schlemper J,Caballero J,Hajnal J V,et al.A deep cascade ofconvolutional neural networks for dynamic MR image reconstruction[J].IEEEtransactions on medicalimaging,2018,37(2):491-503.

[0012] [7].X.Du and Y.He,“Gradient-guided convolutional neural network formriimage super-resolution,”Applied Sciences,vol.9,no.22,p.4874,2019.

[0013] [8].B.Yaman,S.A.H.Hosseini,and M. “Zero-shot self-supervisedlearning for mri reconstruction,”arXiv preprint arXiv:2102.07737,2021.

[0014] [9].D.Ulyanov,A.Vedaldi,and V.Lempitsky,“Deep image prior,”inProceedings of the IEEE conference on computer vision and patternrecognition,2018,pp.9446–9454.

[0015]

[10] .D.Zhao,F.Zhao,and Y.Gan,“Reference-driven compresse dsensing mrimage reconstruction using deep convolutional neural networks without pre-training,”Sensors,vol.20,no.1,p.308,2020.

[0016]

[11] . F. Hashimoto, K. Ote, T. Oida, A. Teramoto, and Y. Ouchi, "Compressed-sensing magnetic resonance image reconstruction using an iterative convolutional neural network approach," Applied Sciences, vol. 10, no. 6, p. 1902, 2020. SUMMARY

[0017] The present application aims to overcome the shortcomings of the prior art, and provide an undersampled magnetic resonance image reconstruction method based on reference images and data correction, which has fast convergence speed and high reconstruction accuracy.

[0018] In order to achieve the above-mentioned application purposes, the technical solutions adopted are as follows:

[0019] An undersampled magnetic resonance image reconstruction method based on reference images and data correction, comprising the following steps:

[0020] Obtain a full-sampling T1 weighted image as a reference image I ref Obtain a full-sampling T2 weighted image I T2 , and convert it into undersampled k-space data y;

[0021] Initialize an image sequence I s And initialize the mutual information value sequence MI s ;

[0022] Use a convolutional neural network based on a residual module to establish an undersampled magnetic resonance image reconstruction model based on the reference image I ref And the undersampled k-space data y;

[0023] Train the undersampled magnetic resonance image reconstruction model;

[0024] Record the reconstructed image and its mutual information value with the reference image during the reconstruction process;

[0025] Select the best reconstructed image based on the mutual information value;

[0026] Iteratively correct the k-space data of the best reconstructed image to obtain the final reconstructed image.

[0027] Preferably, a full-sampling T2 weighted image I T2 Is obtained and converted into undersampled k-space data y, which specifically comprises:

[0028] The weighted image I T2The Fourier transform is performed to obtain k-space data, and the k-space data is multiplied by an undersampling mask M to obtain corresponding undersampling k-space data y, as follows:

[0029] y = M O FI T2

[0030] Wherein, F represents the Fourier transform.

[0031] Preferably, the undersampling mask M is a Cartesian undersampling mask, and the sampling rate is 10%.

[0032] Preferably, a convolutional neural network based on a residual module is used to establish an undersampling magnetic resonance image reconstruction model based on a reference image I ref and undersampling k-space data y, and the model specifically comprises:

[0033] The convolutional neural network is stacked by multiple residual modules, and the input is the reference image I ref , and the output is the reconstructed image I t ; the mapping function from the input end to the output end is f (Θ|I ref ), and the mapping function has parameters Θ = {W1, W2, … W L ; B1, B2, … B L}, wherein W l represents the weight matrix of the lth layer, B l represents the bias of the lth layer, and L is the total number of network models; given the undersampling k-space data y and the corresponding reference image I ref as the network input, and the reconstructed image as the network output, the loss function is defined as:

[0034] I' = f (Θ|I ref )

[0035] I t = f dc (I') = |F -1 ((1-M) O FI' + y)|

[0036] E (Θ) = ‖y-M O Ff dc (I ′ )‖ 2

[0037] Wherein, F represents the Fourier transform, FI' represents the Fourier transform of the image I', F -1 represents the inverse Fourier transform, 1 represents a matrix with the same size as M and all values are 1, O represents the point multiplication between matrices, f dc represents the data correction operation, and ||·|| 2 represents the square of the matrix norm.

[0038] Preferably, the training undersampled magnetic resonance image reconstruction model comprises the following steps:

[0039] The optimal value of the parameter Θ in the mapping function f(Θ|I ref ) is estimated by minimizing the loss function E(Θ) The minimization of the loss function is realized by an adaptive gradient descent algorithm and a standard backpropagation algorithm.

[0040] Preferably, the reconstruction images in the reconstruction process and their mutual information values with the reference images are recorded, comprising the following steps:

[0041] Every n times of training, the i-th reconstruction image I i is stored in the reconstruction image sequence I s = I s ∪I i , the mutual information value of I i with the reference image I ref is calculated and stored in MI s = MI s ∪MI i , wherein

[0042]

[0043] wherein p(x) is the probability density of the gray level x in the image I i , p(y) is the probability density of the gray level y in the image I ref , and p(x, y) is the joint probability density of the gray level x in the image I i and the gray level y in the image I ref .

[0044] Preferably, the optimal reconstruction image is selected based on the mutual information values, comprising the following steps:

[0045] The optimal reconstruction magnetic resonance image I0∈I s is selected from I s , i is the subscript corresponding to the maximum value in the mutual information value set, and the following applies:

[0046]

[0047] Preferably, the optimal reconstruction image is iteratively corrected in k-space data to obtain a final reconstruction image, comprising the following steps:

[0048] The data of I0is corrected K times to obtain the final reconstruction image I K ,

[0049] I k = f dc (I k-1 )k = 1, 2, 3…K.

[0050] From the above description of the present application, compared with the prior art, the present application has the following beneficial effects:

[0051] The method of the present application realizes the reconstruction of undersampled images by using unsupervised machine learning, introduces a reference image as input, designs a convolutional neural network model to extract reference image features to guide the entire reconstruction process, and uses a k-space data correction module to accelerate the convergence speed of the reconstruction in the reconstruction process; the method selects the best output reconstructed image according to the mutual information value of the network model output and the reference image, so as to improve the accuracy of the reconstructed image. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The flowchart of the undersampled magnetic resonance image reconstruction method based on reference images and data correction of the present embodiment;

[0053] Figure 2 The flowchart of the present embodiment;

[0054] Figure 3 Fig. 4 is a schematic diagram of Cartesian, Gaussian and Poisson undersampling masks with a sampling rate of 10%; wherein (a) represents a Cartesian undersampling mask; (b) represents a Gaussian undersampling mask; (c) represents a Poisson undersampling mask;

[0055] Figure 4 Fig. 6 is a schematic diagram of a residual module;

[0056] Figure 5 Fig. 7 is a schematic diagram of a data correction module;

[0057] Figure 6 Fig. 8 is a curve of the mutual information value of the reconstructed image and the reference image in the training process, and a curve of the peak signal-to-noise ratio of the reconstructed image and the real image;

[0058] Figure 7 Fig. 9 is the MRI brain super-resolution reconstruction result of the undersampled magnetic resonance data under the Cartesian mask with a sampling rate of 10%, wherein (a) represents a real full-sampling T2 weighted image; (b) represents a reference full-sampling T1 weighted image; (c) represents an undersampling zero-padding reconstructed T2 image; (e) represents a supervised convolutional neural network reconstructed T2 image; (g) represents a T2 image reconstructed by the method; (d) represents a reconstructed error image of the image in (c); (f) represents a reconstructed error image of (e); (h) is a reconstructed error image of (g). DETAILED DESCRIPTION

[0059] The present application will be further described below with reference to the accompanying drawings.

[0060] The present application is an example of a multi-resolution reconstruction procedure of multi-contrast MRI brain atlas using convolutional neural network, which is a detailed description of the method proposed in the present application.

[0061] Referring to Figure 1 and Figure 2 , the present embodiment is an undersampled magnetic resonance image reconstruction method based on reference images and data correction, comprising the following steps:

[0062] S101, introducing full-sampled magnetic resonance images of different contrasts as reference images.

[0063] The data set used in the present example is a magnetic resonance image from the NAMIC MRI database (http: / / hdl.handle.net / 1926 / 1687), with an image pixel size of 1x1mm, a slice thickness of 1mm, and a data size of 256x256x176.

[0064] Among them, the full-sampled high-resolution T1 weighted image I ref .

[0065] The undersampling process is as follows:

[0066] For the full-sampled high-resolution T2 weighted image I T2 , first Fourier transform to k-space, point multiplication of k-space data and undersampling mask, to get the corresponding undersampled T2 weighted k-space data y.

[0067] Referring to Figure 3 , the undersampling mask includes Cartesian undersampling mask, Gaussian undersampling mask and Poisson undersampling mask. The Cartesian undersampling mask M is used in the present embodiment, and the sampling rate is 10%. Among them, the column with value 1 in M is the sampling column, and the column with value 0 in M is not sampled.

[0068] y=M⊙FI T2

[0069] S102, initializing the image sequence and initializing the mutual information value sequence.

[0070] Specifically, the initialized image sequence I s ={}, and the initialized mutual information value sequence MI s ={}.

[0071] S103, using a convolutional neural network based on a residual module to establish an undersampled magnetic resonance image reconstruction model based on reference images and undersampled k-space data y.

[0072] Specifically, the reconstruction model is composed of one convolutional layer at the beginning and one convolutional layer at the end, and eight residual modules in the middle. The first convolutional layer has 32 convolutional kernels with a size of 3*3; each residual module has two convolutional layers, and the first convolutional layer of each layer has 32 convolutional kernels with a size of 3*3. The output of the residual module is the result of cross-adding between the input of the residual module and the output of the second convolutional layer. The last layer is a convolutional kernel with a size of 1*32*3*3.

[0073] The residual module is shown in Figure 4 .

[0074] The mean square error between the output of the model and the corresponding undersampled T2 weighted data y is taken as the loss function E(Θ) of the model:

[0075] I′=f(Θ|I ref )

[0076] I t =f dc (I′)=|F -1 ((1-M)⊙FI′+y)|

[0077] E(Θ)=‖y-M⊙Ff dc (I ′ )‖ 2

[0078] Where F represents the Fourier transform, FI′ represents the Fourier transform of the image I′, F -1 represents the inverse Fourier transform, 1 represents a matrix with all values being 1 and the same size as M, ⊙ represents the dot product between matrices, f dc represents the data correction operation, and ||·|| 2 represents the square of the matrix norm.

[0079] Specifically, the data correction module is shown in Figure 5 .

[0080] S104, training the undersampled magnetic resonance image reconstruction model.

[0081] The training of the model is to estimate the optimal value of the parameter Θ in the mapping function f(Θ|I ref ) by minimizing the loss function E(Θ). The minimization of the loss function is realized by the adaptive gradient descent algorithm and the standard back propagation algorithm.

[0082] S105, recording the reconstructed image in the reconstruction process and the mutual information value thereof and the reference image.

[0083] The entire training process is set to 5000 iterations, and the reconstructed image and its mutual information value with the reference image are stored every 100 iterations. Store the i×100th reconstructed image I i To reconstruct image sequence I s =I s ∪I i , calculate I i With reference image I ref The mutual information value is stored in MI s =MI s ∪MI i .

[0084]

[0085] Where p(x) is the i The probability density of gray level x, p(y) is shown in Figure I ref The probability density of gray level y, p(x,y) refers to the image I i The gray level x and image I red The gray level is the joint probability density of y.

[0086] See also Figure 6 As shown in FIG, the mutual information value between the reconstructed image and the reference image during the training process of this embodiment, as well as the peak signal-to-noise ratio between the reconstructed image and the real image changes with the training process value. Figure 5 As can be seen, in the early stages of training, as the number of training times increases, the mutual information value between the reconstructed image and the reference image increases, and the quality of the reconstructed image also improves accordingly. However, in the later stages of training, the quality of the reconstructed image begins to decline, indicating that the training has overfitted, and the mutual information value between the reconstructed image and the reference image also decreases. This shows that the mutual information value can be used as a basis for judging whether the reconstructed image is optimal.

[0087] S106, selecting the best reconstructed image as the network output.

[0088] byI s Select the optimal reconstructed magnetic resonance image I0∈I s , i is the subscript corresponding to the maximum value in the mutual information value set,

[0089]

[0090] S107 , performing iterative k-space data correction on the optimal reconstructed image to obtain a final reconstructed image.

[0091] Specifically, data correction is performed on I0 100 times to obtain the final reconstructed image I 100 ,as follows:

[0092] Ik = f dc (I k-1 ) k = 1, 2, 3…100.

[0093] Specifically, the schematic diagram of the data correction module is shown in Figure 5 .

[0094] Further, referring to Figure 7 , it is the under-sampling magnetic resonance data under the Cartesian mask with a sampling rate of 10%, the MRI brain map super-resolution reconstruction results of the supervised method and the method; wherein (a) represents the real full-sampling T2 weighted image; (b) represents the reference full-sampling T1 weighted image; (c) represents the under-sampling zero-padding reconstructed T2 image; (e) represents the supervised convolutional neural network reconstructed T2 image; (g) represents the T2 image reconstructed by the method; (d) represents the reconstruction error map of the (c) image; (f) represents the reconstruction error map of (e); (h) is the reconstruction error map of (g).

[0095] It can be seen from Figure 7 that the method has small reconstruction error and high reconstruction accuracy.

[0096] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application using this concept shall be regarded as an act of infringing the protection scope of the present application.

Claims

1. A method of undersampled magnetic resonance image reconstruction based on reference images and data correction, characterized by, The method comprises the following steps: acquire a fully sampled T1 weighted image as a reference image I ref ; acquire a fully sampled T2 weighted image I T2 and convert to undersampled k-space data y; initializing an image sequence I s and an initializing mutual information value sequence MI s ; Using a convolutional neural network based on a residual module, a reference image I ref and an undersampled magnetic resonance image reconstruction model of the undersampled k-space data y; training an undersampled magnetic resonance image reconstruction model; recording the reconstructed images in the reconstruction process and their mutual information values with the reference images; selecting the best reconstructed image based on the mutual information values; iteratively correcting the k-space data of the best reconstructed image to obtain a final reconstructed image; acquiring a fully sampled T2 weighted image I T2 and converting to undersampled k-space data y, in particular comprising: The weighted image I T2 Fourier transforming to k-space, and multiplying the k-space data with an undersampling mask M to obtain corresponding undersampling k-space data y, as follows: y = M O FI T2 wherein F represents Fourier transform; recording the reconstructed images in the reconstruction process and their mutual information values with the reference images, specifically comprising: Every n-th training interval, store the i-th reconstructed image I i to the sequence of reconstructed images I s = I s ∪ I i where the mutual information value I i with the reference image I ref is calculated and stored into MI s = MI s ∪ MI i where where P(x) is the probability density of gray level x in image I i p(y) is the probability density of gray level y in image I ref p(x,y) refers to the joint probability density of gray level x in image I i and gray level y in image I ref ; recording the reconstructed images in the reconstruction process and their mutual information values with the reference images, specifically comprising: Every n-th training interval, the i-th reconstructed image I is stored i to the sequence of reconstructed images I s = I s ∪ I i The mutual information value MI between I and the reference image I is calculated and stored into MI i = MI ref ∪ MI s = MI s ∪ MI i where where p(x) is the probability density of gray level x in image I i p(y) is the probability density of gray level y in image I ref p(x,y) refers to the joint probability density of gray level x in image I i and gray level y in image I ref ; Using a convolutional neural network based on a residual module, a reference image I ref And the undersampling magnetic resonance image reconstruction model of the undersampling k-space data y, specifically comprising: The convolutional neural network is stacked by multiple residual models, an input is a reference image I ref , an output is a reconstructed image I t ; a mapping function from the input to the output is f(Θ|I ref ), the mapping function has parameters Θ={W1, W2, …W L ; B1, B2, …B L}, wherein W l represents a weight matrix of the lth layer, B l represents a bias of the lth layer, and L is a total number of network models; given the undersampled k-space data y and the corresponding reference image I ref as the network input, the reconstructed image is taken as the network output, and a loss function is defined as: I' = f(Θ | I ref ) I t = f dc (I') = |F -1 ((1 - M) O FI' + y) | E(Θ) = ||y - M Θ Ff dc (I')‖ 2 where F denotes the Fourier transform, F I' denotes the Fourier transform of the image I', F -1 denotes the inverse Fourier transform, 1 denotes a matrix of size M filled with ones, denotes the pointwise multiplication between matrices, f dc denotes the data modification operation, ||·|| denotes the matrix norm. 2 denotes the square of the matrix norm.

2. The reference image and data correction based undersampled magnetic resonance image reconstruction method of claim 1, wherein, The undersampling mask M is a Cartesian undersampling mask, and the sampling rate is 10%.

3. The reference image and data correction based undersampled magnetic resonance image reconstruction method of claim 1, wherein, Training an undersampled magnetic resonance image reconstruction model, specifically comprising: The optimal values of the parameters Θ in the mapping function f(Θ| I ref ) are estimated by minimizing the loss function E(Θ) The minimization of the loss function is achieved by an adaptive gradient descent algorithm and the standard backpropagation algorithm.

4. The reference image and data correction based undersampled magnetic resonance image reconstruction method of claim 3, wherein, Selecting the best reconstructed image based on the mutual information values, specifically comprising: I0= arg max I∈I I s selecting an optimal reconstructed magnetic resonance image I0∈I s i is the index corresponding to the maximum value in the set of mutual information values, as follows:

5. The reference image and data correction based undersampled magnetic resonance image reconstruction method of claim 4, wherein, Iteratively correcting the k-space data of the best reconstructed image to obtain a final reconstructed image, specifically comprising: Data is corrected K times to I0 to obtain the final reconstructed image I K , I k = f dc (I k-1 )k = 1,2,3...K.

Images