A training method for MRI reconstruction networks based on data from multiple scanners

Abstract

A training method for an MRI reconstruction network combining data from multiple scanners, which makes full use of the compensatory effect of detail information from high-resolution scanner data on low-resolution scanner data to improve the reconstruction performance of a deep neural network on a target scanner; first, the auxiliary scanners and target scanners are sorted according to the resolution; second, in each batch of the training process, the network is trained in sequence using batch data randomly generated from data from different scanners according to a strategy of resolution from large to small; this process is iterated for multiple rounds until the number of training rounds reaches a preset maximum value. The present invention can effectively organize MRI data with different resolutions from different manufacturers to train the network, and can effectively train the target scanner. Compared with using only a single scanner data or multiple scanner data in disorder, the method provided by the present invention achieves better reconstruction effects and has strong practicality.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, in particular to the field of medical image technology of magnetic resonance imaging, and is mainly used for realizing accelerated reconstruction of medical magnetic resonance imaging. Background Art

[0002] Magnetic resonance imaging (MRI) is one of the most important non-invasive examination methods in clinical medical imaging. It is another major advancement in the imaging industry after CT. Its imaging quality is better than that of B-ultrasound and CT, but it does not produce ionizing radiation like CT and other imaging methods. It has the advantages of temperature sensitivity, multi-directional, multi-parameter, multi-modal, and multi-faceted imaging capabilities, and has been widely used in various clinical applications such as angiography, neurology, and cardiology examinations. Today, MRI is widely used in clinical practice, and at least 60 million cases are examined using MRI technology every year around the world.

[0003] However, slow imaging speed and high dielectric artifacts remain two major problems in the MRI field for many years. This will lead to the following three problems: (1) it will cause high examination costs; (2) it will greatly affect the comfort and compliance of patients, resulting in local artifacts in magnetic resonance images due to involuntary movements of patients; (3) it is difficult to apply to the screening of diseases that require high diagnostic time, such as stroke. Since the emergence of MRI in the 1970s, improving imaging speed has been a major ongoing research goal and one of the research hotspots in recent years.

[0004] Recent studies have shown that image reconstruction techniques using compressed sensing technology and deep neural networks can achieve fast and accurate reconstruction of highly undersampled magnetic resonance images with an acceleration ratio higher than the Shannon-Nyquist sampling ratio. It is well known that the training of deep neural networks usually requires large-scale training data, and too little training data may lead to overfitting of the network. However, in reality, MRI training data is often seriously insufficient. This is because the acquisition cost of MRI data is usually expensive: it takes a long time to acquire data, the cost of the MRI scanner itself is relatively high, and high labor costs are required. Therefore, in order to obtain large-scale training data, it is often necessary to use data collected from multiple MRI scanners to train deep networks.

[0005] However, when using multiple scanners for network training, two problems arise: (1) The problem of resolution differences between scanners: The resolution of magnetic resonance images acquired by different scanners from different manufacturers is often different, and it is difficult to organize these data together for batch training of deep networks; (2) Experiments show that for a given target scanner, it is not possible to obtain better reconstruction performance by training the network using data from more auxiliary scanners. For example, for a 3T target scanner, if multiple 1.5T auxiliary scanner data are mixed at the same time to train the deep network, it is often not as good as using only the training data from the target scanner. The present invention provides solutions to these two problems and proposes an effective training method for MRI reconstruction networks based on data from multiple scanners. Summary of the invention

[0006] In order to obtain the best possible reconstruction effect for the undersampled MRI images of the target scanner, it is necessary to make full use of as rich MRI data as possible to train the deep network model. The present invention makes full use of the data collected by different scanners to train the network model. Since the resolutions of different scanners are generally different (the size of the corresponding k-space data is also different), it is difficult to mix the data of multiple scanners together for batch training. The present invention proposes that in a training batch, the data of different scanners are used in turn according to the size of the resolution of each scanner to train the deep network model.

[0007] The specific technical solutions provided by the present invention are as follows:

[0008] A method for training an MRI reconstruction network based on multiple scanner data comprises the following steps:

[0009] Step 1: Note that the target scanner is scanner No. 0, and select the L collected by the target scanner. 0 Personal k-space MRI data constitutes a collection in, Represents a complex number domain.

[0010] Step 2: Select the k-space MRI data collected by D auxiliary scanners to form a set Y = {Y 1 ,Y 2 ,…,Y D},in, And M d and N d Must meet: M d >M d-1 >M 0 or N d >N d-1 >N 0 .

[0011] Step 3: Build a deep neural network model for MRI reconstruction on the target scanner CNN (·; Θ), where Θ is f CNN Parameter set.

[0012] Step 4: Construct a training model from the k-space MRI data collected by the D+1 scanner. CNN For The training set for the dth scanner is constructed as follows:

[0013]

[0014] in, Here, F -1 represents the two-dimensional inverse Fourier transform, represents the mask matrix for undersampling the MRI k-space data by a factor of u. Express The k-space data is undersampled by a factor of u.

[0015] Step 5: Set the training set X of the target scanner 0 Further split into training subsets and validation subset And let

[0016] Step 6 Based on the above D+1 scanner data: X 0 ,X 1 ,X 2 ,…,X D , training f CNN (·; Θ), get the optimal parameter set Θ * .

[0017] Further, the specific steps of step 6 are as follows:

[0018] Step 6.1 Randomly initialize f CNN The parameter set Θ of (·; Θ) is Θ (0) ;

[0019] Step 6.2: Let the number of iterations e = 1, and the number of iterations t = 1;

[0020] Step 6.3 For each scanner d∈{0,1,2,…,D}, set the training set X d Randomly split into B subsets:

[0021] X d =X d；1 ∪X d；2 …∪X d；B ;

[0022] Step 6.4: Let iterative batch b = 1, and select scanner d = D;

[0023] Step 6.5 For all By CNN (·; Θ (t-1) )estimate The reconstructed image: And construct a set of reconstructed images and their fully sampled images:

[0024]

[0025] Step 6.6 Use the optimization algorithm to solve the following objective formula:

[0026]

[0027] in, is the loss function;

[0028] Step 6.7 On the validation set Evaluate the deep neural network f CNN (·; Θ (t) )'s reconstruction performance:

[0029]

[0030] in, is the quality assessment function of MRI images;

[0031] Step 6.8: Let d = d-1, t = t+1, and iterate the above steps 6.5 to 6.7 until d = 0;

[0032] Step 6.9: Let b = b + 1, and iterate the above steps 6.5 to 6.8 until b = B;

[0033] Step 6.10: Let e=e+1, and iterate the above steps 6.3 to 6.9 until e=E, where E is the preset maximum number of training rounds;

[0034] Step 6.11 Select the current deep neural network f CNN The optimal parameter set for (·; Θ) in,

[0035] The beneficial effects of the present invention are: it can effectively organize MRI data with different resolutions from different manufacturers to train the network, and can effectively train the target scanner. Compared with using only a single scanner data or disordered multi-scanner data, the method provided by the present invention achieves better reconstruction effect and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a training flow chart of the MRI reconstruction network based on multiple scanner data proposed in the present invention.

[0037] Figure 2 It is a diagrammatic illustration of the main training process of training an MRI reconstruction network based on multiple scanner data proposed in the present invention, wherein at both ends of the network, from top to bottom, there are under-sampled data from scanners No. 1, No. 0 and No. 2, and the dotted arrows indicate that the network input or output is MRI data from an auxiliary scanner; during the training process, the network is trained using data from scanners No. 2, No. 1 and No. 0 in order of resolution from high to low.

[0038] Figure 3 A visualization of the k-space MRI data acquired by the target scanner and two auxiliary scanners used in the present invention is given, wherein the first row shows the k-space MRI data of the target scanner (scanner No. 0) Siemens Aera 1.5T, with a resolution of 224×128; the second row shows the k-space MRI data of the auxiliary scanner No. 1 Philips Ingenia 3T, with a resolution of 336×261; the third row shows the k-space MRI data of the auxiliary scanner No. 2 Siemens Verio 3T, with a resolution of 512×288.

[0039] Figure 4 The MRI images generated by the target scanner and two auxiliary scanners used in the present invention are presented, and from the first row to the third row are: the MRI image of the target scanner No. 0 Siemens Aera 1.5T, the MRI image of the auxiliary scanner No. 1 Philips Ingenia 3T, and the MRI image of the auxiliary scanner No. 2 Siemens Verio 3T.

[0040] Figure 5 is the network structure diagram of the deep cascade network used in the present invention, where Conv i,j represents the jth convolutional layer of the ith subnet, DC represents the data consistency operator, and y u is the u-fold undersampled MRI image in the input k-space, and ⊕ represents the sum operator.

[0041] Figure 6 (a) is an example of a center under-masked matrix, Figure 6 (b) is an example of a Gaussian mask matrix.

[0042] Figure 7 The central mask matrix is ​​used Figure 3An MRI image generated by undersampling the fully sampled k-space data in .

[0043] Figure 8 is an example of an artificially interpolated magnetic resonance image, wherein sub-image (a) is an image acquired by the target scanner, with a resolution of 224×128; sub-image (b) is an image acquired by interpolating the image acquired by the target scanner by a factor of 1.5 in the X-axis direction (i.e., horizontal direction) using the bilinear interpolation method, with a resolution of 224×192; sub-image (c) is an image acquired by interpolating the image acquired by the target scanner by a factor of 1.5 in the Y-axis direction (i.e., vertical direction) using the bilinear interpolation method, with a resolution of 336×128; (d) is an image acquired by interpolating the image acquired by the target scanner by a factor of 1.5 in both the X-axis and Y-axis directions using the bilinear interpolation method, with a resolution of 336×192.

[0044] Figure 8 The reconstruction effects of deep cascade networks trained on different scanner data on 8x undersampled MRI data from scanner 0 are compared. For comparison, two regions of interest are selected from each reconstructed image and displayed enlarged and placed below the corresponding reconstructed image. The training data used from the first column on the left to the last column on the right come from the scanners: {0}, {0,1}, {0,2}, {0,1,2}; Here, represents the reconstructed image obtained by directly using the “zero filling” method (i.e., directly filling the missing data in the undersampled k-space with 0) without using any training data from any scanner; {i 1 ,i 2 ,…} represents the i-th 1 No. i 2 The target scanner is highlighted. Here, the i-th 1 Scanner No. is the target scanner.

[0045] Fig. 9 Comparison of the reconstruction effects of deep cascade networks trained on different scanner data on 12x undersampled MRI data from scanner 2. The first row uses the following training data from left to right: {2}, {2,0}, {2,1}, {0,1,2}, the training data used in the second row from left to right comes from the scanner: {2}, {2,2-X}, {2,2-Y}, {2,2-X,2-Y,2-XY}. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings.

[0047] Reference Figures 1 to 9 , a training method for MRI reconstruction network based on multiple scanner data, the training process is as follows Figure 1 As shown, Figure 2 A visual representation of the main training steps is given, which includes the following steps:

[0048] Step 1: Note that the target scanner is scanner No. 0, and select the L collected by the target scanner. 0 Personal k-space MRI data, forming a collection in, Represents a complex number domain.

[0049] Step 2: Select the k-space MRI data collected by D auxiliary scanners to form a set Y = {Y 1 ,Y 2 ,…,Y D},in, And M d and N d Must meet: M d >M d-1 >M 0 or N d >N d-1 >N 0 . Figure 3 Using Siemens Aera 1.5T as the target scanner and PhilipsIngenia 3T and Siemens Verio 3T as auxiliary scanners, a visualization of the acquired k-space MRI data is given; Figure 4 The corresponding MRI images are presented.

[0050] Step 3: Build a deep neural network model for MRI reconstruction on the target scanner CNN (·; Θ), where Θ is f CNN The parameter set, f CNN (·; Θ) can be any neural network that can be used for image reconstruction. Commonly used networks include U-type networks (UNet), deep cascade networks, Figure 5 Taking the deep cascade network as an example, f is given CNN An example of .

[0051] Step 4: Construct training f from the MRI data collected by the above D+1 scanner CNN For The training set for the dth scanner is constructed as follows:

[0052]

[0053] in, Here, F -1 represents the two-dimensional inverse Fourier transform, represents the mask matrix for undersampling the MRI k-space data by a factor of u. Express The k-space data is undersampled by u times. The mask matrix can use a central mask matrix or a Gaussian mask matrix. Figure 6 (a) gives an example of a center mask matrix, Figure 6 (b) gives an example of a Gaussian under-masked matrix; Figure 7 Given the central mask matrix pair Figure 3 An MRI image generated by undersampling the fully sampled k-space data in .

[0054] Step 5: Set the training set X of the target scanner 0 Further split into training subsets and validation subset And let

[0055] Step 6: Based on the training data of D+1 scanners above: X 0 ,X 1 ,X 2 ,…,X D Training CNN (·; Θ), get the optimal parameter set Θ * .

[0056] Further, the process of step 6 is as follows:

[0057] Step 6.1 Randomly initialize f CNN The parameter set Θ of (·; Θ) is Θ (0) , the initialization method can be to sample from a Gaussian distribution with a mean of 0 and a variance of 1 as the initial weight, and set the initial value of the bias term to 0;

[0058] Step 6.2: Let the number of iterations e = 1, and the number of iterations t = 1;

[0059] Step 6.3 For each scanner d∈{0,1,2,…,D}, set the training set X d Randomly split into B subsets:

[0060] X d =X d；1 ∪X d；2 …∪X d；B ;

[0061] Step 6.4: Let iterative batch b = 1, and select scanner d = D;

[0062] Step 6.5 For all By CNN (·; Θ (t-1) )estimate The reconstructed image: And construct a set of reconstructed images and their fully sampled images:

[0063]

[0064] Step 6.6 Use the optimization algorithm to solve the following objective formula:

[0065]

[0066] in, is the loss function. Common optimization algorithms include stochastic gradient descent and Adam algorithm; Norm, norm or Charbonnier penalty function, where the Charbonnier penalty function is defined as: where x 1 、x 2 are the reconstructed image and the original image respectively, ∈ is a constant, which can be set to ∈=0.001;

[0067] Step 6.7 On the validation set Evaluate the deep neural network f on CNN (·; Θ (t) )'s reconstruction performance:

[0068]

[0069] in, is the quality assessment function of the MRI image, and the peak signal-to-noise ratio (PSNR) or the structural similarity (SSIM) can be used as the quality assessment function;

[0070] Step 6.8: Let d = d-1, t = t+1, and iterate the above steps 6.3 to 6.7 until d = 0;

[0071] Step 6.9: Let b = b + 1, and iterate the above steps 6.5 to 6.8 until b = B;

[0072] Step 6.10: Let e=e+1, and iterate the above steps 6.3 to 6.9 until e=E, where E is the preset maximum number of training rounds;

[0073] Step 6.11 Select the current deep neural network f CNN The optimal parameter set for (·; Θ) in,

[0074] The following experiments are conducted to verify the beneficial effects of the deep neural network training method combining multiple scanner data proposed in the present invention.

[0075] Siemens Aera 1.5T (scanner No. 0) was selected as the target scanner with a resolution of 224 × 128; Philips Ingenia 3T and Siemens Verio 3T were selected as the No. 1 and No. 2 auxiliary scanners with resolutions of 336 × 261 and 512 × 288, respectively. 17 subjects were scanned: 5 subjects were scanned by each auxiliary scanner and 7 subjects were scanned by the target scanner to obtain the corresponding T2-weighted magnetic resonance images, each modality containing 260 slices in the axial direction. Figure 3 The k-space data collected by these three scanners are shown. Figure 4 The corresponding MRI images are shown. The training set and the validation set are constructed according to the method described in step 4 of the invention. Specifically, the MRI images of 5 subjects are selected from the data collected by each scanner as the training set, and the MRI images of the remaining two subjects (both from the target scanner) are selected as the validation set and the test set, respectively. A deep cascade network is selected, and its network structure is as follows Figure 5 As shown, as f CNN , the undersampled MRI images of the target scanner are reconstructed.

[0076] The Adam algorithm was used to optimize the objective function, with the momentum size set to 0.9, the batch size set to 5, the initial learning rate set to 0.001, and the learning rate decreased by 10 times after every 40 epochs, for 70 epochs. In order to quantitatively evaluate the reconstruction performance of the network, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used to evaluate the quality of the reconstructed magnetic resonance images.

[0077] Table 1 reports the reconstruction performance achieved by the deep cascade network on the test set when the downsampling factor u=8 is used for auxiliary training using training data from different scanners. Figure 8 The reconstruction effects of each MRI reconstruction image are visually compared. In Table 1, represents the reconstructed image obtained by directly using the “zero filling” method (i.e., directly filling the missing data in the undersampled k-space with 0) without using any training data from any scanner; {i 1 ,i 2 ,…} represents the i-th 1 No. i 2 The target scanner is highlighted. Here, the i-th 1Scanner No. is the target scanner. From Table 1 and Figure 8 The visualization of the first row shows that with the increase of auxiliary scanners, the reconstruction performance and quality of the network are gradually enhanced.

[0078]

[0079] Table 1

[0080]

[0081] Table 2

[0082] Furthermore, in order to verify that data augmentation does not necessarily lead to good reconstruction performance, Table 2 and Fig. 9 The first row uses the No. 2 scanner with the highest resolution as the target scanner, and the No. 0 and No. 1 scanners as auxiliary scanners, and performs u=12 times downsampling on the MRI data of the target scanner to verify the influence of the training data of different scanners on the reconstruction performance of the neural network. It can be seen that in this case, because the resolution relationship between the auxiliary scanner and the target scanner violates the constraint conditions in step 4 of the invention, data augmentation not only fails to bring better reconstruction performance, but makes the reconstruction performance of the network worse.

[0083] When the resolution of the target scanner itself is high, it is difficult to obtain training data with higher resolution from existing scanners. In order to effectively improve the reconstruction effect of the neural network, a method of manually augmenting the data set can be used. It can be seen from step 4 of the present invention that the resolution of each MRI image in the manually augmented data set must be higher than that of the target scanner at least in the horizontal direction (X-axis) or the vertical direction (Y-axis). Therefore, the following augmentations can be made: interpolating the augmented data set in the horizontal direction (X-axis), interpolating the augmented data set in the vertical direction (Y-axis), and interpolating the augmented data set in the horizontal (X-axis) and vertical (Y-axis) directions at the same time. When the target scanner is scanner No. 2, the present invention uses a bilinear interpolation method to simulate the MRI scanner, and interpolates the MRI data of scanner No. 2 along the X-axis direction, the Y-axis direction, and the (X, Y)-axis direction, respectively, to obtain an augmented data set, and the interpolation methods for generating these augmented data subsets are recorded as scanners No. 2-X, No. 2-Y, and No. 2-XY, respectively. For u=12, Tables 3 and Fig. 9 The second row compares the quantification results and the visualization results. It can be seen that the data augmentation method proposed in the present invention can also effectively improve the reconstruction effect of the under-sampled data of the high-resolution MRI scanner.

[0084]

[0085] Table 3

[0086] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A method for training an MRI reconstruction network based on multiple scanner data, characterized in that: The training method comprises the following steps: Step 1: The target scanner is designated as scanner No. 0, and the k-space MRI data of individual L0 collected by the target scanner is selected to form a set in, represents a complex domain; Step 2: Select the k-space MRI data collected by D auxiliary scanners to form a set Y = {Y 1 ,Y 2 ,…,Y D },in, And M d and N d Must meet: M d >M d-1 >M0 or N d >N d-1 >N0; Step 3: Build a deep neural network model for MRI reconstruction on the target scanner CNN (·; Θ), where Θ is f CNN Parameter set; Step 4: Construct a training model from the k-space MRI data collected by the D+1 scanner. CNN For the training set The training set for the dth scanner is constructed as follows: in, Here, F -1 represents the two-dimensional inverse Fourier transform, represents the mask matrix for undersampling the MRI k-space data by a factor of u. Express k-space data after u-fold undersampling; Step 5: Set the training set X of the target scanner 0 Further split into training subsets and validation subset And let Step 6 Based on the above D+1 scanner data: X 0 ,X 1 ,X 2 ,…,X D , training f CNN (·; Θ), get the optimal parameter set Θ ★ ; The process of step 6 is as follows: Step 6.1 Randomly initialize f CNN The parameter set Θ of (·; Θ) is Θ (0) ; Step 6.2: Let the number of iterations e = 1, and the number of iterations t = 1; Step 6.3 For each scanner d∈{0,1,2,…,D}, set the training set X d Randomly split into B subsets: X d =X d；1 ∪X d；2 …∪X d；B ; Step 6.4: Let iterative batch b = 1, and select scanner d = D; Step 6.5 For all By CNN (·; Θ (t-1) )estimate The reconstructed image: And construct a set of reconstructed images and their fully sampled images: Step 6.6 Use the optimization algorithm to solve the following objective formula: Where l(·,·) is the loss function, the optimization algorithm is stochastic gradient descent and Adam algorithm, and the l1 norm, l2 norm or Charbonnier penalty function is used. The Charbonnier penalty function is defined as: Where x1 and x2 are the reconstructed image and the original image respectively, and ∈ is a constant; Step 6.7 On the validation set Evaluate the deep neural network f on CNN (·; Θ (t) )'s reconstruction performance: Among them, Q(·,·) is the quality assessment function of MRI images, and the peak signal-to-noise ratio or structural similarity is used as the quality assessment function; Step 6.8: Let d = d-1, t = t+1, and iterate the above steps 6.5 to 6.7 until d = 0; Step 6.9: Let b = b + 1, and iterate the above steps 6.5 to 6.8 until b = B; Step 6.10: Let e=e+1, and iterate the above steps 6.3 to 6.9 until e=E, where E is the preset maximum number of training rounds; Step 6.11 Select the current deep neural network f CNN The optimal parameter set for (·; Θ) in,

Images