Dual-domain super-resolution reconstruction method of lung multi-core MRI based on artificial intelligence

Abstract

The present invention provides a dual-domain super-resolution reconstruction method for multi-core MRI of the lung based on artificial intelligence, including constructing a k-space reconstruction network and an image domain reconstruction network. The k-space reconstruction network is used to reconstruct the upsampled multi-core low-resolution k-space domain data, and then the k-space reconstruction network and the image domain reconstruction network are used to obtain a super-resolution multi-core image. The present invention adopts a dual-domain reconstruction network to effectively utilize the respective characteristics of the data in the k-space and image domains; and adopts image fusion technology to effectively utilize the correlation between multi-core data in space. Compared with traditional super-resolution reconstruction methods, the present invention can learn richer features and effectively improve the super-resolution reconstruction effect of multi-core magnetic resonance images of the lungs.

Description

Technical Field

[0001] The present invention belongs to the field of imaging technology, and in particular relates to a lung multi-core MRI dual-domain super-resolution reconstruction method based on artificial intelligence. Background Art

[0002] Magnetic resonance imaging (MRI) is non-invasive and non-radioactive, and can provide rich structural and functional information at the overall level. It has important and extensive applications in clinical diagnosis and preclinical research [Mehmet A, Steen M, Sebastian W, et al. Magn. Reson. Med., 2019, 439: 453.]. 1 H) is the most abundant nucleus in living bodies that can be observed by magnetic resonance imaging and has the highest gyromagnetic ratio. Classical MRI usually uses protons ( 1 H) as the imaging observation nucleus. 1 H has a larger chemical shift range, which can provide more comprehensive functional and metabolic information. In recent years, the MRI capability of magnetic resonance imaging to observe multi-nuclear elements has been continuously explored, especially for the development of lung MRI technology.

[0003] Lung diseases such as chronic obstructive pulmonary disease (COPD) and lung cancer are major diseases that threaten the lives and health of the Chinese people. Among them, there are nearly 100 million COPD patients in China, and the incidence and mortality of lung cancer account for 17.9% and 23.8% of all malignant tumors [https: / / gco.iarc.fr / ], ranking first among malignant tumors. [Spinelli A, Pellino G.Brit.J. Surg., 2020, 107:785.]. Multi-core MRI technology is expected to provide effective support for the diagnosis and treatment of major lung diseases.

[0004] However, multi-core MRI technology is limited by factors such as large data collection volume, long time consumption, and irreversible polarization of hyperpolarized gas during imaging, making it difficult to obtain high-quality multi-core MRI data of the lung. Recently, deep learning has achieved success in the field of super-resolution image reconstruction [Wei.L, Jia.H, Narendra.A.IEEE Trans. Pattern Anal., 2019, 41:2599]. By training pairs of low-resolution and high-resolution images with convolutional neural networks, low-resolution data can be effectively upgraded to high-resolution data, thereby improving data quality. Although deep learning algorithms can obtain high-quality super-resolution reconstructed images, a large number of high-resolution images are still required as labels. Multi-core MRI is susceptible to noise and artifacts, and there are fewer high-quality multi-core MRI images. It is difficult to extract rich image features from a small number of training set data, which affects the final super-resolution reconstruction results. Summary of the invention

[0005] The purpose of the present invention is to address the above-mentioned defects of the prior art and provide an artificial intelligence-based dual-domain super-resolution reconstruction method for lung multi-core MRI. The phase information in the k-space is retained by using a complex-valued convolutional layer, and segmentation and reconstruction tasks are learned at the same time, so that the convolutional neural network can make full use of the spatial correlation of multi-core data, as well as the dual-domain information of the k-space domain and the image domain, and learn richer features, effectively improving the image super-resolution reconstruction effect.

[0006] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0007] The artificial intelligence-based dual-domain super-resolution reconstruction method for lung multi-core MRI includes the following steps:

[0008] Step 1, constructing a training set and a test set, each of which includes multiple sample pairs, each of which includes multi-core high-resolution k-space data and corresponding multi-core low-resolution k-space data, performing inverse Fourier transform on the multi-core high-resolution k-space data in the training set to obtain a multi-core high-resolution image, and then taking the maximum value of the multi-core high-resolution image to obtain a multi-core high-resolution fused image;

[0009] Step 2: construct a k-space reconstruction network;

[0010] Step 3, inputting the multi-core low-resolution k-space data in the training set generated in step 1 into the k-space reconstruction network constructed in step 2 to obtain reconstructed multi-core k-space data;

[0011] Step 4: Construct an image domain reconstruction network;

[0012] Step 5: Perform a two-dimensional Fourier inverse transform on the reconstructed multi-core k-space data obtained in step 3 to obtain multi-core image domain data, then take the maximum value of the multi-core image domain data to obtain multi-core fusion data, and input the multi-core image domain data and the multi-core fusion data into the image domain reconstruction network constructed in step 4 to obtain a super-resolution multi-core reconstructed image and a super-resolution multi-core fusion image I. A , and then take the maximum value of the super-resolution multi-core reconstructed image to fuse it to obtain the fused super-resolution multi-core reconstructed image I B ;

[0013] Step 6: Define the loss function;

[0014] Step 7: According to the loss function set in step 6, the k-space reconstruction network in step 2 and the image domain reconstruction network in step 4 are trained as a whole using the training set generated in step 1. After the training is completed, the model parameters are saved.

[0015] Step 2 as described above includes the following steps:

[0016] The k-space reconstruction network includes multiple cascaded complex-valued convolutional layers, multiple ReLu activation layers, an upsampling module, and a k-space data consistency layer.

[0017] The upsampling module in the k-space reconstruction network includes an upsampling layer and a complex-valued convolution layer.

[0018] The number of input channels of the k-space reconstruction network is the number of cores in the multi-core k-space data.

[0019] The mapping function f(KDC) of the k-space data consistency layer is expressed by the following formula:

[0020] f(KDC)=k r ×(1-mask1)+k u

[0021] Among them, k r is the k-space data output by the upsampling module in the k-space reconstruction network, k u The multi-core low-resolution k-space data is input to the k-space reconstruction network. Mask1 is a matrix with a central area value of 1 and other values ​​of 0.

[0022] The image domain reconstruction network in step 4 as described above includes a plurality of cascaded residual modules, a plurality of complex-valued convolution modules, an upsampling module and an image domain data consistency layer.

[0023] The residual module includes multiple complex-valued convolutional layers and ReLu activation layers.

[0024] The upsampling module in the image domain reconstruction network includes an upsampling layer and a complex-valued convolution layer.

[0025] The mapping function f(IDC) of the image domain data consistency layer is expressed as follows:

[0026] f(IDC)=IFFT2(FFT2(I r )×(1-mask2)+k l )

[0027] Among them, FFT2 and IFFT2 represent two-dimensional Fourier transform and two-dimensional inverse Fourier transform respectively. r The image output by the upsampling module of the image domain reconstruction network, k l For the reconstructed multi-core k-space data in step 3, Mask2 is a matrix with the same size as the multi-core high-resolution image in step 1, with the value of the central area being 1 and the rest being 0.

[0028] The loss function in step 6 as described above is any combination of the following five parts, which are: the mean square error loss between the reconstructed multi-core k-space data in step 3 and the corresponding multi-core high-resolution k-space data in step 1; the mean absolute error loss between the super-resolution multi-core reconstructed image obtained in step 5 and the corresponding multi-core high-resolution image in step 1; the super-resolution multi-core fusion image I output by the image domain reconstruction network in step 5 A The L1 loss between the corresponding multi-core high-resolution fusion image in step 1; the super-resolution multi-core fusion image I output by the image domain reconstruction network in step 5 A The super-resolution multi-core reconstructed image I fused in step 5 B The L1 loss between the corresponding multi-core high-resolution fused image in step 1 and the super-resolution multi-core reconstructed image I fused in step 5 B The L1 loss between .

[0029] Compared with the prior art, the present invention has the following beneficial effects

[0030] The present invention directly inputs low-resolution multi-core k-space data into a convolutional neural network model (k-space reconstruction network + image domain reconstruction network) to obtain a super-resolution reconstructed image. The network uses a complex-valued convolution layer to better utilize the phase information in the k-space data. Compared with traditional super-resolution reconstruction methods, the utilization efficiency of complex-valued data is improved, which helps to improve reconstruction accuracy.

[0031] In addition, through the idea of ​​image fusion, constraining the reconstruction effect of the fused image in the network and loss function can better utilize the correlation of multi-core information, which can not only effectively improve the reconstruction accuracy, but also explore the value of multi-core data fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1It is a flow chart of the present invention.

[0033] Figure 2 It is a schematic diagram of the structure of the k-space reconstruction network.

[0034] Figure 3 It is a schematic diagram of the upsampling module structure.

[0035] Figure 4 It is a schematic diagram of the residual module structure.

[0036] Figure 5 This is the reconstruction result of the test set. DETAILED DESCRIPTION

[0037] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0038] like Figure 1 As shown, a dual-domain super-resolution reconstruction method for lung multi-core MRI based on artificial intelligence specifically includes the following steps:

[0039] Step 1: Multi-core high-resolution (resolution = 3 mm) k-space data (H, 129 Xe core) and the corresponding multi-core low-resolution (resolution = 12 mm) k-space data (H, 129 Xe cores) form sample pairs, one part of the sample pairs constitutes a training set, and the other part of the sample pairs constitutes a test set. The multi-core high-resolution k-space data (H, 129 Xe core) and multi-core low-resolution k-space data (H, 129 Xe core) can be existing historical data.

[0040] It can also be obtained by the following methods: using Flash, 3DbSSFP sequence to scan 94 volunteers to obtain multi-core high-resolution k-space data (H, 129 Xe core) and the corresponding multi-core low-resolution k-space data (H, 129 Xe core), the high-resolution k-space data matrix size is 96×96, the number of layers is 24, and the super-resolution method is 4 times higher than the low-resolution data, that is, the low-resolution k-space data matrix size is 24×24, and the sampling method is Cartesian sampling.

[0041] In this embodiment, a total of 929 multi-core high-resolution k-space data (H, 129 Xe core) and the corresponding multi-core low-resolution k-space data (H, 129 Xe core), of which 801 sample pairs are used as training sets and 128 sample pairs are used as test sets.

[0042] Afterwards, the multi-core high-resolution k-space data (H, 129 Xe core) to perform inverse Fourier transform to obtain a multi-core high-resolution image (H, 129 Xe core), and then multi-core high-resolution images (H, 129 Xe core) takes the maximum value to obtain a multi-core high-resolution fusion image (H, 129 Xe core).

[0043] Step 2: Construct a k-space reconstruction network. The schematic diagram of the k-space reconstruction network is as follows: Figure 2 As shown, the k-space reconstruction network includes 5 cascaded complex-valued convolutional layers, 4 ReLu activation layers, an upsampling module and a k-space data consistency layer.

[0044] The schematic diagram of the upsampling module structure in the k-space reconstruction network is as follows Figure 3 As shown, it includes 1 upsampling layer, 1 complex-valued convolution layer, and the upsampling multiple is 2.

[0045] The number of input channels of the k-space reconstruction network is 2, and the number of output channels is 2.

[0046] The convolution kernel size of the complex-valued convolution layer in the k-space reconstruction network is 3×3 and the number of channels is 64.

[0047] The mapping function f(KDC) of the k-space data consistency layer is expressed by the following formula:

[0048] f(KDC)=k r ×(1-mask1)+k u

[0049] Among them, k r is the k-space data output by the upsampling module in the k-space reconstruction network, k u The multi-core low-resolution k-space data (H, 129 Xe core), mask1 is a 48×48 matrix, the 24×24 values ​​in the center area are 1, and the rest are 0.

[0050] Step 3: Input the multi-core low-resolution k-space data in the training set generated in step 1 into the k-space reconstruction network constructed in step 2 to obtain the reconstructed multi-core k-space data (H, 129 Xe core).

[0051] Step 4: Construct an image domain reconstruction network. The image domain reconstruction network includes 24 cascaded residual modules, 2 complex-valued convolutional layers, an upsampling module, and an image domain data consistency layer. The complex-valued convolutional layer structure in the image domain reconstruction network is as follows: Figure 2As shown in Fig. 2, the structure of the complex-valued convolutional layer in the image domain reconstruction network is the same as that in the k-space reconstruction network, that is, the convolution kernel size of the complex-valued convolutional layer in the image domain reconstruction network is 3×3, the number of channels is 64, and the residual module structure is as follows: Figure 4 As shown in , the residual module includes 2 complex-valued convolutional layers and 1 ReLu activation layer. The size of the convolution kernel of each complex-valued convolutional layer in the residual module is 3×3, and the number of channels is 64. The schematic diagram of the upsampling module structure in the image domain reconstruction network is shown in Figure 3 As shown, it includes an upsampling layer and a complex-valued convolution layer, and the upsampling multiple is 2. The number of input channels of the image domain reconstruction network is 3, and the number of output channels is 3. The mapping function f(IDC) of the image domain data consistency layer is expressed by the following formula:

[0052] f(IDC)=IFFT2(FFT2(I r )×(1-mask2)+k l )

[0053] Among them, FFT2 and IFFT2 represent two-dimensional Fourier transform and two-dimensional inverse Fourier transform respectively. r The image output by the upsampling module of the image domain reconstruction network, k l For the reconstructed multi-core k-space data (H, 129 Xe core). Mask2 is a 96×96 matrix with 48×48 values ​​in the center area being 1 and the rest being 0.

[0054] Step 5: Reconstruct the multi-core k-space data (H, 129 Xe core) to obtain multi-core image domain data (H, 129 Xe core), and then multi-core image domain data (H, 129 Xe core) takes the maximum value and obtains the multi-core fusion data (H, 129 Xe core). The multi-core image domain data (H, 129 Xe core) and multi-core fusion data (H, 129 Xe core) is input into the image domain reconstruction network constructed in step 4 to obtain the super-resolution multi-core reconstructed image (H, 129 Xe core) and super-resolution multi-core fusion images (H, 129 Xe Core)I A Then the super-resolution multi-core reconstructed image (H, 129 Xe core) takes the maximum value for fusion to obtain the fused super-resolution multi-core reconstructed image I B .

[0055] Step 6: Set the loss function. The loss function of the network is any combination of the following five parts: the reconstructed multi-core k-space data (H, 129 Xe core) and the corresponding multi-core high-resolution k-space data (H, 129 Xe cores) between the mean square error (L2) loss; the super-resolution multi-core reconstructed image (H, 129 Xe core) and the corresponding multi-core high-resolution image (H, 129 Xe cores); the super-resolution multi-core fusion image (H, 129 Xe core)I A The multi-core high-resolution fused image (H, 129 Xe cores); the super-resolution multi-core fusion image (H, 129 Xe core)I A The super-resolution multi-core reconstructed image I fused in step 5 B L1 loss between; the corresponding multi-core high-resolution fusion image in step 1 (H, 129 Xe core) and the super-resolution multi-core reconstructed image I fused in step 5 B The loss function term can also be added, modified, weighted, etc. according to the actual situation.

[0056] Step 7. According to the loss function set in step 6, the k-space reconstruction network in step 2 and the image domain reconstruction network in step 4 are trained as a whole using the training set generated in step 1. The learning rate is initialized to 0.0001 and multiplied by 0.1 after 20 epochs. The batch size is set to 50 and the epoch is set to 500. The network is trained on the Tensorflow 2.0 platform using the ADAM optimizer. After the training is completed, the model parameters are saved.

[0057] Step 8. Input the multi-core low-resolution k-space data in the test set generated in step 1 into the k-space reconstruction network and image domain reconstruction network trained in step 7 to obtain the reconstructed image and perform quantitative evaluation, such as using evaluation indicators such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) to quantitatively analyze the reconstruction results.

[0058] Figure 5 The low-resolution k-space data (H, 129The first column is the reconstruction result of the low-resolution image (H, 129 Xe core), the second column is a high-resolution image (H, 129 Xe nuclei), and the third column is a reconstruction result diagram using the method of the present invention. It can be seen from the result diagram that the artificial intelligence-based lung multi-core MRI dual-domain super-resolution reconstruction method provided by the present invention can reconstruct high-quality magnetic resonance images from low-resolution multi-core k-space data.

[0059] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. An artificial intelligence-based dual-domain super-resolution reconstruction method for lung multi-core MRI, characterized in that: The following steps are involved: Step 1, constructing a training set and a test set, each of which includes multiple sample pairs, each of which includes multi-core high-resolution k-space data and corresponding multi-core low-resolution k-space data, performing inverse Fourier transform on the multi-core high-resolution k-space data in the training set to obtain a multi-core high-resolution image, and then taking the maximum value of the multi-core high-resolution image to obtain a multi-core high-resolution fused image; Step 2: construct a k-space reconstruction network; Step 3, inputting the multi-core low-resolution k-space data in the training set generated in step 1 into the k-space reconstruction network constructed in step 2 to obtain reconstructed multi-core k-space data; Step 4: Construct an image domain reconstruction network; Step 5: Perform a two-dimensional Fourier inverse transform on the reconstructed multi-core k-space data obtained in step 3 to obtain multi-core image domain data, then take the maximum value of the multi-core image domain data to obtain multi-core fusion data, and input the multi-core image domain data and the multi-core fusion data into the image domain reconstruction network constructed in step 4 to obtain a super-resolution multi-core reconstructed image and a super-resolution multi-core fusion image I. A , and then take the maximum value of the super-resolution multi-core reconstructed image to fuse it to obtain the fused super-resolution multi-core reconstructed image I B ; Step 6: Define the loss function; Step 7: According to the loss function set in step 6, the k-space reconstruction network in step 2 and the image domain reconstruction network in step 4 are trained as a whole using the training set generated in step 1. After the training is completed, the model parameters are saved. The step 2 comprises the following steps: The k-space reconstruction network includes multiple cascaded complex-valued convolutional layers, multiple ReLu activation layers, an upsampling module, and a k-space data consistency layer. The upsampling module in the k-space reconstruction network includes an upsampling layer and a complex-valued convolution layer. The number of input channels of the k-space reconstruction network is the number of cores in the multi-core k-space data. The mapping function f(KDC) of the k-space data consistency layer is expressed by the following formula: f(KDC)=k r ×(1-mask1)+k u Among them, k r is the k-space data output by the upsampling module in the k-space reconstruction network, k u The multi-core low-resolution k-space data is input to the k-space reconstruction network. Mask1 is a matrix with the central area value of 1 and the rest of the values ​​​​are all 0. The image domain reconstruction network in step 4 includes a plurality of cascaded residual modules, a plurality of complex-valued convolution modules, an upsampling module and an image domain data consistency layer. The residual module includes multiple complex-valued convolutional layers and ReLu activation layers. The upsampling module in the image domain reconstruction network includes an upsampling layer and a complex-valued convolution layer. The mapping function f(IDC) of the image domain data consistency layer is expressed as follows: f(IDC)=IFFT2(FFT2(I r )×(1-mask2)+kl) Among them, FFT2 and IFFT2 represent two-dimensional Fourier transform and two-dimensional inverse Fourier transform respectively. r is the image output by the upsampling module of the image domain reconstruction network, kl is the reconstructed multi-core k-space data in step 3, Mask2 is a matrix with the same size as the multi-core high-resolution image in step 1, the value in the central area is 1, and the rest of the values ​​are 0.

2. The artificial intelligence-based lung multi-core MRI dual-domain super-resolution reconstruction method according to claim 1, characterized in that: The loss function in step 6 is any combination of the following five parts, which are: the mean square error loss between the reconstructed multi-core k-space data in step 3 and the corresponding multi-core high-resolution k-space data in step 1; the mean absolute error loss between the super-resolution multi-core reconstructed image obtained in step 5 and the corresponding multi-core high-resolution image in step 1; the super-resolution multi-core fusion image I output by the image domain reconstruction network in step 5 A The L1 loss between the corresponding multi-core high-resolution fusion image in step 1; the super-resolution multi-core fusion image I output by the image domain reconstruction network in step 5 A The super-resolution multi-core reconstructed image I fused in step 5 B The L1 loss between the corresponding multi-core high-resolution fusion image in step 1 and the super-resolution multi-core reconstructed image I fused in step 5 B The L1 loss between .

Images