MRI motion artifact correction method based on generative adversarial network

Abstract

The invention discloses an MRI (Magnetic Resonance Imaging) motion artifact correction method based on a generative adversarial network. The method comprises the steps that firstly, a data set is collected, and motion artifact simulation is carried out; then, K space data is preliminarily corrected through a frequency domain generator, data in a space domain is finely corrected through a space domain generator, and data consistency between the K space and the space domain is kept by adopting inter-domain data consistency loss in an optimization stage; and finally, evaluating the application effect of the model by using a loss function. According to the method, artifacts or noise in the image can be effectively removed, a clearer corrected image can be obtained, and then a doctor can more accurately recognize illness state details in the sliced image of the patient.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a method for correcting MRI motion artifacts based on a generative adversarial network. Background Art

[0002] Motion artifacts have always been a common problem in magnetic resonance imaging, and their occurrence can affect doctors' diagnosis. Mild motion artifacts may only produce some noise, but severe motion artifacts may cause changes in the patient's physiological structure on the imaging picture. Of course, re-acquisition may obtain an MRI without motion artifacts, but it will also bring additional time and money costs. At present, motion artifact correction methods mainly include prospective correction and retrospective correction. Prospective correction usually requires additional equipment to obtain real-time information to correct motion artifacts, such as adjusting the parameters of magnetic resonance acquisition through feedback from navigators, adjacent acquisition sequences, or other information to maintain relative stillness between the acquisition data and the patient. In order to be compatible with the magnetic resonance acquisition process, the navigator coordinates usually need to be accurately calibrated, and the navigator sampling speed should be as fast as possible. Therefore, compared with the retrospective correction method without a navigator, the prospective correction method takes more time to generate an MRI. In addition, most prospective correction methods require the tracking device of the navigator to be installed on the patient's body, which also causes discomfort to the patient.

[0003] Retrospective correction does not require physical constraints on patients and additional examination time, so it can bring the best examination experience to patients. In order to remove motion artifacts in MRI, Loktyushin et al. proposed a method of iteratively searching motion trajectories, using spatial gradient entropy to optimize the model and generate clearer images. Cheng et al. implemented an autofocus algorithm by minimizing the local gradient entropy metric. In addition, in order to limit the size of the search space when estimating motion parameters, they also monitored the motion during data acquisition. Samsonov et al. proposed a new iterative method for motion artifact removal. This method is not limited to the construction of motion models and can correct motion artifacts well in any motion model. Jang et al. proposed a motion compensation reconstruction model based on FBP for head motion artifact removal, using iterative nonlinear optimization algorithm and L-BFGS to estimate head motion parameters, and iterative reconstruction of images based on the estimated motion parameters. It can be seen that traditional motion artifact correction methods need to iteratively search for motion trajectories or directly iteratively optimize images, which will consume a lot of computing time and computing resources, so it is difficult to apply to large-scale motion artifact correction.

[0004] In order to address the above defects, researchers have begun to focus on MRI motion artifact retrospective correction models based on deep learning. Deep learning models can achieve end-to-end training, and the test time is faster than traditional retrospective correction methods. Especially with the rapid development of generative adversarial networks in recent years, great progress has been made in the field of image restoration. However, current MRI motion artifact correction methods based on deep learning often only optimize data in K space or airspace, without considering the data consistency between K space and airspace. This makes it impossible to guarantee data consistency in different spaces after motion correction of MRI, and the reliability of the model results cannot be guaranteed. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for MRI motion artifact correction based on a generative adversarial network, the method comprising:

[0006] First, the data set is collected and motion artifact simulation is performed;

[0007] Then, the K-space data is preliminarily corrected by the frequency domain generator, and the data in the airspace is finely corrected by the airspace generator. In the optimization stage, the inter-domain data consistency loss is used to maintain the data consistency between the K-space and airspace.

[0008] Finally, the loss function is used to evaluate the application effect of the model of the present invention.

[0009] Compared with the prior art, the advantage of the present invention is that the model simultaneously corrects the motion artifacts of the data in the K-space and the airspace, and maintains the data consistency between the K-space and the airspace through the inter-domain data consistency loss. The overall framework of the model adopts a generative adversarial network, the generator adopts a convolutional neural network with skip-layer connections and residual blocks, and the discriminator adopts the Warsserstein-1 distance with a gradient penalty term for optimization. The present invention uses the public data sets ADNI1), ABIDE 2), OASIS 3) and Brain 4) to evaluate the model, and it can be seen from the following experiments that the model proposed by the present invention has certain advantages in both objective evaluation indicators and subjective evaluation.

[0010] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0012] Figure 1is a flowchart of a method for correcting MRI motion artifacts based on a generative adversarial network according to an embodiment of the present invention;

[0013] Figure 2 is an architecture diagram of an MRI motion artifact correction method based on a generative adversarial network according to an embodiment of the present invention;

[0014] Figure 3 It is a schematic diagram of the model structure of the frequency domain generator in the present invention;

[0015] Figure 4 It is a schematic diagram of the model structure of the airspace generator in the present invention; DETAILED DESCRIPTION

[0016] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0017] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0018] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0019] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0020] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0021] See also Figure 1 As shown, the motion artifact correction method for velocity encoding magnetic resonance imaging provided comprises the following steps:

[0022] Step S110 , collecting a data set and performing motion artifact simulation.

[0023] The Alzheimer's disease (AD) neuroimaging initiative (ADNI) includes MRI and PET images, which includes 4 sub-datasets, namely ADNI 1, ADNI 2, ADNI 3 and ADNI GO. The present invention selects images with a protocol description of "MP-RAGE" as the original dataset, which includes 1167 MRIs for training and 300 MRIs for testing.

[0024] Autism Brain Imaging Data Exchange (ABIDE) is an autism brain imaging dataset, mainly used to study the early diagnosis of autism in children, which includes 539 ASD cases and 573 normal controls. Through observation of the dataset and numerical comparison of image quality control indicators, the present invention selects images without motion artifacts from the total dataset as training and test sets.

[0025] Open Access Imaging Studies (OASIS) is a project that aims to make brain neuroimaging datasets freely available to the scientific community. It has been used for hypothesis-driven data analysis, the development of neuroanatomical atlases, and the development of segmentation algorithms. It contains 3 different parts to apply to different tasks. OASIS-1 consists of 416 MRI images of subjects aged between 18 and 96 years old. For each subject, there are 3 to 4 separate T1-weighted MRI scans obtained in a single scan in the dataset. The subjects are all right-handed, including men and women, and 100 of the subjects aged over 60 were clinically diagnosed with very mild to moderate Alzheimer's disease.

[0026] Brain is an open brain dataset. The dataset contains data on normal brain, cerebrovascular diseases, tumor diseases, degenerative diseases, inflammation and infectious diseases. Each data is a pair of PET, SPECT, MRI and CT scan images, and each image has multiple slices. This paper uses T1-MRI slice data of 46 patients.

[0027] Since it is difficult to collect motion artifact data of medical images, the present invention uses the method proposed by Medley et al. to simulate motion artifacts in MRI by destroying the K-space data corresponding to the image. The motion artifacts generated by this method are more common in clinical testing. The principle is as follows:

[0028]

[0029] Where s(k x ,k y ) is the original K-space data without motion artifacts, s'(k x ,k y ) is the K space data with motion artifacts after the K space is destroyed.x and k y Respectively represent the K-space coordinates in different phase encoding directions. x ,k y ) is as follows:

[0030] φ(k x , k y )=2π[k x p(k y )+k y q(k y )]

[0031] Among them, p(k y ) and q(k y ) represent the motion functions in different phase shift directions.

[0032] The simulated motion artifact image is obtained by the parameters in the following formula:

[0033]

[0034] In step S120, the K-space data is preliminarily corrected by the frequency domain generator, and the data in the spatial domain is finely corrected by the spatial domain generator. In the optimization stage, the inter-domain data consistency loss is used to maintain the data consistency between the K-space and the spatial domain.

[0035] For the frequency domain generator Gf, the present invention adopts the same network structure as DeepcomplexMRI. Figure 3 shown.

[0036] The complex convolution network Gf introduced in the present invention is a convolutional neural network using a complex convolution kernel, and its input is the reconstructed K-space data with motion artifacts. The output is the reconstructed K-space data after removing motion artifacts. The conversion process is as follows:

[0037]

[0038] where f ma =T(I ma ), and Represents f ma The real and imaginary parts of the matrix are concatenated together, and concate(·) concatenates the real and imaginary parts of the matrix along the last dimension.

[0039] The process of complex convolution kernel convolution is as follows:

[0040]

[0041] Where K = K real +jK imag is the complex convolution kernel, is the input of the first convolution layer of the frequency domain generator. Through this complex convolution method, the correlation between the real and imaginary parts of the K space is preliminarily guaranteed. At the same time, because the complex convolution network does not actually convolve the complex matrix, but convolve the real matrix reconstructed from the complex matrix, the neural network may not be able to accurately learn the relationship between the complex part and the imaginary part. Therefore, the input of the complex convolution network After each cascade block, a residual connection is made with the output of the current block to strengthen the correlation between the real and imaginary parts. Each cascade block consists of a complex convolution network and a data consistency layer. The complex convolution network includes 5 layers of complex convolution layers, each layer uses a 3×3 complex convolution kernel, and except for the last layer, all layers are activated using the ReLU activation function. The number of output channels of each layer is 64. In the entire convolution network, the width and height of the feature map are consistent with the width and height of the input data. The data consistency layer uses a residual connection to ensure data consistency between the reconstructed K-space data and the K-space data before reconstruction.

[0042] For the spatial domain generator G s ,like Figure 4 As shown in the figure, in addition to the first convolution layer and the last convolution layer, it also includes two downsampling modules, two upsampling modules and 9 residual modules. For each downsampling module, it includes a convolution layer, an instance normalization layer and a ReLU activation function. For the upsampling module, except that the transposed convolution layer is used instead of the convolution layer, the other structures are the same as the downsampling module. The downsampling module and the upsampling module are symmetrical in network structure. The upsampling module splices the high-level features and the low-level features of the downsampling module in the last dimension of the data through the skip layer connection. Unlike U-Net, this model does not crop the low-level features, but directly splices them with the high-level features. Among them, the residual module is used to better extract the high-level features of the image without changing the dimension of the features, and each residual module consists of two convolution layers, two instance normalization layers and a ReLU activation function. Global residual connections are also used in the input and output layers of the network, which has achieved good results in image restoration tasks.

[0043] Finally, in order to keep the frequency domain and spatial domain data consistent across domains, the present invention introduces the DomainTransform layer, which is used to implement the G f and G s The cascade network is composed of two parts, and the data consistency between K-space and spatial domain is maintained through inter-domain consistency loss.

[0044] Step S130, finally using the loss function to evaluate the application effect of the model of the present invention.

[0045] During the training process, the loss function of the dual cascade generator proposed in the present invention is as follows:

[0046]

[0047] Among them, L G is the overall loss of the generator, including pixel-level loss and adversarial loss, where pixel-level loss mainly includes mean square error loss (MSE), mean absolute error loss (MAE) and perceptual loss. The adversarial loss is consistent with the adversarial loss proposed in WGAN-GP. In the experiments of this patent, the hyperparameters α, β, γ, λ, and μ are set to 1, 1, and 1×10 respectively. -6 , 1 and 1×10 -3 .

[0048] The adversarial loss is defined as follows:

[0049]

[0050] in represents the original MRI image input to the generator, G and D represent the overall generator and discriminator, respectively.

[0051] The perceptual loss is defined as follows:

[0052]

[0053] Where N represents the number of images. is the label image without motion artifacts, C i , W i and H i are the number of channels, width, and height of the i-th layer output feature map of VGGM, respectively. is the i-th layer of VGGM and all the previous network layers, x, y and z represent the coordinate points in the feature map. VGGM is a pre-trained model based on VGG-19 that is only used for feature extraction.

[0054] Represents the reconstruction loss of the frequency domain generator, which is calculated using the mean absolute error between the input and output. represents the reconstruction loss of the spatial domain generator. and The definition of is as follows:

[0055]

[0056]

[0057] Among them, the loss of the spatial domain generator The calculation is to generate a de-artifacted image and the true label image The mean square error of pixels between .

[0058] In order to maintain the data consistency between frequency domain data and spatial domain data, the present invention introduces the inter-domain consistency loss L DC The inter-domain consistency loss is defined as follows:

[0059]

[0060] in It is the K-space data repaired by the frequency domain generator.

[0061] The local optimization loss used in the present invention is constructed between all spatial domain data. This loss can constrain the model to maintain the local data consistency between the generated samples and the real samples. It is defined as follows:

[0062]

[0063] The kernel is simply designed as a 3×3 matrix with a value of 1.

[0064] In order to more intuitively and accurately measure the specific performance of different models, the present invention uses objective evaluation indicators PSNR, SSIM and RMSE to evaluate and analyze each model. The experiment trained and tested all models 10 times under the same conditions, and calculated the average and standard deviation of the objective evaluation indicators. The results of objective evaluation indicators are shown in Tables 1 and 2. The test results of Pix2pix, SRN, DC-MRI, FSGAN-base and FSGAN on ADNI, ABIDE, OASIS and Brain are compared in the table. It can be seen that FSGAN has achieved the best results in the three evaluation indicators on the four data sets, and DC-MRI has achieved suboptimal results. In order to more intuitively compare the improvement of FSGAN compared to the suboptimal results, Table 3 compares the improvement rate of FSGAN compared to the suboptimal model DC-MRI in detail. It can be seen that FSGAN has improved by 4.25%, 3.62% and 13.88% respectively in PSNR, SSIM and RMSE compared to the current optimal method DC-MRI. Especially in the experiment on the ADNI data set, the performance improvement of FSGAN is more obvious.

[0065] Table 1 Objective evaluation index results of different models on ADNI and ABIDE

[0066] Table 1 Results on objective evaluation metrics of different inodclson ADNI and ABIDE

[0067]

[0068] Table 2 Objective evaluation index results of different models on OASIS and Brain

[0069] Table 2 Results on objective evaluation metrics of different models OASIS and Brain

[0070]

[0071] Table 3 Performance improvement comparison of FSGAN compared with DC-MRI

[0072] Table 3 Comparison of performance between FSGAN and DC-MRI

[0073]

[0074] Through the evaluation of objective evaluation indicators, it can be determined that the FSGAN proposed in the present invention does ensure the similarity between the corrected MRI and the original image, so that it has more realistic texture and structural information. In summary, this patent proposes a cascade generator adversarial network, which performs motion artifact correction on K-space data and spatial domain data through frequency domain generator and spatial domain generator respectively, and maintains data consistency between K-space and spatial domain through inter-domain consistency loss, thereby achieving higher-precision MRI motion artifact correction. Experimental results show that the method proposed in this patent has higher image observation quality and objective evaluation indicators than the current optimal method.

[0075] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0076] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0077] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0078] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0079] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0082] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0083] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used in this patent is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed in this patent. The scope of the present invention is defined by the appended claims.

Claims

1. A method for MRI motion artifact correction based on a generative adversarial network, comprising the following steps: First, the data set is collected and motion artifact simulation is performed; Then, the K-space data is preliminarily corrected by the frequency domain generator, and the data in the airspace is finely corrected by the airspace generator. In the optimization stage, the inter-domain data consistency loss is used to maintain the data consistency between the K-space and airspace. Finally, the loss function is used to evaluate the application effect of the model of the present invention.

2. The method according to claim 1, characterized in that The classification model is trained according to the following steps: To ensure the fairness and reliability of the experiment, the same initialization scheme was used 10 times in each experiment, and the average value was obtained after removing the sensitive values. Due to the different sizes of different data sets, their sizes were uniformly cropped to 256×256 before training. Instance normalization is used after the discriminator convolution layer, so the batch sample number input of the model must be 1. There are 9 residual modules after the downsampling module. The input and output of each residual module always keep the same dimension. Each residual module includes two convolution layers, two instance normalization layers and 1 ReLU activation function.

3. The method according to claim 1, characterized in that The classification model mainly consists of three parts: the generator, the discriminator and the feature extractor VGGM. The generator adopts a cascaded generator structure, which consists of the frequency domain generator G f , Spatial Generator G s And the domain conversion layer DT layer, the complex convolutional network G f Preliminary correction of motion artifacts for K-space data, deep skip-layer residual network G s The spatial data is finely corrected for motion artifacts. Together, the two achieve cross-domain motion artifact correction from coarse to fine. The discriminator uses the discriminator structure in WGAN-GP, and VGGM is VGG-19 pre-trained using the MRI dataset.

4. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

5. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

Images