A method and system for delay-enhanced magnetic resonance image noise reduction and reconstruction

Through the combination of DnCNN and SRGAN-WGAN, the inefficiency and noise training model limitation in DE-MRI image noise reduction and super-resolution reconstruction are solved. Gradient punishment and WGAN loss function are used to achieve efficient image reconstruction and diagnosis Improved accuracy.

CN114677310BActive Publication Date: 2024-12-03CENT SOUTH UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210396331.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-15
Publication Date
2024-12-03
Estimated Expiration
2042-04-15

AI Technical Summary

Technical Problem

The prior art has low efficiency in DE-MRI image noise reduction and super-resolution reconstruction, limited noise training model, gradient disappearance problems, etc., which affect the accuracy of myocardial disease diagnosis.

Method used

DnCNN is used for image preprocessing, combined with SRGAN and WGAN generators and discriminators to fight each other, improve image reconstruction accuracy through gradient punishment and WGAN loss function, accelerate calculation using BN layer and residual learning, optimize the loss function to enhance the model ability to transform.

Benefits of technology

Significantly improves image reconstruction quality, reduces training time, and generates high-resolution images closer to real images, improving diagnostic accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114677310B_ABST
    Figure CN114677310B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for denoising and reconstructing delayed enhanced magnetic resonance images, comprising the following steps: using DnCNN to preprocess DE-MRI images; using the generator and discriminator of SRGAN to confront each other, alternately training to generate images with better quality, and using gradient penalty to solve the problem of gradient vanishing; adding a WGAN loss function on the basis of the original SRGAN loss function to monitor gradient descent and improve the accuracy of image reconstruction. The present invention first performs DnCNN denoising on DE-MRI images to obtain images with high peak signal-to-noise ratio as SRGAN training images, adds a DE-CMR training set, and enhances the generalization ability of the model; secondly, by minimizing the Wasserstein distance and using gradient penalty, the distribution of the generated image is close to that of the real image, avoiding the vanishing and explosion of training, and speeding up the training speed of the model; finally, adding the WGAN loss function improves the accuracy of image reconstruction, so that the image reconstruction quality is significantly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of slope engineering safety assessment, and in particular to a delayed enhancement magnetic resonance image noise reduction and reconstruction method and system. Background Art

[0002] Delayed-enhanced Magnetic Resonance Image (DE-MRI) technology is an effective technology for analyzing myocardial diseases, such as myocardial fibrosis, myocardial infarction, etc. In DE-MRI images, normal myocardial signals are displayed as dark signals, while myocardial fibrosis tissue and necrotic myocardial tissue are displayed as bright signals to generate DE-MRI. Doctors can obtain the decision-making information required for the diagnosis and treatment of cardiac diseases and their pathological analysis through DE-MRI. However, due to the accuracy limitations of imaging equipment, difficulty in patient cooperation, and operator errors, low-quality DE-MRI is inevitably produced, with low resolution, blurred heart, etc. Therefore, the automation of DE-MRI reconstruction is of great clinical significance.

[0003] At present, there are a large number of algorithms applied to image denoising, such as non-local self-similarity, sparse representation, Markov random field and other methods. Non-local self-similarity refers to image blocks with similar structures (generally measured by Euclidean distance) to the current block within a certain range. Non-local similar blocks are not necessarily adjacent to or overlapped with the current block, and are obtained by searching for certain criteria. However, the above methods have problems such as difficulty in optimization and long time consumption. Chen et al. proposed a trainable nonlinear reaction diffusion model, which is a feedforward deep network with a fixed number of gradient descent inference steps. With the development of machine learning and deep learning, multilayer perceptron (MLP) has been successfully applied to image denoising. Yang et al. used a CT image denoising method with Wasserstein distance and perceptual similarity based on generative adversarial network (GAN). In the established feature space, the perceptual similarity loss is compared with the perceptual features of the denoised output, and the perceptual features of the real image are used to suppress noise. The disadvantages of the above methods that cannot be ignored are that they train specific models for specific noise levels and are limited in blind image denoising.

[0004] Super-resolution is to improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image from a low-resolution image through a series of operations is called super-resolution reconstruction. In the actual DE-MRI image diagnosis process, low-resolution images have poor quality and too few texture details, which affects the accuracy of myocardial disease diagnosis. Therefore, in the DE-MRI images of contrast agents, it is necessary to convert low-quality DE-MRI heart images into high-quality, high-resolution DE-MRI images. Harris, Goodman and others proposed super-resolution, believing that there is a relevant mapping relationship between low-resolution images and high-resolution images. If a large number of images can be trained through a deep learning model to learn these mapping relationships, then the use of low-resolution images can reconstruct true high-resolution images. Dong et al. used a deep learning model to solve the super-resolution problem, and used a three-layer convolutional neural network (CNN) to learn the mapping relationship between low-resolution images and high-resolution images, and added mean square error as a loss function to obtain high-quality images.

[0005] At the current stage, there are many developments in image denoising and image super-resolution. Some methods use machine learning to reduce noise, while others use traditional algorithms, such as using opencv to complete the noise reduction operation. However, most of these algorithms infer the color of the current point through adjacent stages. If a node is surrounded by blue and the node itself is white, it is likely to be a noise point and will be removed and turned blue. Viren Jain et al. proposed using CNN to deal with the denoising problem of natural images, and obtained results that are similar to or better than conventional methods (such as wavelet transform and Markov random field). They also explained that a specific form of CNN can be regarded as an approximation of the result of the Markov model inference of image denoising, and can avoid the computational difficulties of the Markov model in the probability learning and inference process, reducing the computational complexity.

[0006] In super-resolution, adversarial neural networks are used to complete training, but the original GAN ​​training is unstable and it is easy to generate features that do not exist in the generated image, resulting in a decrease in image reconstruction accuracy. This problem is an open and challenging problem that GAN has always faced. Arjovsky et al. proposed WGAN to solve the instability problem in the original GAN ​​training, using a near-optimal discriminator to optimize the generator, reduce the Wasserstein distance, and make the distribution of generated images tend to the distribution of real images. Gulrajani et al. found that WGAN has defects, such as only generating low-quality samples and difficulty in model convergence. Summary of the invention

[0007] The purpose of the present invention is to provide an improved method and system for denoising and reconstructing DE-MRI images in view of the deficiencies in the above-mentioned background technology, so as to convert blurry or low-resolution DE-MRI images into high-resolution DE-MRI images for use in patient treatment and doctor diagnosis.

[0008] In order to achieve the above object, the present invention provides a method for denoising and reconstructing a delayed enhanced magnetic resonance image, comprising the following steps:

[0009] S1, DE-MRI image preprocessing using DnCNN;

[0010] S2, uses the generator and discriminator of SRGAN to compete with each other, alternately trains to generate images with better quality, and uses gradient penalty to solve the problem of gradient disappearance;

[0011] S3, based on the original SRGAN loss function, adds the WGAN loss function to monitor the gradient descent and improve the accuracy of image reconstruction.

[0012] Furthermore, the network structure of DnCNN in S1 is a cascade structure, which uses BN layer and residual learning to improve the computing performance.

[0013] Furthermore, in S2, a gradient penalty is added to the discriminator and the generator is optimized using the Adam optimizer.

[0014] Furthermore, the total loss function in S3 is as follows:

[0015]

[0016] Among them, C-MSE is the MSE loss of high-resolution DE-MRI image and low-resolution DE-MRI image, λ 1 and λ 1 is the proportional coefficient, and the C-MSE loss function is as follows:

[0017]

[0018] Among them, W and H are the sizes of the image, D represents the discriminator, and G represents the generator. It represents a low-resolution DE-MRI image. Represents the high-resolution DE-MRI image after noise reduction;

[0019] The equation of the VGG loss function is as follows:

[0020]

[0021] Among them, φ i,jIt is the feature obtained by the jth convolution and i-th pooling layer;

[0022] The equation of the WGAN loss function is as follows:

[0023]

[0024] in, Represents the real image.

[0025] The present invention also provides a denoising and reconstruction system for delayed enhanced magnetic resonance images, comprising an image preprocessing module and an image reconstruction module. The image preprocessing module is used to perform denoising on real images. The image reconstruction module relies on the generator and discriminator of SRGAN to confront each other and alternately train to generate images with better quality. The image reconstruction module uses gradient penalty to solve the problem of gradient disappearance, and monitors gradient descent through the WGAN loss function to improve the accuracy of image reconstruction.

[0026] The above scheme of the present invention has the following beneficial effects:

[0027] The present invention firstly performs DnCNN denoising on DE-MRI images to obtain images with high peak signal-to-noise ratio, which are used as SRGAN training images, and DE-CMR training sets are added to enhance the generalization ability of the model. Secondly, by minimizing the Wasserstein distance and using gradient penalty, the distribution of the generated images is made close to that of the real images, which avoids the disappearance and explosion of training and speeds up the training of the model. Finally, the WGAN loss function is added to improve the accuracy of image reconstruction, so that the image reconstruction quality is significantly improved.

[0028] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a system flow chart of the present invention;

[0030] Figure 2 This is a comparison chart of the weight clipping and gradient penalty results of the present invention;

[0031] Figure 3 This is a comparison chart of the results of adding a BN layer and not adding a BN layer in the present invention;

[0032] Figure 4 This is a comparison chart of the results of fusion verification of the MSE loss function, VGG loss function and WGAN loss function in the present invention;

[0033] Figure 5 This is a comparison chart of the results of the present invention and conventional super-resolution methods such as BICUBIC, SRGAN, and ESRGAN. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] For simplicity of explanation, the method or rule is depicted or described as a series of operations, and its purpose is neither to exhaustively list the experimental operations nor to limit the order of the experimental operations. For example, the experimental operations can be performed in various orders and / or simultaneously, and include other experimental operations that are not described again. In addition, the steps described are not all necessary for the methods and algorithms described herein. These methods and algorithms can be represented as a series of unrelated states through state diagrams or projects.

[0036] Delayed-enhanced Magnetic Resonance Image (DE-MRI) technology is an effective technology for analyzing myocardial structure. In DE-MRI images, normal myocardial signals are displayed as dark signals, while myocardial fibrosis and necrotic myocardial tissue are displayed as bright signals to generate DE-MRI. Doctors can obtain decision-making information required for the diagnosis and treatment of cardiac diseases and their pathological analysis through DE-MRI. However, due to the accuracy limitations of imaging equipment, difficulty in patient cooperation, and operator errors, low-quality DE-MRI is inevitably produced, with low resolution, blurred heart, etc. Nowadays, medical resources are very scarce, and DE-MRI images are conducive to the diagnosis of myocardial diseases in patients. Since it takes 30 minutes to an hour to shoot DE-MRI images, it is a great waste if the DE-MRI images are blurred due to respiratory movement and the like and discarded. Therefore, the present invention proposes a method for denoising and reconstructing DE-MRI images, which converts blurred or low-resolution DE-MRI images into high-resolution DE-MRI images for the treatment of patients and the diagnosis of doctors. The results show that this method can effectively improve the reconstruction effect.

[0037] This method is based on DnCNN (feedforward denoising convolutional neural network) and SRGAN structure, specifically, includes the following steps:

[0038] S1, DE-MRI images are preprocessed using DnCNN to ensure that the input images are clean.

[0039] Among them, DnCNN is improved on the basis of VGG. The network structure of DnCNN is a cascade structure of (convolution, BN, ReLU). Unlike ResNet, there is no long jump connection inside the model, but residual learning is used at the output of the network. The goal of image denoising is to remove the polluted information superimposed on the original clean image and reconstruct a possible clean image. The main task of the model is to obtain a clear image x based on the noisy image y. However, it does not directly output x, but outputs the noisy image v. The formula for the final clear image is x=yR(y). Here R is the model, R(y)≈v, which is called residual learning.

[0040] In order to solve the gradient dispersion effect caused by the deepening of the network layer, DnCNN does not directly learn the noisy image, but uses the l2 norm of the output image and the noisy image as the loss function to train the network. DnCNN is considered to be a residual learning process. The network uses BN layer and residual learning together to improve the performance of the algorithm, modeling the image and achieving denoising in the absence of specific noise.

[0041] During model training, the rule used to judge image loss is the difference between the output error map and the true error map. The loss function is defined as:

[0042]

[0043] Where θ is the network parameter of DnCNN, and the information of the training set is stored in it. The model can use SDG or Adam method to update parameters.

[0044] S2, relying on the generator and discriminator of the adversarial neural network (SRGAN) to compete with each other, alternately trains to generate images with better quality, and uses the gradient penalty (GP) method to solve the problem of the vanishing gradient of the discriminator, thereby improving the convergence speed of the model.

[0045] Among them, the generator network G is responsible for generating generated images that are close to real images, and the discriminator network D is responsible for distinguishing the generated images generated by the generator network from real images. Ideally, when the discriminator network identifies the generated images as real images, the network model is optimal. However, the distribution of images generated by the original GAN ​​generator is random, which can easily make the gradient of the generator disappear. To solve this problem, this method uses the WGAN algorithm to minimize the Wasserstein distance so that the distribution of generated images is infinitely close to that of real images.

[0046] During the training process, the WGAN algorithm converges slowly. Therefore, the gradient penalty WGAN (WGAN-GP) speeds up the convergence of WGAN by adding GP to the discriminator and using the Adam optimizer to optimize the generator. Among them, WGAN-GP adds a regularization term compared to WGAN. This regularization term is the GP in WGAN-GP, that is, the gradient constraint.

[0047] S3, adds a new loss function based on the original SRGAN loss function to monitor the GAN gradient descent, thereby achieving more stable and efficient model training and providing higher perceptual quality for the super-resolution of DE-MRI images.

[0048] In order to generate DE-MRI images with better high-frequency details, the sum of the C-MSE loss function and the VGG loss function is used as the content loss function of the adversarial neural network, and the WGAN loss function is used to solve the problem of gradient disappearance during training. The total loss function is as follows:

[0049]

[0050] In equation (1), C-MSE is the MSE loss between high-resolution DE-MRI images and low-resolution DE-MRI images. 1 and λ 1 After repeated tests, different results are obtained. The best result is found from different results to obtain the optimal λ 1 and λ 2 Preferably, λ 1 Take 0.01, λ 2 Take 0.001.

[0051]

[0052] In equation (2), W and H are the sizes of the image, D represents the discriminator in the adversarial neural network, and G represents the generator in the adversarial neural network. It represents a low-resolution DE-MRI image. Represents a high-resolution DE-MRI image after noise reduction.

[0053] Applying the VGG network can enhance the details of the images generated by the generator and improve the quality of the generated DE-MRI images. The equation of the VGG loss function is:

[0054]

[0055] In equation (3), φ i,jIt is the feature obtained by the jth convolution and i-th pooling layer.

[0056] In order to deal with the problem of the vanishing loss function of the GAN generator, the Wasserstein distance is used to optimize the generator. The adversarial loss of the generator is defined as the probability that the discriminator recognizes the generated high-resolution DE-MRI image as the original high-resolution image. The equation is as follows:

[0057]

[0058] In equation (4), Represents a real image.

[0059] The method in this embodiment is implemented by Python 3.6, TensorFlow 1.6.0 and Pytorch 0.4.1. The graphics processor uses NVIDIA GTX 1080T. The Adam optimizer is used to adjust the GAN network, where the parameter β=0.9, the batch normalization size is 32, the learning rate of the residual network is 10-4, the learning rate decay is 0.1, and the Wasserstein gradient suppression method is used to supervise the gradient descent to avoid gradient disappearance and explosion. The number of training epochs is changed from 10,000 in the past to 1,500. Since it is found that the network begins to converge after 1,500 epochs, unnecessary training time is abandoned, the amount of network calculation is reduced, and the speed of network reconstruction is accelerated. The model training time is about 12h and 10min, and the test time is about 8min and 25s.

[0060] Figure 2 This method verifies the solution of adding the method to prevent the gradient from disappearing, and compares the weight clipping and gradient penalty solutions. Figure 2 It can be seen that the gradient penalty has a better effect.

[0061] In the denoising neural network, the optimization is performed using the BN layer for optimization experiments. In this embodiment, the BN layer is added to the DnCNN in each ResBlock. Figure 3 It can be seen that adding the BN layer can significantly improve the quality of super-resolution reconstruction, which verifies the correctness of adding the BN layer.

[0062] The original GAN ​​uses MSE and VGG as the generator loss function. This method combines WGAN with the former two as the loss function of SRGAN. The MSE loss function, VGG loss function and WGAN loss function are fused and verified respectively. In computer vision, numerical evaluation criteria such as PSNR and SSIM are usually used to verify the quality of the reconstructed image. Figure 4As shown in Figure 1, the MSE loss function, VGG loss function and WGAN loss function were ablated and verified one by one, where a is the only MSE loss function, b is the only VGG loss function, c is the only WGAN loss function, d is the combination of VGG and WGAN, e is the combination of MSE and WGAN, f is the combination of VGG and MSE, g is the combination of MSE, VGG and WGAN, V Avg-PSNR 、V Avg-SSIM are the values ​​of Avg-PSNR and Avg-SSIM, respectively. The results show that the SRGAN model combined with the MSE loss function, VGG loss function, and WGAN loss function can effectively improve the PSNR and SSIM scores of the generated images.

[0063] In order to further verify the DnSRGAN model of this method, the test images were divided into 12 groups, each with 25 time series frames. Representative images were selected to show the intuitive visual experience of this method (DnSRGAN) and conventional super-resolution methods such as bicubic, SRGAN, and ESRGAN. It can be clearly seen that the DE-MRI images generated by this method are closer to the real images than the real images generated by other methods. From the values ​​in Table 1, it can be seen that the Avg-PSNR and Avg-SSIM of this method are significantly higher than those of the other three methods. From the perspective of average PSNR / SSIM, this method is better than other methods. Figure 5 The histogram in more clearly demonstrates the advantages of this method, indicating that this method can better reconstruct DE-MRI images of complex structures.

[0064] Table 1: Comparison of Avg-PSNR and Avg-SSIM values

[0065]

[0066] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for denoising and reconstructing delayed enhancement magnetic resonance images, characterized in that: The steps include: S1, DE-MRI image preprocessing using DnCNN; S2, uses the generator and discriminator of SRGAN to compete with each other, alternately trains to generate images with better quality, and uses gradient penalty to solve the problem of gradient disappearance; S3, adds WGAN loss function to monitor gradient descent based on the original SRGAN loss function to improve the accuracy of image reconstruction; The network structure of DnCNN in S1 is a cascade structure, which uses BN layer and residual learning to improve the computing performance, model the image and achieve denoising in the absence of specific noise; During model training, the rule used to judge image loss is the difference between the output error map and the true error map; for training data The loss function is defined as: Where θ is the network parameter of DnCNN, in which the information of the training set is stored, and the model uses SDG or Adam method to update the parameters; In S2, a gradient penalty is added to the discriminator and the generator is optimized using the Adam optimizer. The total loss function in S3 is as follows: Among them, C-MSE is the MSE loss of high-resolution DE-MRI images and low-resolution DE-MRI images, λ1 and λ1 are proportional coefficients, and the C-MSE loss function is as follows: Among them, W and H are the sizes of the image, D represents the discriminator, and G represents the generator. It represents a low-resolution DE-MRI image. Represents the high-resolution DE-MRI image after noise reduction; The equation of the VGG loss function is as follows: Among them, φ i,j It is the feature obtained by the jth convolution and i-th pooling layer; The equation of the WGAN loss function is as follows: in, Represents the real image.

2. A system for denoising and reconstructing delayed enhancement magnetic resonance images, using the method for denoising and reconstructing delayed enhancement magnetic resonance images as claimed in claim 1, characterized in that: The method comprises an image preprocessing module and an image reconstruction module. The image preprocessing module is used to perform noise reduction on real images. The image reconstruction module relies on the generator and discriminator of SRGAN to confront each other and alternately train to generate images with better quality. The image reconstruction module adopts gradient penalty to solve the problem of gradient disappearance and monitors gradient descent through the WGAN loss function to improve the accuracy of image reconstruction.

Citation Information

Patent Citations

  • Single image super-resolution reconstruction method based on conditional generative adversarial network

    CN110136063A

  • Super resolution using a generative adversarial network

    US20180075581A1