A MRI image denoising method based on joint optimization denoising of multiple denoisers
Through the joint optimization method of IBM3D multi-parallel noise denoiser and deep learning quality improvement noise denoiser, the details loss and computational complexity of the MRI image noise denoiser algorithm are solved, efficient image noise denoiser and detail recovery are achieved, and the model is improved convergence speed and accuracy.
Patent Information
- Application Number
- CN202011061992.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-09-30
AI Technical Summary
The existing MRI image noise reduction algorithm has problems such as loss of detailed information, unclear recovery, complex and time-consuming calculations. Deep learning networks are difficult to converge in gradient dispersion and explosion problems, resulting in poor noise processing results.
Using a joint optimization method based on IBM3D multi-parallel noise denoiser and image quality improvement noise denoiser, MRI images are segmented into small images for parallel noise reduction, and the final fusion is carried out through deep learning quality improvement noise denoiser, combining block matching and Wiener filtering algorithm, image recovery using weighted averaging and deconvolution layers.
The MRI image noise reduction efficiency and effect are improved, image details are maintained, training process is shortened, the convergence speed and accuracy of the model are improved, and initialization sensitivity is reduced.
Smart Images

Figure CN112132768B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of medical image processing, artificial intelligence and machine vision technology, and in particular to an MRI image denoising method based on joint optimization denoising of multiple denoisers. Background Art
[0002] Magnetic resonance imaging technology has the advantages of high resolution, multi-mode, and non-invasive imaging, and can display the three-dimensional structural information of the human body. Now, MRI has been widely used in scientific research and human clinical diagnosis. However, due to the limitations of the magnetic resonance imaging mechanism and the human body, the image will inevitably introduce noise, especially in high-resolution and fast imaging, the image noise will be more serious, and this noise will blur the image details, reduce the signal-to-noise ratio of the image, and affect clinical diagnosis and subsequent analysis tasks;
[0003] Some existing denoising algorithms have the disadvantages of easy loss of key detail information, unclear image restoration, high computational complexity and great time consumption. At the same time, as the network depth increases, deep learning will encounter serious problems of gradient diffusion and gradient explosion, resulting in network failure to converge. Therefore, the present invention proposes an MRI image denoising method based on multi-denoiser joint optimization denoising to solve the problems existing in the prior art. Summary of the invention
[0004] In view of the above problems, the purpose of the present invention is to propose an MRI image denoising method based on joint optimization denoising of multiple denoisers. The MRI image denoising method based on joint optimization denoising of multiple denoisers divides an MRI image into several small images, and inputs them into multiple parallel IBM3D denoisers for denoising respectively. The multiple denoised images are optimized and fused, and then the optimized and fused images are input into a quality improvement denoiser based on deep learning to obtain the final denoised MRI image. The MRI image can be denoised well, and the details of the MRI image can be restored and preserved.
[0005] To achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: a MRI image denoising method based on multi-denoiser joint optimization denoising, including IBM3D-based multi-parallel denoiser, multi-parallel IBM3D optimization denoiser, image quality improvement denoiser and model training, wherein the IBM3D-based multi-parallel denoiser is an image video denoising algorithm based on block matching and IBM3D, by searching for similar blocks and filtering in the transform domain to obtain a block evaluation value, and finally weighting each point in the image to obtain a final denoised image, wherein the IBM3D denoising algorithm has the following two steps:
[0006] Step 1: Initial Estimate
[0007] ① Block-by-block estimation, grouping each block in the noisy image, finding its overlapping similar blocks and then gathering them into a three-dimensional array;
[0008] ② Combined with hard thresholding, the three-dimensional array is filtered by a three-dimensional guided filter. The noise is reduced by hard thresholding the filtered coefficients, and then inverse filtering is performed to obtain the estimated values of all image blocks in the group, and then these estimated values are returned to their original positions in the corresponding order;
[0009] ③ Aggregate and estimate the overlapping image blocks obtained, and obtain an initial estimate of the real image by weighted averaging them;
[0010] Step 2: Final Estimate
[0011] ① Block-by-block estimation: for each block in the first step of the image, find the position of similar blocks in the basic estimated image through block matching. Through these positions, two three-dimensional arrays are obtained, one from the noisy image and the other from the image in the first step;
[0012] ② Combined with Wiener filtering, the two formed three-dimensional arrays are transformed in three dimensions, the energy spectrum in the initial estimated image is used as the energy spectrum to perform Wiener filtering on the noisy three-dimensional array, and then the inverse transformation is used to obtain the estimation of all image blocks in the group, and then these estimated values are returned to their original positions;
[0013] ③ Aggregate and estimate the overlapping local blocks, and obtain the final estimate of the real image by taking weighted average of them.
[0014] The multi-parallel IBM3D optimized denoiser is composed of a plurality of parallel IBM3D denoisers. Assume that there are n parallel IBM3D denoisers, respectively represented by D1, D2, ..., D n Indicated by d1, d2, ..., d n Indicates that:
[0015] d1=D1(x), d2=D2(x), ..., d n =D n (x) (1)
[0016] In formula (1), x represents the input image. The above output is weighted optimized to obtain the final denoised image, which is represented by y:
[0017]
[0018] In formula (2), W = [w1, w2, ..., w n ] T represents the weighted vector, D = [d1, d2, ..., d n] represents the IBM3D denoising output vector, and y is obtained by solving W.
[0019] The image quality improvement and noise reduction device includes a feature extraction layer, a nonlinear mapping layer and a deconvolution layer.
[0020] The model training uses the DATAS image dataset as the training set. First, MRI images are selected from DATAS to construct noisy image / clear image training image pairs, and then the MRI images are cropped into 51*51 size, with a momentum of 0.9. The SGD optimization method with a weight decay coefficient of 0.0001 and a decay rate of 0.9 is used. Batch normalization is used to transform the initial data distribution into a quasi-Gaussian distribution to accelerate the convergence speed.
[0021] A further improvement is that: the method for solving W is to use the minimum mean square error MSE to constrain the solution of W.
[0022] Suppose there is a noisy image / clear image sample set {x i , X i |i=1,2,…,m}, and m>n, for each input x i , and its corresponding weighted output is y i ,
[0023]
[0024] where d 1j =D1(x j ), d 2j =D2(x j ), …, d nj =D n (x j ), for all sample sets, the overall mean square error of the output is minimized, that is, the following formula (4) is minimized:
[0025]
[0026] Write equation (4) in the form of a vector
[0027] MSE=Y-DW (5)
[0028] We can use the least squares method to find W. We can differentiate the above formula so that the derivative is equal to 0. Then
[0029] W=(D T D) -1 D T Y (6)
[0030] A further improvement is that the image quality enhancement denoiser combines the respective characteristics and advantages of the enhanced deep super-resolution network and the deepened deep convolutional network to establish a deep residual convolutional denoising network.
[0031] A further improvement is that the feature extraction layer is composed of a feature convolution kernel Conv and an activation function PReLU, the feature convolution kernel is composed of 32 convolution kernels of size 3×3×c, thereby generating 32 feature maps, and then the activation function PReLU is used to perform a nonlinear mapping transformation on the feature map to form a nonlinear feature map, and the step size of the feature convolution is set to 1, and c represents the number of channels.
[0032] Further improvements are: the nonlinear mapping layer is composed of 12 residual convolution layers in series, each residual convolution layer is composed of a convolution kernel, batch normalization, an activation function and a convolution kernel, that is, it is composed of Conv+BN+PReLU+Conv, the nonlinear mapping layer is composed of 7 layers of residual convolution layers, each layer contains 32 convolution kernels of size 3×3, completes the convolution filtering of 32 extracted feature maps, and adds batch normalization BN between convolution and PReLU, and sets the convolution step size to 2.
[0033] A further improvement is that the deconvolution layer Deconv is located at the end of the network, and a group of deconvolution filters are used to perform denoising and restoration on the output 32 residual convolution images. Deconvolution can learn the restoration kernel of image features and is regarded as the inverse operation of convolution. The deconvolution layer uses a 7×7 convolution kernel and the step size is set to 1.
[0034] A further improvement is that in the model training, when the training error remains constant at a certain value within 8 consecutive epochs of training, the iterative training is stopped.
[0035] A further improvement is that the DATAS image dataset used in the model training contains 1,000 training images and 200 test images, including a wealth of MRI images of human knee joints, brain, and lungs. In the model training, 600 MRI images are selected from the DATAS image dataset to construct noise image / clear image training image pairs.
[0036] The beneficial effects of the present invention are as follows: the present invention divides an MRI image into several small images, and inputs them into multiple parallel IBM3D denoisers for denoising respectively. Under the premise of not changing the denoising effect, the algorithm efficiency and the denoising effect are greatly improved. The denoised multiple images are optimized and fused, and then the optimized and fused images are input into a quality improvement denoiser based on deep learning to obtain the final denoised MRI image. The MRI image can be denoised well, and the details of the MRI image can be restored and maintained. At the same time, the model training mechanism can accelerate the convergence speed in the training process, improve the accuracy of the model, and reduce the sensitivity of the model initialization. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 This is a principle block diagram of the multi-denoiser joint optimization denoiser of the present invention.
[0038] Figure 2 This is a flow chart of the IBM 3D noise reduction algorithm of the present invention.
[0039] Figure 3 This is a principle block diagram of the multi-parallel IBM3D optimized noise reducer of the present invention.
[0040] Figure 4 This is a principle block diagram of the image quality improvement and noise reduction device of the present invention. DETAILED DESCRIPTION
[0041] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.
[0042] according to Figure 1 , 2 As shown in , 3, and 4, this embodiment provides an MRI image denoising method based on joint optimization denoising of multiple denoisers, including a multi-parallel denoiser based on IBM3D, a multi-parallel IBM3D optimization denoiser, an image quality improvement denoiser and a model training, wherein the multi-parallel denoiser based on IBM3D is an image and video denoising algorithm based on block matching and IBM3D, which searches for similar blocks and filters them in the transform domain to obtain a block evaluation value, and finally weights each point in the image to obtain a final denoised image, wherein the IBM3D denoising algorithm has the following two steps: the first step, initial estimation
[0043] ① Block-by-block estimation, grouping each block in the noisy image, finding its overlapping similar blocks and then gathering them into a three-dimensional array;
[0044] ② Combined with hard thresholding, the three-dimensional array is filtered by a three-dimensional guided filter. The noise is reduced by hard thresholding the filtered coefficients, and then inverse filtering is performed to obtain the estimated values of all image blocks in the group, and then these estimated values are returned to their original positions in the corresponding order;
[0045] ③ Aggregate and estimate the overlapping image blocks obtained, and obtain an initial estimate of the real image by weighted averaging them;
[0046] Step 2: Final Estimate
[0047] ① Block-by-block estimation: for each block in the first step of the image, find the position of similar blocks in the basic estimated image through block matching. Through these positions, two three-dimensional arrays are obtained, one from the noisy image and the other from the image in the first step;
[0048] ② Combined with Wiener filtering, the two formed three-dimensional arrays are transformed in three dimensions, the energy spectrum in the initial estimated image is used as the energy spectrum to perform Wiener filtering on the noisy three-dimensional array, and then the inverse transformation is used to obtain the estimation of all image blocks in the group, and then these estimated values are returned to their original positions;
[0049] ③ Aggregate and estimate the overlapping local blocks, and obtain the final estimate of the real image by taking weighted average of them.
[0050] The multi-parallel IBM3D optimized denoiser is composed of a plurality of parallel IBM3D denoisers. Assume that there are n parallel IBM3D denoisers, respectively represented by D1, D2, ..., D n Indicated by d1, d2, ..., d n Indicates that:
[0051] d1=D1(x), d2=D2(x), …, d n =D n (x) (1)
[0052] In formula (1), x represents the input image. The above output is weighted optimized to obtain the final denoised image, which is represented by y:
[0053]
[0054] In formula (2), W = [w1, w2, ..., w n ] T represents the weighted vector, D = [d1, d2, ..., d n ] represents the IBM3D denoising output vector, and the minimum mean square error (MSE) is used to constrain the solution of W to obtain y;
[0055] Suppose there is a noisy image / clear image sample set {x i , X i |i=1,2,…,m}, and m>n, for each input x i , and its corresponding weighted output is y i ,
[0056]
[0057] where d 1j =D1(x j ), d 2j =D2(x j ), …, d nj =D n (x j ), for all sample sets, the overall mean square error of the output is minimized, that is, the following formula (4) is minimized:
[0058]
[0059] Write equation (4) in the form of a vector
[0060] MSE=Y-DW (5)
[0061] We can use the least squares method to find W. We can differentiate the above formula so that the derivative is equal to 0. Then
[0062] W=(D T D) -1 D T Y (6)
[0063] The image quality improvement denoiser combines the respective characteristics and advantages of the enhanced deep super-resolution network and the deepened deep convolutional network to establish a deep residual convolutional denoising network. The image quality improvement denoiser includes a feature extraction layer, a nonlinear mapping layer and a deconvolution layer.
[0064] The feature extraction layer is composed of a feature convolution kernel Conv and an activation function PReLU. The feature convolution kernel is composed of 32 convolution kernels of size 3×3×c, thereby generating 32 feature maps. Then, the activation function PReLU is used to perform a nonlinear mapping transformation on the feature map to form a nonlinear feature map. Assuming the step size of the feature convolution is 1, c represents the number of channels. The feature extraction layer of the present invention has 18 layers.
[0065] The nonlinear mapping layer is composed of 12 residual convolution layers connected in series, each residual convolution layer is composed of a convolution kernel, batch normalization, an activation function and a convolution kernel, that is, it is composed of Conv+BN+PReLU+Conv. The nonlinear mapping layer is composed of 7 residual convolution layers, each layer contains 32 convolution kernels of size 3×3, which complete the convolution filtering of 32 extracted feature maps, and the batch normalization BN is added between the convolution and PReLU to solve the problem of covariate conversion within the network. The convolution step size is set to 2.
[0066] The deconvolution layer Deconv is located at the end of the network, and uses a group of deconvolution filters to perform noise reduction and restoration on the output 32 residual convolution images. Different from the traditional interpolation method, deconvolution can learn the restoration kernel of image features and is regarded as the inverse operation of convolution. The deconvolution layer uses a 7×7 convolution kernel and the step size is set to 1.
[0067] Each layer of the deep residual convolutional denoising network uses PReLU as the activation function, and its output is:
[0068] F(x)=max(x, 0)+amin(0, x) (7)
[0069] Compared with the activation function ReLU, the activation function PReLU only increases the amount of calculation to achieve higher accuracy, and avoids the deadlock phenomenon caused by the activation function ReLU.
[0070] The idea of local residual learning and the skip connection method between different layers are adopted to form an enhanced residual convolution structure. The high-frequency features of the noise image are first extracted through the convolution layer. After that, the feature image extracted by the previous convolution layer is added after every two convolution layers. That is, the inputs of all identity branches in the residual block remain the same. It can not only transmit more image information to the deep network, but its skip branches also help the back propagation of gradients during training to avoid overfitting.
[0071] The model training uses the DATAS image dataset as the training set. First, 600 MRI images are selected from DATAS to construct noise image / clear image training image pairs, and then the MRI images are cropped into 51*51 size, the momentum is 0.9, and the SGD optimization method with a weight decay coefficient of 0.0001 and a decay rate of 0.9 is used. Batch normalization is used to convert the initial data distribution into a quasi-Gaussian distribution to accelerate the convergence speed. When the training error size is fixed at a certain value within 8 consecutive epoch training, the iterative training is stopped.
[0072] The DATAS image dataset used in model training contains 1,000 training images and 200 test images, including a wealth of MRI images of human knee joints, brain, and lungs.
[0073] The specific operation steps of the present invention are:
[0074] A training set was established and 600 MRI images were selected from DATAS to construct noisy image / clear image training image pairs. Then the MRI images were cropped into 51*51 training image pairs, with a total of 200,000 training samples.
[0075] A test set was established, 100 MRI test images were selected from the test images of the DATAS image dataset, noisy image / clear image test image pairs were constructed, and then the MRI images were cropped into test image pairs of size 51*51. A total of 10,000 test samples were collected.
[0076] Data enhancement: In order to further enrich the number of training data sets and test data sets, the samples in these two data sets are rotated and translated to enhance the data of the training set and the test set.
[0077] The model was established, and a multi-parallel IBM3D optimized denoiser model and an image quality improvement denoiser model were designed respectively. Then, these two denoisers were combined into a multi-denoiser joint optimized denoiser model for denoising and detail reconstruction of MRI images.
[0078] Model training: Use the training set images to train the denoising model, determine the parameters of the model, and establish an end-to-end learning model.
[0079] Model testing,The denoising model is tested with a test set to verify the accuracy and reliability of the proposed denoising model.
[0080] This MRI image denoising method based on joint optimization denoising of multiple denoisers divides an MRI image into several small images, and inputs them into multiple parallel IBM3D denoisers for denoising respectively. Under the premise of not changing the denoising effect, the algorithm efficiency and denoising effect are greatly improved. The denoised multiple images are optimized and fused, and then the optimized and fused images are input into a quality improvement denoiser based on deep learning to obtain the final denoised MRI image. The MRI image can be denoised well, and the details of the MRI image can be restored and maintained. At the same time, the model training mechanism can accelerate the convergence speed in the training process, improve the accuracy of the model, and reduce the sensitivity of the model initialization.
[0081] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for denoising MRI images based on joint optimization of multiple denoisers, characterized in that: The invention comprises an IBM3D-based multi-parallel denoiser, a multi-parallel IBM3D optimized denoiser, an image quality improvement denoiser and model training. The IBM3D-based multi-parallel denoiser is an image and video denoising algorithm based on block matching and IBM3D, which searches for similar blocks and filters them in the transform domain to obtain a block evaluation value, and finally weights each point in the image to obtain a final denoised image. The IBM3D denoising algorithm has the following two steps: Step 1: Initial Estimate ① Block-by-block estimation, grouping each block in the noisy image, finding its overlapping similar blocks and then gathering them into a three-dimensional array; ② Combined with hard thresholding, the three-dimensional array is filtered by a three-dimensional guided filter. The noise is reduced by hard thresholding the filtered coefficients, and then inverse filtering is performed to obtain the estimated values of all image blocks in the group, and then these estimated values are returned to their original positions in the corresponding order; ③ Aggregate and estimate the overlapping image blocks obtained, and obtain an initial estimate of the real image by weighted averaging them; Step 2: Final Estimate ① Block-by-block estimation: for each block in the first step of the image, find the position of similar blocks in the basic estimated image through block matching. Through these positions, two three-dimensional arrays are obtained, one from the noisy image and the other from the image in the first step; ② Combined with Wiener filtering, the two formed three-dimensional arrays are transformed in three dimensions, the energy spectrum in the initial estimated image is used as the energy spectrum to perform Wiener filtering on the noisy three-dimensional array, and then the inverse transformation is used to obtain the estimation of all image blocks in the group, and then these estimated values are returned to their original positions; ③ Aggregate and estimate the obtained overlapping local blocks, and obtain the final estimate of the real image by weighted averaging them; The multi-parallel IBM3D optimized denoiser is composed of a plurality of parallel IBM3D denoisers. Assume that there are n parallel IBM3D denoisers, respectively represented by D1, D2, ..., D n Indicated by d1, d2, ..., d n Indicates that: d1=D1(x), d2=D2(x), …, d n =D n (x) (1) In formula (1), x represents the input image. The above output is weighted optimized to obtain the final denoised image, which is represented by y: In formula (2), W = [w1, w2, ..., w n ] T represents the weighted vector, D = [d1, d2, ..., d n ] represents the IBM3D denoising output vector, and y is obtained by solving W; The image quality enhancement and denoising device comprises a feature extraction layer, a nonlinear mapping layer and a deconvolution layer; The model training uses the DATAS image dataset as the training set. First, MRI images are selected from DATAS to construct noisy image / clear image training image pairs, and then the MRI images are cropped into 51*51 size, with a momentum of 0.
9. The SGD optimization method with a weight decay coefficient of 0.0001 and a decay rate of 0.9 is used. Batch normalization is used to transform the initial data distribution into a quasi-Gaussian distribution to accelerate the convergence speed.
2. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The method for solving W is to use the minimum mean square error MSE to constrain the solution of W; Suppose there is a noisy image / clear image sample set {x i , X i |i=1,2,…,m}, and m>n, for each input x i , and its corresponding weighted output is y i , where d 1j =D1(x j ), d 2j =D2(x j ), …, d nj =D n (x j ), for all sample sets, the overall mean square error of the output is minimized, that is, the following formula (4) is minimized: Write equation (4) in the form of a vector MSE=Y-DW (5) We can use the least squares method to find W. We can differentiate the above formula so that the derivative is equal to 0. Then 3. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The image quality improvement denoiser combines the respective characteristics and advantages of the enhanced deep super-resolution network and the deepened deep convolutional network to establish a deep residual convolutional denoising network.
4. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The feature extraction layer is composed of a feature convolution kernel Conv and an activation function PReLU. The feature convolution kernel is composed of 32 convolution kernels of size 3×3×c, thereby generating 32 feature maps. The activation function PReLU is then used to perform a nonlinear mapping transformation on the feature map to form a nonlinear feature map. The step size of the feature convolution is set to 1, and c represents the number of channels.
5. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The nonlinear mapping layer is composed of 12 residual convolution layers connected in series, each residual convolution layer is composed of a convolution kernel, batch normalization, an activation function and a convolution kernel, that is, it is composed of Conv+BN+PReLU+Conv, and the nonlinear mapping layer is composed of 7 layers of residual convolution layers, each layer contains 32 convolution kernels of size 3×3, completes the convolution filtering of 32 extracted feature maps, and adds batch normalization BN between convolution and PReLU, and sets the convolution step size to 2.
6. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The deconvolution layer Deconv is located at the end of the network, and uses a group of deconvolution filters to perform noise reduction and restoration on the output 32 residual convolution images. Deconvolution can learn the restoration kernel of image features and is regarded as the inverse operation of convolution. The deconvolution layer uses a 7×7 convolution kernel and the step size is set to 1.
7. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: In the model training, when the training error size remains fixed at a certain value within 8 consecutive epoch trainings, the iterative training is stopped.
8. The MRI image denoising method based on joint optimization denoising of multiple denoisers according to claim 1, characterized in that: The DATAS image dataset used in the model training includes 1,000 training images and 200 test images, including abundant MRI images of human knee joints, brain, and lungs. In the model training, 600 MRI images are selected from the DATAS image dataset to construct noise image / clear image training image pairs.
