A method for image denoising based on noise level estimation

Abstract

The present invention discloses a method for image denoising based on noise level estimation, which is based on feature dimension reduction and convolutional neural network. The noise level estimation module of the present invention has the advantages of accurate estimation and high computational efficiency. Noise reduction is performed after the noise level is estimated in advance, which overcomes the problem that the commonly used FFDNet and other models are heavily dependent on prior parameters, and replaces the continuous convolution part of the FFDNet model with a densely connected block, which greatly improves the feature extraction ability of the model. The present invention can quickly and efficiently complete image denoising and has a wide range of application value.

Description

Technical Field

[0001] The invention belongs to the technical field of computer image processing, and in particular relates to an image denoising method based on noise level estimation. Background Art

[0002] When collecting images, image noise is often generated, which will have an adverse effect on subsequent image processing, such as image segmentation, image classification, etc. With the development of computer image technology, people have carried out a lot of research in the field of image denoising. Dabov et al. (Image denoising by sparse 3-D transform-domain collaborative filtering) proposed a three-dimensional block matching (BM3D) algorithm, which matches similar images in the spatial domain, obtains the change relationship between images in the frequency domain, and finally uses this information to perform denoising in the transform domain, but the computational complexity of the algorithm is high. Zhang et al. (Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising) incorporates the idea of ​​residual learning into the deep convolutional network and proposes the DnCNN denoising model. By learning the distribution of image noise, the noise signal is removed and a good denoising effect is achieved. However, since the convolution process of the DnCNN network does not change the image scale, the model parameters are too large and the execution efficiency is low. In view of this, Zheng et al. (Toward afast and flexible solution for CNN-based image denoising[J].IEEE Transactions on Image Processing) proposed the FFDnet denoising model, which adds a noise level map (NLM) as an auxiliary input to the input of the DnCNN network, and the network performs downsampling and upsampling at the input and output stages respectively. Compared with the DnCNN model, it not only has better denoising ability, but also has higher execution efficiency. However, as a non-blind denoising model, its denoising effect still depends on the accuracy of the input noise level. Therefore, in order to give full play to the denoising performance of this type of non-blind denoising network, it is necessary to estimate the noise level of the noisy image.

[0003] The traditional noise level estimation (NLE) method is to separate the noise signal from the noisy image, and then estimate the noise level based on the characteristics of the noise signal. For example, Liu et al. (Single-image noise level estimation for blind denoising) proposed a two-stage NLE method based on singular value decomposition (SVD). The algorithm first uses SVD to make a rough estimate of the noise level, and then analyzes the changes in the singular values ​​of the known noise level image to correct the rough estimate. The method has excellent noise level estimation ability, but the amount of calculation is very large. To solve this problem, Yu et al. (AFast Noise Level Estimation Algorithm Based on Convolutional Neural Network) used a deep learning method to directly train noisy images, and input the noisy image into the convolutional neural network (CNN) to map the corresponding noise level value. The algorithm can directly estimate the noise level end-to-end, with high execution efficiency, but the prediction accuracy is low. Summary of the invention

[0004] The present invention mainly overcomes the deficiencies in the prior art and aims to provide an image denoising method based on noise level estimation.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] Step 1: Preprocess the original image data. The specific processing method is as follows:

[0007] (1) Add Gaussian noise of different noise levels to the image and resize the image to n×n pixel size;

[0008] (2) Divide the preprocessed image dataset into a training set and a test set, using the original data without noise as the training target;

[0009] Step 2: Construct a noise level estimation network module. The entire noise level estimation network consists of principal component analysis dimensionality reduction and convolutional neural network. The specific processing method is as follows:

[0010] (1) collecting the original noisy image, dividing it into a number of image blocks of the same size and digitizing them to obtain an image matrix, and calculating the covariance matrix of the feature matrix and the eigenvalues ​​of the covariance matrix;

[0011] (2) The Pearson, Spearman and Kendall correlation coefficients are used to calculate the correlation between the eigenvalue and the noise level. The mean square error of each correlation index is calculated to determine the weight. The larger the variance, the more information there is. Therefore, the index with a large mean square error is given a larger weight. The feature threshold k is set, and the principal component analysis method is used to select the eigenvalue with a higher correlation coefficient with the noise level as the main feature. The correlation coefficient weight calculation formula is as follows:

[0012]

[0013] Where W1, W2, W3 are the weights of Pearson, Spearman and Kendall correlation coefficients respectively; s1, s2, s3 are the standard deviations of Pearson, Spearman and Kendall correlation coefficients respectively;

[0014] (3) The reduced image is input into a convolutional neural network consisting of 1 1×1, m 3×3 convolutions, m linear rectification activation functions, (m-1) pooling layers, and 1 fully connected layer to obtain the estimated noise level;

[0015] Step 3: Construct a noise reduction network module. The specific processing method is as follows:

[0016] (1) Construct a dense connection module. The entire dense connection module consists of 3×3 convolution and linear rectification activation function. The module calculation structure is as follows:

[0017] x1=r(Con 3×3 (x in ))

[0018] x2=r(Con 3×3 (x1+x in ))

[0019] x out = r(Con 3×3 (x1+x2))

[0020] In the formula, x in and x out They represent the input and output of the model respectively, r() represents the linear rectification activation function calculation operation, and Con() represents the 3×3 convolution operation.

[0021] (2) The entire denoising network consists of n densely connected modules, 1 downsampling layer, and 1 upsampling layer;

[0022] Step 4: Use the training set obtained in step 1 to train the network models built in steps 2 and 3. Use the cross entropy loss function and the root mean square error loss function to calculate the errors of the two parts and construct an adaptive weight joint loss function. Optimize the two networks at the same time, use the peak signal-to-noise ratio to objectively evaluate the network model, and save the best model parameters.

[0023] The formula of the combined loss function is as follows:

[0024]

[0025] Where L N is the cross entropy loss function corresponding to the noise level estimation network, L D is the root mean square error loss function corresponding to the denoising network, M represents the category of the given noise level value; when the predicted noise level is the same as the actual noise level, p c The value is 1, otherwise it is 0; q c It represents the probability when the predicted noise level is c noise level, N is the total number of pixels, is the real image, λ is the weight of the loss function; the selection of the loss function weight is adaptively updated according to the loss of the previous round of training, and the adaptive weight coefficient makes the cross entropy loss and the root mean square error loss always equal to achieve the effect of balancing the network; the calculation formula of the weight coefficient is as follows:

[0026] λ=L' N / (L' N / L' D )

[0027] Where λ is the weight coefficient of the current training round, L' N is the noise estimation loss of the previous round of training, L' D is the noise reduction loss of the previous round of training;

[0028] The noise level estimated in step 2 is reconstructed into a noise map and input into the denoising network together with the noisy image to obtain the denoised image.

[0029] The method for image denoising based on noise level estimation provided by the present invention is based on feature dimension reduction and convolutional neural network. The noise level estimation module of the present invention has the advantages of accurate estimation and high computational efficiency. Noise reduction is performed after the noise level is estimated in advance, which overcomes the problem that the commonly used FFDNet model is heavily dependent on prior parameters, and replaces the continuous convolution part of the FFDNet model with a densely connected block, greatly improving the feature extraction ability of the model. Experimental verification shows that the present invention can quickly and efficiently complete image denoising and has a wide range of application value.

[0030] Beneficial effects:

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] Based on feature dimension reduction and convolutional neural network, the noise level estimation module of the present invention has the advantages of accurate estimation and high computational efficiency. It performs noise reduction after estimating the noise level first, overcoming the problem that the commonly used FFDNet model is heavily dependent on prior parameters, and replaces the continuous convolution part of the FFDNet model with a densely connected block, greatly improving the feature extraction ability of the model. Experimental verification shows that the present invention can effectively remove image noise and has a wide range of application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The flowchart of image denoising based on noise level estimation is shown;

[0034] Figure 2 This is the structure diagram of the image denoising network based on noise level estimation;

[0035] Figure 3 This is a structural diagram of a densely connected module;

[0036] Figure 4 Denoising effects of different algorithms: (a) original image, (b) noisy image, (c) BM3D algorithm, (d) DnCNN algorithm, (e) FFDNet algorithm, and (f) NDNet algorithm. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0038] A method for image denoising based on noise level estimation, the specific calculation method includes the following steps:

[0039] Step 1: Preprocess the original image data. The specific processing method is as follows:

[0040] (1) Add Gaussian noise of different noise levels to the image and resize the image to a pixel size of 256×256;

[0041] (2) Divide the preprocessed image dataset into a training set and a test set, using the original data without noise as the training target;

[0042] Step 2: Construct a noise level estimation network module. The entire noise level estimation network consists of principal component analysis dimensionality reduction and convolutional neural network. The specific processing method is as follows:

[0043] (1) collecting the original noisy image, dividing it into a number of image blocks of the same size and digitizing them to obtain an image matrix, and calculating the covariance matrix of the feature matrix and the eigenvalues ​​of the covariance matrix;

[0044] (2) The Pearson, Spearman and Kendall correlation coefficients are used to calculate the correlation between the eigenvalue and the noise level. The mean square error of each correlation index is calculated to determine the weight. The larger the variance, the more information there is. Therefore, the index with a large mean square error is given a larger weight. The feature threshold k is set, and the principal component analysis method is used to select the eigenvalue with a higher correlation coefficient with the noise level as the main feature. The correlation coefficient weight calculation formula is as follows:

[0045]

[0046] Where W1, W2, W3 are the weights of Pearson, Spearman and Kendall correlation coefficients respectively; s1, s2, s3 are the standard deviations of Pearson, Spearman and Kendall correlation coefficients respectively;

[0047] (3) The reduced image is input into a convolutional neural network consisting of one 1×1, four 3×3 convolutions, four linear rectification activation functions, three pooling layers, and one fully connected layer to obtain the estimated noise level;

[0048] Step 3: Construct a noise reduction network module. The specific processing method is as follows:

[0049] (4) Construct a dense connection module. The entire dense connection module consists of 3×3 convolution and linear rectification activation function

[0050] The module calculation structure is as follows:

[0051] x1=r(Con 3×3 (x in ))

[0052] x2=r(Con 3×3 (x1+x in ))

[0053] x out = r(Con 3×3 (x1+x2))

[0054] In the formula, x in and x out They represent the input and output of the model respectively, r() represents the linear rectification activation function calculation operation, and Con() represents the 3×3 convolution operation.

[0055] (5) The entire denoising network consists of n densely connected modules, a downsampling layer, and an upsampling layer;

[0056] Step 4: Use the training set obtained in step 1 to train the network models built in steps 2 and 3. Use the cross entropy loss function and the root mean square error loss function to calculate the errors of the two parts and construct an adaptive weight joint loss function. Optimize the two networks at the same time, use the peak signal-to-noise ratio to objectively evaluate the network model, and save the best model parameters.

[0057] The formula of the combined loss function is as follows:

[0058]

[0059] Where L N is the cross entropy loss function corresponding to the noise level estimation network, L D is the root mean square error loss function corresponding to the denoising network, M represents the category of the given noise level value; when the predicted noise level is the same as the actual noise level, p c The value is 1, otherwise it is 0; q c It represents the probability when the predicted noise level is c noise level, N is the total number of pixels, is the real image, λ is the weight of the loss function; the selection of the loss function weight is adaptively updated according to the loss of the previous round of training, and the adaptive weight coefficient makes the cross entropy loss and the root mean square error loss always equal to achieve the effect of balancing the network; the calculation formula of the weight coefficient is as follows:

[0060] λ=L' N / (L' N / L' D )

[0061] Where λ is the weight coefficient of the current training round, L' N is the noise estimation loss of the previous round of training, L' D is the noise reduction loss of the previous round of training;

[0062] The noise level estimated in step 2 is reconstructed into a noise map, which is input into the denoising network together with the noise image to obtain the denoised image;

[0063] Step 5: Demonstration of the effect of the implementation method of the present invention. The following table presents the noise removal effect of the present invention, as shown in Table 1:

[0064] Table 1 Comparison of PSNR values ​​after denoising by different denoising algorithms

[0065] Tab.1Comparison of PSNR values ​​after noise reduction of different noise reduction algorithms

[0066]

[0067] The table compares the classic denoising algorithms BM3D, DnCNN, and FFDNet with the denoising algorithms NDNet and DNet in this paper. The DNet algorithm is used as an ablation experiment (removing the NLE module based on the NDNet algorithm). The PSNR value is used to evaluate the denoising effects of the above algorithms. From the experimental results, it can be seen that the denoising effect of the method of the present invention is the best.

[0068] The above description is not intended to impose any form of limitation on the present invention. Although the present invention has been disclosed through the above embodiments, it is not intended to limit the present invention. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. An image denoising method based on noise level estimation, characterized by comprising the following steps: Step 1: Preprocess the original image data. The specific processing method is as follows: (1) Add Gaussian noise of different noise levels to the image and resize the image to n×n pixel size; (2) Divide the preprocessed image dataset into a training set and a test set, using the original data without noise as the training target; Step 2: Construct a noise level estimation network module. The entire noise level estimation network consists of principal component analysis dimensionality reduction and convolutional neural network. The specific processing method is as follows: (1) collecting the original noisy image, dividing it into a number of image blocks of the same size and digitizing them to obtain an image matrix, and calculating the covariance matrix of the feature matrix and the eigenvalues ​​of the covariance matrix; (2) Pearson, Spearman and Kendall correlation coefficients are used to calculate the correlation between the eigenvalue and the noise level, and the mean square error of each correlation index is calculated to determine the weight. The larger the variance, the more information there is, so the index with a large mean square error value is given a larger weight; Set the feature threshold k, use the principal component analysis method, select the k eigenvalues ​​with the highest correlation coefficient with the noise level, and the correlation coefficient weight calculation formula is as follows: Where W1, W2, W3 are the weights of Pearson, Spearman and Kendall correlation coefficients respectively; s1, s2, s3 are the standard deviations of Pearson, Spearman and Kendall correlation coefficients respectively; (3) The reduced image is input into a convolutional neural network consisting of 1 1×1, m 3×3 convolutions, m linear rectification activation functions, (m-1) pooling layers, and 1 fully connected layer to obtain the estimated noise level; Step 3: Construct a noise reduction network module. The specific processing method is as follows: (1) Construct a dense connection module. The entire dense connection module consists of 3×3 convolution and linear rectification activation function. The module calculation structure is as follows: x1=r(With 3×3 (x in )) x2=r(With 3×3 (x1+x in )) x out =r(With 3×3 (x1+x2)) In the formula, x in and x out Represent the input and output of the model respectively, r() represents the linear rectification activation function calculation operation, and Con() represents the 3×3 convolution operation; (2) The entire denoising network consists of n densely connected modules, a downsampling layer, and an upsampling layer; Step 4: Use the training set obtained in step 1 to train the network models built in steps 2 and 3. Use the cross entropy loss function and the root mean square error loss function to calculate the errors of the two parts and construct an adaptive weight joint loss function. Optimize the two networks at the same time, use the peak signal-to-noise ratio to objectively evaluate the network model, and save the best model parameters. The formula of the joint loss function is as follows: Where L N is the cross entropy loss function corresponding to the noise level estimation network, L D is the root mean square error loss function corresponding to the denoising network, M represents the category of the given noise level value; when the predicted noise level is the same as the actual noise level, p c The value is 1, otherwise it is 0; q c It represents the probability when the predicted noise level is c noise level, N is the total number of pixels, is the real image, λ is the weight of the loss function; the selection of the loss function weight is adaptively updated according to the loss of the previous round of training, and the adaptive weight coefficient makes the cross entropy loss and the root mean square error loss always equal to achieve the effect of balancing the network; the calculation formula of the weight coefficient is as follows: λ=L' N / (L' N / L' D ) Where λ is the weight coefficient of the current training round, L' N is the noise estimation loss of the previous round of training, L' D is the noise reduction loss of the previous round of training; Step 5: Reconstruct the noise level estimated in step 2 into a noise map, and input it together with the noise image into the denoising network trained in step 4 to obtain the denoised image.

Images