An image denoising method based on decoupled deep dictionary learning
By decomposing the image into structural and texture images and learning using adaptive dictionary, the problem of model degradation and slow convergence during image denoising in the prior art is solved, and efficient image recovery in high-noise environments is achieved.
Patent Information
- Application Number
- CN202210572476.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-25
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2042-05-25
AI Technical Summary
In the process of image denoising, the prior art has problems such as severe model degradation, slow network convergence, and inadaptive dictionary, especially in high noise environments, it is difficult to effectively restore the fine structure of the image.
Using a method based on decoupled deep dictionary learning, the image is decomposed into structural images and texture images, an adaptive dictionary is used and the sensitivity penalty operator is introduced, and the image fusion technology is used to resist noise and model degradation, and the denoising effect is improved by using deep learning models.
Effectively utilizing deep learning capabilities improves the microstructure recovery ability of image denoising, solves the problems of model degradation and slow convergence, and significantly improves the denoising performance.
Smart Images
Figure CN114881885B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a natural source image denoising method, in particular to an image denoising method based on decoupled deep dictionary learning, and belongs to the field of image denoising. Background Art
[0002] With the rapid development of information science and technology, people have been doing more and more research on target detection, object recognition, image retrieval, etc. However, many applications such as target detection, object recognition, image retrieval, etc. require the input of digital images that are as clear as possible. Digital images may be contaminated by noise for various reasons during the acquisition and storage process. Therefore, image denoising is a very important topic.
[0003] At present, the existing denoising methods include: denoising using K-SVD dictionary training algorithm, denoising using nonlinear reaction diffusion algorithm (TNRD), denoising using median filtering, denoising using wavelet transform, and denoising using three-dimensional block matching algorithm (BM3D) block matching sparse three-dimensional transform domain collaborative filtering algorithm. Among them, denoising using BM3D block matching sparse three-dimensional transform domain collaborative filtering algorithm is the current optimal denoising method, and has achieved a high level in denoising effect. The denoising using BM3D algorithm is based on the information inside the noise image. The larger the standard deviation of Gaussian noise or the worse the lighting conditions, the lower the signal-to-noise ratio of the image, the less useful information can be used inside the image, the more noise interference is received, and theoretically the denoising effect will be worse. When the information that can be used inside the image becomes less, denoising can be performed with the help of external information. Therefore, scholars have proposed a method for image denoising based on deep learning.
[0004] Since the traditional dictionary denoising model lacks shift invariance, Cristina proposed convolutional dictionary learning (CDicL), which uses convolution operations to replace matrix multiplication in signal representation to solve the problem. However, this method learns a universal dictionary to represent all images, which reduces the flexibility of the model. There are also many technologies that integrate dictionary learning and DNN for image denoising, which use manual construction of prior knowledge and do not effectively utilize the learning ability of DNN. Based on this, it is necessary to propose a new dictionary learning framework based on the CDicL model. Summary of the invention
[0005] In response to the above-mentioned problems, the present invention provides an image denoising method based on decoupled deep dictionary learning, which uses image decomposition technology to decompose the input image into a structural image and a texture image. After decomposing each part, an adaptive dictionary is used to ensure the denoising effect. A sensitivity penalty operator is added to prevent redundancy. At the same time, image fusion technology is used to restore the image, which solves the problems of severe model degradation, slow network convergence, and non-adaptive dictionary in the current mainstream methods.
[0006] The technical solution adopted by the present invention to solve the above technical problems is as follows:
[0007] An image denoising method based on decoupled deep dictionary learning specifically includes the following steps:
[0008] Step 1, preprocessing the image data;
[0009] Step 2, solve the decoupled deep dictionary learning model and construct a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet;
[0010] Step 3, setting the loss function and training parameters of the constructed neural network model;
[0011] Step 4, input the preprocessed image into the neural network model set in step 3 for parameter training to obtain a network model with better denoising effect;
[0012] Step 5: Input the image to be tested into the network model obtained in step 4 to obtain a clear image after denoising.
[0013] The step 1 is specifically as follows:
[0014] Different natural image datasets are collected and combined, Gaussian white noise is added to the images, and then modern variational techniques are used to decompose the noisy images into structural images and texture images. Finally, the images are cropped into image blocks of 128*128 size.
[0015] The deep decoupled dictionary learning model described in step 2 is:
[0016]
[0017] Where N means using a set of N training sample pairs; Y means a complete image after adding noise; subscripts 1 and 2 represent the representation terms of the structure image and texture image respectively; X means the coefficient matrix of the image; D means the adaptive dictionary of the image; P means the sensitivity decoupling operator introduced for each part of the image; λ means the regularization parameter; φ means the coefficient prior; ψ means the regularization term.
[0018] In step 2, the specific method for solving the structural image and texture image (i.e., X, D, P,) in the decoupled deep dictionary learning model includes the following steps, where the structural image is X1, D1, P1, the texture image is X2, D2, P2, and the subscript s represents the representation item of the common part of the structural image and the texture image:
[0019] Step 21: Introduce auxiliary variable D aux , X aux , P aux , and solve it;
[0020] As shown in the following formula, FFT (fast Fourier transform) can effectively obtain X aux Closed-form solution:
[0021]
[0022] In the formula, F -1 Denotes the inverse FFT, defines F(·) as the two-dimensional Fourier transform, and lets d = F(D), x * =F(X * ), p=F(P), represents the complex conjugate of d, represents the Hadamard product, A↑C means expanding the channel dimension of A to C, a X is a hyperparameter;
[0023] As shown in the following formula, the least square method is used to calculate D aux and P aux Solve and finally get D aux and P aux The closed-form solution of is:
[0024]
[0025]
[0026] where vec(·) is the vectorization operator, d s =vec(D s ), p s =vec(P s ), vec -1 (·) indicates inverse vectorization, for and y=u k (Y s ),u a represents the expansion operator with kernel size a, is a circular filling operator with width; a D and aP is a hyperparameter, and the subscript s represents the common part of the structure image and the texture image;
[0027] Step 22: Build the SplitNet module;
[0028] The SplitNet module takes the noise level σ as input and predicts the hyperparameters of each stage; the SplitNet module consists of 2 convolutional layers (Conv) with kernel size 1 and a SoftPlus layer to ensure that all hyperparameters are positive.
[0029] Step 23: Build the InputNet module to initialize X, D, and P
[0030] The InputNet module uses the noise image Y and the noise level σ as outputs to initialize the coefficients; the InputNet module consists of 2 Conv layers (each with 64 channels) and a ReLU (Rectified Linear Units) activation function, and outputs the initialization coefficients given the input image Y.
[0031] Step 24: Build LNM-X module, LNM-D module, and LNM-P module;
[0032] For the construction of the LNM-X module, the solution of X at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0033]
[0034] where β X It is a hyperparameter, which can be decomposed by SplitNet; is the value of the auxiliary variable of the image coefficient at stage t, Represents X s,(t-1) The value of
[0035] The LNM-X module learns prior knowledge on the coefficient X and acts as an implicit regularizer. The U-shaped network structure is adopted to ensure the effectiveness of the LNM-X module in the image restoration task. The LNM-X module consists of 7 blocks, of which the first 3 blocks downsample the feature map by convolution, and the last 3 blocks upsample the feature map by transposed convolution. Each block consists of several residual units, each of which consists of 2 Conv layers with ReLU activation and skip connection. The Conv layers in the first 4 blocks have 64, 128, 256, and 512 channels respectively.
[0036] For the construction of the LNM-D module, the solution of D at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0037]
[0038] where β D is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the image dictionary at stage t, Indicates D s,(t-1) The value of
[0039] The LNM-D module learns prior knowledge on the dictionary D; the LNM-D module consists of 6 Conv layers with ReLU activation, and there is no ReLU after the last Conv layer; each Conv layer has 16 channels, and there is a skip connection between the first and last units;
[0040] For the construction of the LNM-P module, the solution of P at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0041]
[0042] where β P is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the decoupling operator at stage t, Indicates P s,(t-1) The value of
[0043] The LNM-P module learns prior knowledge on the penalty operator; the LNM-P module consists of 6 Conv layers with ReLU activation, and there is no ReLU after the last Conv layer; each Conv layer has 16 channels, and there is a skip connection between the first and last units;
[0044] The model-driven network is used to fuse the different components of the image, namely the structural image and the texture image, in order to resist noise, resist model degradation and help the network converge.
[0045] The step 3 is as follows:
[0046] We use L1 function as the loss function for the output of each level and set the loss weight on the last stage to 1. For the other T-1 stages, we set the weight to 1 / T-1. The batch size is set to 32 and trained for 1e6 iterations, decaying by 0.5 every 2e5 iterations. All T levels share the same parameters. We train a shared model for all noise levels, which is set to {15, 25, 50}.
[0047] The step 4 is specifically as follows:
[0048] The processed image is taken as input, and the noise-free image is obtained through the constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet. Then, the loss is calculated with the label image, and the parameters are trained through the gradient descent strategy. Finally, after several rounds of training, a network model with good denoising effect is obtained. Among them, the loss function adopts L1 loss (mean abs error); the optimizer adopts Adam optimizer, and the learning rate is initially 1e-4 and decays with continuous training, decaying by 0.5 times every 2e5 iterations.
[0049] According to a second aspect of the present invention, there is provided an image denoising system based on decoupled deep dictionary learning, comprising:
[0050] Image dataset preprocessing module: collect different natural image datasets and combine them, add Gaussian white noise to the images, and then use modern variational techniques to decompose the noisy images into structural images and texture images. Finally, the images are cropped into image blocks of 128*128 size.
[0051] Decoupled deep dictionary learning model solving module: solves the decoupled deep dictionary learning model and builds a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet;
[0052] Processing module: Input the preprocessed image into the above-constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet for parameter training to obtain a network model with better denoising effect; and input the image to be tested into the above-mentioned network model to obtain a clear image after denoising.
[0053] According to a third aspect of the present invention, there is provided a computer device, comprising a memory, a central processing unit, a graphics processing unit and a computer program, wherein the central processing unit is connected to the memory, the memory is used to store the computer program for implementing the above-mentioned image denoising method, and when the computer device is running, the central processing unit loads and executes the computer program stored in the memory, so that the computer device implements each step of the above-mentioned image denoising method.
[0054] The technical solution of the present invention can produce the following technical effects:
[0055] 1. The present invention introduces decoupling in the image denoising process to ensure that the noisy image can be decomposed into two different components, namely the structural image and the texture image. After the decomposition, convolutional sparse perception is constructed for each component, and a penalty operator is added based on the decoupling model to prevent entanglement and redundancy between different decoupling models. At the same time, the present invention uses the concept of dictionary learning to learn the prior knowledge of X, D and P from the training data, and learns specific dictionaries for the structural image and the texture image respectively. The dictionary can not only adapt to the image content, but also perceive the global information of the image, and effectively utilizes the learning ability of DNN, so that it can well restore the subtle structure of the image even in a severe noise environment.
[0056] 2. The present invention utilizes a model-driven network to fuse different components of an image, thereby resisting noise and model degradation, and helping the network converge. It solves the problems of severe model degradation, slow network convergence, and non-adaptive dictionaries existing in current mainstream methods.
[0057] 3. The present invention is based on a deep learning convolutional neural network, which is an end-to-end network model and reduces the complexity of network design. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 This is the overall framework diagram of the decoupled deep dictionary learning model described in the present invention;
[0059] Figure 2 A schematic diagram of the structure of the LNM-X module provided in an embodiment of the present invention;
[0060] Figure 3 A schematic diagram of the structure of the LNM-P module and the LNM-D module provided in an embodiment of the present invention;
[0061] Figure 4 A schematic diagram of the structure of the SplitNet module and the InputNet module provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.
[0063] In this embodiment, Figure 1 The overall architecture of the decoupled deep dictionary learning model of the present invention is shown. The image denoising method based on decoupled deep dictionary learning described in this embodiment specifically includes the following steps:
[0064] Step 1: Preprocess the image data
[0065] Different natural image datasets are collected and combined, and three levels of additive Gaussian white noise {15, 25, 50} are added to the dataset to obtain noisy images of different levels; then the noisy images after adding noise are decomposed into structural images and texture images using modern variational techniques, such as Figure 1 As shown, U and V represent the structural image and the texture image respectively; finally, the structural image and the texture image are cropped into image blocks of size 128*128.
[0066] Step 2: Solve the decoupled deep dictionary learning model and build a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet;
[0067] The deep decoupled dictionary learning model of this embodiment is improved based on the CDicL model. The deep decoupled dictionary learning model can be written as:
[0068]
[0069] Where N means using a set of N training sample pairs; Y means a complete image after adding noise; subscripts 1 and 2 represent the representation terms of the structure image and texture image respectively; X means the coefficient matrix of the image; D means the adaptive dictionary of the image; P means the sensitivity decoupling operator introduced for each part of the image; λ means the regularization parameter; φ means the coefficient prior; ψ means the regularization term.
[0070] First, solve X1, D1, and P1 in the model (i.e., the structural image). The solution method for the texture image is the same as that for the structural image. In order to simplify the expression, we use the subscript s to represent the general part. Unless otherwise specified, all formulas can be used for both the structural image and the texture image. aux , X aux , P aux , and solve it; as shown in the following formula, FFT (fast Fourier transform) is used to effectively obtain X aux Closed-form solution:
[0071]
[0072] In the formula, F -1 Denotes the inverse FFT, defines F(·) as the two-dimensional Fourier transform, and lets d = F(D), x * =F(X * ), p=F(P), represents the complex conjugate of d, represents the Hadamard product, A↑C means expanding the channel dimension of A to C, a Xis a hyperparameter;
[0073] For D aux and P aux , since the dimensions of these two are much lower than the dimension of X, the system is overdetermined, so as shown in the following formula, the least squares method is used to solve it (such as the LU solver provided by PyTorch), and finally its closed-form solution can be obtained:
[0074]
[0075]
[0076] where vec(·) is the vectorization operator, d s =vec(D s ), p s =vec(P s ), vec -1 (·) indicates inverse vectorization, for and y=u k (Y s ),u a represents the expansion operator with kernel size a, is a circular filling operator with width; a D and a P is a hyperparameter, and the subscript s represents the common part of the structure image and the texture image;
[0077] First build the SplitNet module, such as Figure 4 As shown in the figure, where α and β both represent hyperparameters; the SplitNet module takes the noise level σ as input and predicts the hyperparameters of each stage; the SplitNet module consists of 2 convolutional layers (Conv) with a kernel size of 1 and a SoftPlus layer to ensure that all hyperparameters are positive.
[0078] Secondly, construct an InputNet module, which takes the noise image Y and the noise level σ as output to initialize the coefficients; the InputNet module consists of 2 Conv layers (each with 64 channels) and a ReLU (Rectified Linear Units) activation function, and outputs the initialization coefficient X given the input image Y; D and P are also initialized by this module.
[0079] Then construct LNM-X module, LNM-D module, and LNM-P module;
[0080] For the construction of the LNM-X module, the solution of X at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0081]
[0082] where β X It is a hyperparameter, which can be decomposed by SplitNet; is the value of the auxiliary variable of the image coefficient at stage t, Represents X s,(t-1) The value of
[0083] like Figure 2 As shown, the LNM-X module learns prior knowledge on the coefficient X and acts as an implicit regularizer. The U-net network structure is adopted to ensure the effectiveness of the LNM-X module in the image restoration task. The LNM-X module consists of 7 blocks, of which the first 3 blocks downsample the feature map by convolution, and the last 3 blocks upsample the feature map by transposed convolution. Each block consists of several residual units, and each residual unit consists of 2 Conv layers with ReLU activation and skip connection. The Conv layers in the first 4 blocks have 64, 128, 256, and 512 channels respectively.
[0084] For the construction of the LNM-D module, the solution of D at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0085]
[0086] where β D is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the image dictionary at stage t, Indicates D s,(t-1) The value of
[0087] The LNM-D module learns prior knowledge on the dictionary D; Figure 3 As shown, since the spatial size of D is much smaller than that of X, a shallower network is sufficient to provide sufficient learning ability; the LNM-D module consists of 6 Conv layers with ReLU activation, and there is no ReLU after the last Conv layer; each Conv layer has 16 channels, and there is a skip connection between the first and last units;
[0088] For the construction of the LNM-P module, the solution of P at stage t can be obtained through the deep decoupled dictionary learning model as follows:
[0089]
[0090] where β P is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the decoupling operator at stage t, Indicates P s,(t-1) The value of
[0091] The LNM-P module learns prior knowledge on the penalty operator; Figure 4 As shown, the LNM-P module consists of 6 Conv layers with ReLU activation and no ReLU after the last Conv layer. Each Conv layer has 16 channels and there is a skip connection between the first and last units.
[0092] Step 3: Set the loss function and training parameters of the constructed neural network model;
[0093] The L1 function is used as the loss function for each level output, and the loss weight on the last stage is set to 1. For the other T-1 stages, this embodiment sets the weight to 1 / T-1, and the batch size is set to 32 for 1e6 iterations, and decays by 0.5 times every 2e5 iterations. All T levels share the same parameters, and a shared model is trained for all noise levels, which is set to {15, 25, 50}.
[0094] Step 4: Input the preprocessed image into the neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet constructed in step 3 for parameter training to obtain a network model with better denoising effect;
[0095] The processed image is taken as input, and the noise-free image is obtained through the constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet. Then, the loss is calculated with the label image, and the parameters are trained through the gradient descent strategy. Finally, after several rounds of training, a network model with good denoising effect is obtained. Among them, the loss function adopts L1 loss (mean abs error); the optimizer adopts Adam optimizer, and the learning rate is initially 1e-4 and decays with continuous training, decaying by 0.5 times every 2e5 iterations.
[0096] Step 5: Use the image used for testing as input for testing. The trained network outputs the image and compares it with the previous image to obtain measurement indicators such as SSIM and PSNR.
[0097] In order to verify the effectiveness of the present invention, we use different mainstream denoising methods to test on the same data set and compare them with the embodiments of the present invention.
[0098] Comparative Example 1: According to the DnCNN method in (hang, Kai, et al. "Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising." IEEE transactions on imageprocessing 26.7 (2017): 3142-3155.);
[0099] Comparative Example 2: According to the NBNet method in (Cheng, Shen, et al. "Nbnet: Noise basis learning for image denoising with subspace projection." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021.);
[0100] Comparative Example 3: According to the Real Image Denoising with Feature Attention in (Anwar S,Barnes N.Real image denoising with featureattention[C].Proceedings of the IEEE / CVF international conference on computervision.2019:3155-3164.);
[0101] Comparative Example 4: According to the self-supervised denoising method of a single noisy image in (Huang T, Li S, Jia X, et al. Neighbor2neighbor: Self-supervised denoising from single noisy images[C]. Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2021: 14781-14790.);
[0102] Comparative Example 5: According to the FFDNet method in (Zhang K, Zuo W, Zhang L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising [J]. IEEE Transactions on Image Processing, 2018, 27 (9): 4608-4622.);
[0103] Comparative Example 6: According to the BRDNet method in (Tian C, Xu Y, Zuo W. Image denoising using deep CNN with batch renormalization[J]. Neural Networks, 2020, 121: 461-473.).
[0104] Comparative Example 7: According to the DCDicL method in (Zheng H, Yong H, Zhang L. Deep convolutional dictionary learning for image denoising [C] Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 630-641.).
[0105] The training set used for the grayscale model is Train400. For the color model, BSDS500 and part of Waterloo exploration are used as training sets. In order to ensure the fairness of the experiment, all training sets use the above data sets, and the test sets use six data sets respectively. The experimental results are shown in Tables 1 and 2; Table 1 shows the PSNR (dB) / SSIM (%) of the grayscale image experimental results of the embodiment of the present invention; Table 2 shows the PSNR (dB) / SSIM (%) of the color image experimental results of the embodiment of the present invention. It can be seen from Table 1 that in the comparative experiment of the Set12 data set, combined with different noise levels, the method proposed in the present invention improves the PSNR by an average of 1.49dB, an increase of 4%, compared with the DCDicL method. Compared with the DnCNN method with poor performance, the method proposed in the present invention improves the PSNR by 2.2dB. In the comparative experiment of the BSD68 data set, the denoising method proposed in the present invention has a greater improvement in denoising performance. It is 1.9dB higher than DCDicL on average in PSNR. It is incomparable to other denoising methods. It can be seen from Table 2 that the method proposed in the present invention also has significant improvements in various indicators on the color data set.
[0106] Table 1
[0107]
[0108] Table 2
[0109]
[0110] The present embodiment also provides an image denoising system based on decoupled deep dictionary learning, which includes: an image data set preprocessing module, a decoupled deep dictionary learning model solving module and a processing module; the image data set preprocessing module collects different natural image data sets and combines them, adds Gaussian white noise to the image, and then uses modern variational technology to decompose the noisy image into a structural image and a texture image, and finally crops the image into an image block of size 128*128; the decoupled deep dictionary learning model solving module is used to solve the decoupled deep dictionary learning model and construct a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet; the processing module is responsible for inputting the preprocessed image into the above-constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet for parameter training to obtain a network model with better denoising effect; and inputting the image to be tested into the above-mentioned network model to obtain a clear image after denoising.
[0111] This embodiment also provides a computer device, including a memory, a central processing unit (CPU) and a graphics processing unit (GPU), and a computer program, wherein the CPU is connected to the memory, and the memory is used to store a computer program for implementing the image denoising method as described above. When the computer device is running, the CPU loads and executes the computer program stored in the memory, so that the computer device implements each step of the image denoising method as described above.
[0112] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.
Claims
1. An image denoising method based on decoupled deep dictionary learning, characterized in that: The specific steps include: Step 1, preprocessing the image data; Step 2, solve the decoupled deep dictionary learning model and construct a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet; Step 3, setting the loss function and training parameters of the constructed neural network model; Step 4, input the preprocessed image into the neural network model set in step 3 for parameter training to obtain a network model with better denoising effect; The specific method for solving the structural image and texture image (X, D, P,) in the decoupled deep dictionary learning model includes the following steps, where the structural image is X1, D1, P1, the texture image is X2, D2, P2, and s represents the representation item of the common part of the structural image and the texture image: Step 21: Introduce auxiliary variable D aux , X aux , P aux , and solve it; As shown in the following formula, X can be effectively obtained by fast Fourier transform. aux Closed-form solution: In the formula, F -1 Denotes the inverse FFT, defines F(·) as the two-dimensional Fourier transform, and lets d = F(D), x * =F(X * ), p=F(P), represents the complex conjugate of d, represents the Haldmann product, A↑C means expanding the channel dimension of A to C, a X is a hyperparameter; As shown in the following formula, the least square method is used to calculate D aux and P aux Solve and finally get D aux and P aux The closed-form solution of is: where vec(·) is the vectorization operator, d s =vec(D s ), p s =vec(P s ), vec -1 (·) indicates inverse vectorization, for and y=u k (Y s ), ua represents the expansion operator with kernel size a, is a circular filling operator with width; a D and a P is a hyperparameter, and the subscript s represents the common part of the structure image and the texture image; Step 22: Build the SplitNet module; The SplitNet module takes the noise level σ as input and predicts the hyperparameters of each stage; the SplitNet module consists of 2 convolutional layers Conv kernel size 1 and a SoftPlus layer to ensure that all hyperparameters are positive; Step 23: Build the InputNet module to initialize X, D, and P The InputNet module uses the noise image Y and the noise level σ as outputs to initialize the coefficients; the InputNet module consists of two Conv layers and a ReLU activation function, each of the Conv layers has 64 channels, and outputs the initialization coefficients by giving the input image Y; Step 24: Build LNM-X module, LNM-D module, and LNM-P module; For the construction of the LNM-X module, the solution of X at stage t can be obtained through the decoupled deep dictionary learning model as follows: where β X It is a hyperparameter, which can be decomposed by SplitNet; is the value of the auxiliary variable of the image coefficient at stage t, Represents X s,(t-1) The value of The LNM-X module learns prior knowledge on the coefficient X and acts as an implicit regularizer. The U-shaped network structure is adopted to ensure the effectiveness of the LNM-X module in the image restoration task. The LNM-X module consists of 7 blocks, of which the first 3 blocks downsample the feature map by convolution, and the last 3 blocks upsample the feature map by transposed convolution. Each block consists of several residual units, each of which consists of 2 Conv layers with ReLU activation and skip connection. The Conv layers in the first 4 blocks have 64, 128, 256, and 512 channels respectively. For the construction of the LNM-D module, the solution of D at stage t can be obtained through the decoupled deep dictionary learning model as follows: where β D is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the image dictionary at stage t, Indicates D s,(t-1) The value of The LNM-D module learns prior knowledge on the dictionary D; the LNM-D module consists of 6 Conv layers with ReLU activation, and there is no ReLU after the last Conv layer; each Conv layer has 16 channels, and there is a skip connection between the first and last units; For the construction of the LNM-P module, the solution of P at stage t can be obtained through the decoupled deep dictionary learning model as follows: where β P is a hyperparameter, which can be decomposed by SplitNet. is the value of the auxiliary variable of the decoupling operator at stage t, Indicates P s,(t-1) The value of The LNM-P module learns prior knowledge on the penalty operator; The LNM-P module consists of 6 Conv layers with ReLU activation, and there is no ReLU after the last Conv layer; each Conv layer has 16 channels, and there is a skip connection between the first and last units; Use the model-driven network to fuse the different components of the processed image, namely the structural image and the texture image; Step 5: Input the image to be tested into the network model obtained in step 4 to obtain a clear image after denoising.
2. The image denoising method based on decoupled deep dictionary learning according to claim 1, characterized in that: The step 1 is specifically as follows: Different natural image datasets are collected and combined, Gaussian white noise is added to the images, and then modern variational techniques are used to decompose the noisy images into structural images and texture images. Finally, the structural images and texture images are cropped into image blocks of 128*128 size respectively.
3. The image denoising method based on decoupled deep dictionary learning according to claim 2, characterized in that: The decoupled deep dictionary learning model described in step 2 is: Where N means using a set of N training sample pairs; Y means a complete image after adding noise; subscripts 1 and 2 represent the representation terms of the structure image and texture image respectively; X means the coefficient matrix of the image; D means the adaptive dictionary of the image; P means the sensitivity decoupling operator introduced for each part of the image; λ means the regularization parameter; φ means the coefficient prior; ψ means the regularization term.
4. The image denoising method based on decoupled deep dictionary learning according to claim 3, characterized in that: The step 3 is as follows: Use the L1 function as the loss function for each level output, and set the loss weight on the last stage to 1. For the other T-1 stages, set the weight to 1 / T-1, and set the batch size to 32 for 1e 6 Iterative training, every 2e 5 The noise level is decayed by a factor of 0.5 per iteration; all T levels share the same parameters, and a shared model is trained for all noise levels, which is set to {15, 25, 50}.
5. The image denoising method based on decoupled deep dictionary learning according to claim 1, characterized in that: The step 4 is specifically as follows: The processed image is used as input, and the noise-free image is obtained through the constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet. Then, the loss is calculated with the label image, and the parameters are trained through the gradient descent strategy. Finally, after several rounds of training, a network model with good denoising effect is obtained; the loss function uses L1 loss; the optimizer uses Adam optimizer, and the initial learning rate is 1e 4 And it decays with continuous training, every 2e 5 The decay is 0.5 times per iteration.
6. An image denoising system based on decoupled deep dictionary learning generated by an image denoising method based on decoupled deep dictionary learning according to any one of claims 1 to 5, characterized in that: include: Image dataset preprocessing module: collect different natural image datasets and combine them, add Gaussian white noise to the images, and then use modern variational techniques to decompose the noisy images into structural images and texture images. Finally, the images are cropped into image blocks of 128*128 size. Decoupled deep dictionary learning model solving module: solves the decoupled deep dictionary learning model and builds a neural network model including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet, and uses the model-driven network to fuse different components of the processed image; Processing module: The preprocessed image is input into the constructed neural network including InputNet, LNM-X, LNM-D, LNM-P, and SplitNet for parameter training to obtain a network model with better denoising effect; and the image to be tested is input into the above network model to obtain a clear denoised image.
7. A computer device, characterized in that: The invention comprises a memory, a central processing unit (CPU) and a graphics processing unit (GPU), wherein the central processing unit is connected to the memory, the memory is used to store a computer program for implementing the image denoising method according to any one of claims 1 to 5, and when the computer device is running, the central processing unit loads and executes the computer program stored in the memory, so that the computer device implements each step of the image denoising method according to any one of claims 1 to 5.