An image denoising method based on improved spatial attention

Abstract

This invention discloses an image denoising method based on improved spatial attention. First, the original image and the corresponding noisy image are used as training samples to construct a training sample set. Next, an image denoising model is constructed, which includes a basic feature layer for extracting local features from the noisy image and a spatial attention layer for extracting spatial feature information from the noisy image. The basic feature layer has 13 layers, each including a convolutional feature extraction module, and the spatial attention layer has two layers, including a traditional spatial attention extraction module and an additional spatial domain information extraction module. The constructed image denoising model is then trained using the training set to obtain a trained image denoising model. Finally, when applying the model, the noisy image is input into the trained image denoising model, and the corresponding denoised image is calculated and output. The invention adds an additional pooling operation to the spatial attention layer to effectively extract spatial information from the internal noisy image.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to an image denoising method based on improved spatial attention. Background Art

[0002] Image denoising involves restoring a clean image from a noisy one. It's a typical low-level computer vision task, widely used in fields such as remote sensing and medical imaging. Image denoising tasks can be broadly categorized into four types: additive white Gaussian noise denoising, mixed noise image denoising, blind noise denoising, and real image denoising. From a practical perspective, the closer the noise is to real-world conditions, the more practical the research.

[0003] There are many image denoising methods based on image degradation models, such as traditional spatial domain processing, frequency domain processing, three-dimensional filtering methods based on block matching, wavelet denoising algorithms, dictionary-based methods, principal component analysis algorithms, total variation methods, etc. Although these methods have achieved good performance in image denoising, there are still some challenges: (1) manual parameter selection, and (2) the optimization algorithm of the model is relatively complex.

[0004] Due to their strong adaptive learning capabilities, deep learning methods have become the leading technology for image denoising, achieving excellent results in preserving local details and textures. Following the successful application of AlexNet in image processing, many researchers have proposed CNN-based image denoising methods. While these methods have improved denoising performance, they still have some drawbacks, such as the high computational cost due to deeper networks and complex structures, and the fact that the information extracted from deeper networks may not always be valid. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides an image denoising method based on improved spatial attention, which can quickly remove Gaussian noise in the image and improve the visual effect of the image.

[0006] The present invention is achieved through the following technical solutions:

[0007] An image denoising method based on improved spatial attention comprises the following steps:

[0008] Step 1: Take the original image and the corresponding noise image as a training sample to construct a training sample set;

[0009] Step 2: Construct an image denoising model, which includes a basic feature layer for extracting local features of the noisy image and a spatial attention layer for extracting spatial feature information of the noisy image. The basic feature layer has 13 layers, including a convolutional feature extraction module, and the spatial attention layer has two layers, including a traditional spatial attention extraction module and an additional spatial domain information extraction module.

[0010] Step 3: Use the training set to train the constructed image denoising model to obtain a trained image denoising model;

[0011] Step 4: When applying, the noisy image is input into the trained image denoising model, and the corresponding denoised image is calculated and output.

[0012] Furthermore, in step 2, the basic feature layer consists of two groups of convolutional layers with a kernel size of 3. The basic feature layer maintains the number of channels as 64, and a short skip connection is added between the input and the last ReLU function.

[0013] Furthermore, in step 2, the spatial attention layer is divided into two parts: the first part is two groups of spatial feature extraction modules, and the second part is a group of spatial domain information extraction modules.

[0014] Furthermore, the first part of the traditional spatial attention extraction module calculates the average and maximum values ​​of all channels to obtain two sets of information, and then convolves these two sets of information to obtain spatial attention information. The convolution kernel size of this part is 3×3 kernel, and the activation function uses ReLU function; the second part of the additional spatial domain information extraction module extracts the features of the local plane position of the tensor in each channel. In order to obtain the matrix of the local average value, the size of each local window is set to 8×8, and the average value within each window is calculated. This process can be rewritten as,

[0015] E 21 = P(X), (1)

[0016] Where P is the pooling function with a kernel size of 8, X is the input matrix, and since E 21 If the size of is not equal to the size of the input X, use the nearest interpolation function N to adjust the matrix E 21 The size of

[0017] E 22 = N(E 21 ), (2)

[0018] Through equations (1) and (2), we get the matrix E 22, and then use the convolution layer after the ReLU function to convert the 64 feature maps into 1 channel as additional spatial information. Therefore, two sets of spatial information are obtained. Next, the two sets of spatial information are connected with the input feature X. In order to connect with the bottom layer, a convolution layer is used to convert the channel from 66 to 64. Finally, the output of the improved attention layer is obtained.

[0019] Furthermore, in step 3, during the training process, the noise image is processed by the convolution layer, the basic feature layer, and the spatial attention layer in sequence, and the loss is calculated based on the noise image. Then, the loss is back-propagated to update the neural network weight parameters.

[0020] Furthermore, when training the image denoising model, the input layer size is 64×64; when performing image restoration operations, the input layer size is the actual size of the image; the loss function used in training is the L2 loss function, and the training optimizer used is the Adam optimizer. In the grayscale image denoising task, the initial value of the learning rate is set to 10 -3 ; In the color image denoising task, the initial value of the learning rate is 10 -4 ; Preprocess the images in the training set. For gray images, each batch of training data includes 1000 40×40 image blocks. For color images, each batch of training data includes 128 40×40 image blocks. The training data is forward propagated to calculate the L2 loss with the clear image, and then the model parameters are updated through backpropagation of the loss. After training 100 batches, the model parameters are saved.

[0021] Furthermore, in step 4, the noisy image is input into the trained image denoising model, the saved model parameters are loaded, the calculation is forwarded, and the restored noise-free image is output.

[0022] Compared with existing technologies, the present invention has the following advantages:

[0023] 1. This paper adds an additional pooling operation in the spatial attention layer to effectively extract the spatial information of the internal noisy image.

[0024] 2. The neural network architecture proposed in this paper is lightweight. The network depth is relatively small, and the network architecture only includes the basic feature extraction layer and the spatial attention layer.

[0025] 3. The number of spatial attention layers of the neural network of the present invention is significantly lower than the number of basic feature extraction layers, which reduces the computational cost and improves the performance of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will be further described below with reference to the accompanying drawings.

[0027] Figure 1Improved spatial attention image denoising network model;

[0028] Figure 2 Flowchart for building and training an image denoising model;

[0029] Figure 3 Flowchart of image denoising using image denoising model;

[0030] Figure 4 Gray image noise reduction effect diagram;

[0031] Figure 5 Average PSNR and SSIM values ​​for image denoising on different datasets;

[0032] Figure 6 Color image noise reduction effect diagram. DETAILED DESCRIPTION

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Example 1:

[0035] The present invention provides an image denoising method based on improved spatial attention, which specifically includes two parts: constructing an image denoising model and denoising a noisy image using the model.

[0036] Construct an image denoising model, such as Figure 1 As shown, it mainly includes the following steps:

[0037] Step 1: Prepare a training sample set. Add Gaussian noise to the original image to obtain the noise image corresponding to the original image. Use the original image and the corresponding noise image as a training sample to construct a training sample set.

[0038] The gray image, color image, and corresponding noise image are divided into 180*180 image blocks, and each group of image blocks constitutes a training sample.

[0039] Step 2: Build an image denoising model, which consists of a convolutional layer, a basic feature layer, and an improved spatial attention layer, such as Figure 1 As shown, it specifically includes a basic feature module for extracting local features of noisy images and a spatial attention layer for extracting spatial feature information of noisy images.

[0040] The basic feature layer consists of two sets of convolutional layers with a kernel size of 3, and the number of channels in the basic feature layer is kept at 64. A short skip connection is added between the input and the last ReLU function. The improved spatial attention module can be divided into two parts: the first part is two sets of traditional spatial attention extraction modules, and the second part is a set of additional spatial domain information extraction modules.

[0041] The first part of the spatial feature extraction module is a traditional spatial attention module, which calculates the average and maximum values ​​of all channels to obtain two sets of information, and then convolves these two sets of information to obtain spatial attention information. The convolution kernel size of this part is 3×3 kernel, and the activation function uses the ReLU function. The second layer of additional spatial domain information extraction module extracts the features of the local plane position of the tensor in each channel. In order to obtain the matrix of local average values, the size of each local window is set to 8×8, and the average value within each window is calculated. This process can be rewritten as,

[0042] E 21 = P(X), (1)

[0043] Where P is the pooling function with a kernel size of 8, and X is the input matrix. 21 If the size of is not equal to the size of the input X, use the nearest interpolation function N to adjust the matrix E 21 The size of

[0044] E 22 = N(E 21 ), (2)

[0045] Through equations (1) and (2), we get the matrix E 22 . Then, a convolutional layer after the ReLU function is used to convert the 64 feature maps into 1 channel as additional spatial information. Thus, two sets of spatial information are obtained. Next, the two sets of spatial information are concatenated with the input feature X. In order to connect with the bottom layer, a convolutional layer is used to convert the channels from 66 to 64. Finally, the output of the improved attention layer is obtained.

[0046] Step 3: Use the training set to train the constructed image denoising model to obtain a trained image denoising model. That is, during the training process, the noisy image is processed by the convolution layer, basic feature layer and spatial attention layer in sequence, and the loss is calculated based on the noisy image. Then, the loss is back-propagated to update the neural network weight parameters.

[0047] In this paper, the training set includes grayscale and color images. The grayscale image training set uses 500 images from the DIV2K dataset, which are cut into 180×180 grayscale images. The color images are directly from the DIV2K HR dataset. Each image is randomly cut, and four 180×180 RGB images are selected as the training set.

[0048] When training the image denoising model, the input layer size is 64×64; when performing image restoration, the input layer size is the actual image size. The training loss function used is the L2 loss function, and the Adam optimizer is used. For the grayscale image denoising task, the initial learning rate is set to 10-3. For the color image denoising task, the initial learning rate is 10-4. The training set images are preprocessed. For grayscale images, each training batch consists of 1000 40×40 image patches, and for color images, each batch consists of 128 40×40 image patches. The training data is forward-propagated to calculate the L2 loss with the clear image, and this loss is then back-propagated to update the model parameters. After training for 100 batches, the model parameters are saved.

[0049] Use this image denoising model to denoise the noisy image:

[0050] After the image denoising model is trained, when it is applied, Figure 3 As shown in the figure, the noisy image is input into the trained image denoising model, the saved model parameters are loaded, the calculation is passed forward, and the restored noise-free image is output.

[0051] The neural network structure of the above image denoising model is very simple. This image denoising method can quickly remove image noise and improve the visual effect of the image.

[0052] Example 2

[0053] Grayscale image denoising. DIV2K images were used as the training set for grayscale images. These images were sliced ​​into 180×180 grayscale images. Before feeding the image data into the neural network model, the training set images were further preprocessed. Four different degrees of scaling were applied to the training set. The scaled images were then sliced ​​into 40×40 pixel slices. Each slice was rotated and flipped to generate several sets of image slices.

[0054] The training set image data is input into the image denoising model. The image denoising model is as follows: Figure 1 The loss function used in training is the L2 loss function, the training optimizer used is the Adam optimizer, and the initial value of the learning rate is set to 10 -3 Each batch of training data consists of 1000 40×40 image blocks. The training data is forward propagated to calculate the L2 loss with the clear image, and then the model parameters are updated through back propagation of the loss. After training 100 batches, the model parameters are saved. The specific implementation process is as follows Figure 2 shown.

[0055] After the image denoising model is trained, denoise the images in the datasets Set12 and BSD68. Input the noisy image into the trained image denoising model, load the saved model parameters, pass the calculation forward, and output the restored noise-free image. The specific application process is as follows: Figure 3 shown. Figure 4 Shows the image noise reduction effect when the noise value is 50. Figure 5 The average values ​​of PSNR and SSIM on the datasets Set12 and BSD68 are given. Figure 4 and Figure 5 It can be seen that the constructed image denoising model can quickly remove image noise and improve the visual effect of the image.

[0056] Example 3

[0057] Color image denoising. The color images used were from the DIV2K HR dataset. Each image was randomly sliced, and four 180×180 RGB images were selected as the training set. Before feeding the image data into the neural network model, the training set images were further preprocessed. The training set was scaled to four different degrees and then sliced ​​into 40×40 pixel slices. Each slice was rotated and flipped to obtain several sets of image slices.

[0058] The training set image data is input into the image denoising model. The image denoising model is as follows: Figure 1 The loss function used in training is the L2 loss function, the training optimizer used is the Adam optimizer, and the initial value of the learning rate is set to 10 -4 Each batch of training data consists of 128 40×40 image blocks. The training data is forward propagated to calculate the L2 loss with the clear image, and then the model parameters are updated through back propagation of the loss. After training 100 batches, the model parameters are saved. The specific implementation process is as follows Figure 2 shown.

[0059] After the image denoising model is trained, the images in the dataset CBSD68 are denoised. The noisy image is input into the trained image denoising model, the saved model parameters are loaded, the calculation is forwarded, and the restored noise-free image is output. The specific application process is as follows: Figure 3 shown. Figure 6 The image denoising effects of different noise values ​​are shown. Figure 6 It can be seen that the constructed image denoising model can quickly remove image noise and improve the visual effect of the image.

[0060] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An image denoising method based on improved spatial attention, characterized in that: The following steps are involved: Step 1: Take the original image and the corresponding noise image as a training sample to construct a training sample set; Step 2: Construct an image denoising model, which includes a basic feature layer for extracting local features of the noisy image and a spatial attention layer for extracting spatial feature information of the noisy image. The basic feature layer has 13 layers in total, including a convolutional feature extraction module. The spatial attention layer has 2 layers in total, including a traditional spatial attention extraction module and an additional spatial domain information extraction module. The spatial attention layer is divided into two parts: the first part is two groups of spatial feature extraction modules, and the second part is a group of spatial domain information extraction modules. The first part of the traditional spatial attention extraction module calculates the average and maximum values ​​of all channels to obtain two groups of information, and then convolves the two groups of information to obtain spatial attention information. The convolution kernel size of this part is 3×3 kernel, and the activation function uses ReLU function. The second part of the additional spatial domain information extraction module extracts the features of the local plane position of the tensor in each channel. In order to obtain the matrix of the local average value, the size of each local window is set to 8×8, and the average value within each window is calculated. This process is rewritten as follows: AND 21 = P(X), (1) Where P is the pooling function with a kernel size of 8, X is the input matrix, and since E 21 If the size of is not equal to the size of the input X, use the nearest interpolation function N to adjust the matrix E 21 The size of AND 22 = N(E 21 ), (2) Through equations (1) and (2), we get the matrix E 22 , Then the convolution layer after the ReLU function is used to convert the 64 feature maps into 1 channel as additional spatial information. Therefore, two sets of spatial information are obtained. Next, the two sets of spatial information are connected with the input feature X. In order to connect with the bottom layer, the convolution layer is used to convert the channel from 66 to 64. Finally, the output of the improved attention layer is obtained; Step 3: Use the training set to train the constructed image denoising model to obtain a trained image denoising model; Step 4: When applying, the noisy image is input into the trained image denoising model, and the corresponding denoised image is calculated and output.

2. The image denoising method based on improved spatial attention according to claim 1, characterized in that: In step 2, the basic feature layer consists of two groups of convolutional layers with a kernel size of 3. The basic feature layer maintains the number of channels as 64, and a short skip connection is added between the input and the last ReLU function.

3. The image denoising method based on improved spatial attention according to claim 1, characterized in that: In step 3, during the training process, the noise image is processed by the convolution layer, the basic feature layer, and the spatial attention layer in sequence. The loss is calculated based on the noise image, and then the loss is back-propagated to update the neural network weight parameters.

4. The image denoising method based on improved spatial attention according to claim 1, characterized in that: When training the image denoising model, the input layer size is 64×64; when performing image restoration operations, the input layer size is the actual size of the image; the loss function used in training is the L2 loss function, and the training optimizer used is the Adam optimizer. In the grayscale image denoising task, the initial value of the learning rate is set to 10 −3 ; In the color image denoising task, the initial value of the learning rate is 10 −4 ; Preprocess the images in the training set. For gray images, each batch of training data includes 1000 40×40 image blocks. For color images, each batch of training data includes 128 40×40 image blocks. The training data is forward propagated to calculate the L2 loss with the clear image, and then the model parameters are updated through backpropagation of the loss. After training 100 batches, the model parameters are saved.

5. The image denoising method based on improved spatial attention according to claim 1, characterized in that: In step 4, the noisy image is input into the trained image denoising model, the saved model parameters are loaded, the calculation is passed forward, and the restored noise-free image is output.

Images