Cascade discrete wavelet multiband decomposition attention image denoising method and device

Abstract

The invention discloses a cascaded discrete wavelet multiband decomposition attention image denoising method and device, and the method comprises the following steps: building a CDW-MDA image denoising network which employs an encoder-sub-network-decoder structure; inputting a noise image and extracting high and low frequency band characteristic patterns under multiple scales; global and local key feature information is enhanced; evaluating the importance of texture detail components and weighting noise features of different frequency bands; enhancing selective features of the processed high and low frequency band feature maps, and fusing the high and low frequency band feature maps; and converting the fused high-low frequency band feature map to obtain a residual image, and finally obtaining a target image. According to the image denoising method, clearer texture details can be generated, blurring is avoided, and more texture detail information is reserved in the restored image; noise can be reduced while image feature textures are kept, edge information is reserved, and a better visual effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and image denoising, and in particular to a cascaded discrete wavelet multi-band decomposition attention image denoising method and device. Background Art

[0002] Image noise can significantly reduce image quality, so eliminating image noise is crucial to improving the visual quality of images. Currently, the mainstream image denoising methods include model-based and learning-based methods.

[0003] Model-based image denoising methods use optimization strategies based on well-defined image priors or noise statistical models. These methods have good interpretability and strong generalization capabilities. They use various image prior information to achieve image denoising. Typical prior information includes smoothness in the image domain, sparsity in the transform domain, non-local self-similarity and low rank in the patch domain. However, model-based image denoising methods have the following disadvantages: (1) They require manual adjustment of parameters; (2) They require complex optimization algorithms to achieve the desired denoising effect; (3) They have poor ability to retain image edge detail features; and (4) They cannot remove complex real noise.

[0004] Image denoising methods based on deep learning have many advantages over traditional image denoising methods. First, these methods usually do not require manual hyperparameter setting or rely on human knowledge of image priors. Second, they perform better in removing both synthetic and real noise. The current general image denoising task architecture uses noisy images and clean target image pairs for training, and processes synthetic noisy images such as Additive White Gaussian Noise (AWGN) to achieve good visual quality. However, real image denoising remains a challenging task. Real image noise comes from processing steps in the camera system, such as mosaic, gamma correction, and compression diversification. In recent years, researchers have attempted to model real noise and construct denoising models based on convolutional neural networks (CNNs) to process real-world noisy images. The typical CNN real image denoising method chooses U-Net as the backbone architecture. The repeated downsampling operations in U-Net will cause serious damage to the high-frequency information of the image and cannot reconstruct high-quality images. Therefore, the introduction of Wavelet Transform (WT) can further improve the performance of image denoising. Wavelet transform has the functions of multi-resolution analysis and gradual subdivision in time and frequency domains, which can decompose feature maps into different frequencies for further processing; by separating high-frequency information in the image from low-frequency information, wavelet transform can greatly reduce the information loss during downsampling in CNN. In addition, wavelet transform is reversible and has been widely used in image processing. Many researchers have introduced wavelet transform into image restoration tasks. Discrete wavelet transform (DWT) can be easily converted into convolution form and compatible with CNN. However, image denoising methods based on deep learning still have the following disadvantages: (1) Fixed wavelet filter selection and limited decomposition levels may not capture signal features at all scales; (2) When the noise level is high or when dealing with complex real noise, the denoising performance may deteriorate because the noise may propagate through different sub-bands during the wavelet decomposition process; (3) Existing methods directly mix all frequency information or directly discard high-frequency information, resulting in artifacts and distortion caused by the interaction of different frequency information, and the inability to retain image edge detail features, especially in areas with complex textures and structures. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a cascaded discrete wavelet multi-band decomposition attention image denoising method and device, which combines the advantages of strong model interpretability and strong CNN learning ability, and builds an image denoising network model with an encoder-subnetwork-decoder structure to reconstruct high-quality images.

[0006] The present invention solves the above problems through the following technical solutions:

[0007] A cascaded discrete wavelet multi-band decomposition attention image denoising method, comprising the following steps:

[0008] Step A: building a cascaded discrete wavelet multi-band decomposition attention image denoising network, wherein the image denoising network adopts an encoder-subnetwork-decoder structure;

[0009] Step B: inputting the noisy image into the encoder of the image denoising network, and the encoder performs multi-scale cascaded discrete wavelet transform and convolution block operations on the input noisy image to extract high and low frequency band feature maps at multiple scales;

[0010] Step C: sending the high and low frequency band feature maps at multiple scales extracted from the encoder to each level of the subnetwork of the image denoising network to enhance the global and local key feature information of the high and low frequency band feature maps and suppress noise;

[0011] Step D: Decompose the high and low frequency band feature maps enhanced in step C through multi-band attention to evaluate the importance of texture detail components, and weight the noise characteristics of different frequency bands;

[0012] Step E: enhancing the selective features of the high and low frequency band feature maps processed in step D by fusion of multi-band selection features, improving the ability to remove noise of different scales, and then fusing the high and low frequency band feature maps;

[0013] Step F: Send the fused high and low frequency band feature maps to the decoder to obtain a residual image after convolution block and inverse discrete wavelet transform, and add the residual image to the input noise image to obtain the denoised target image.

[0014] As a further improvement, the image denoising network includes a multi-scale cascaded discrete wavelet transform structure, a multi-band feature enhancement module, a multi-band decomposition attention module and a multi-band selection feature fusion module, wherein a multi-scale cascaded discrete wavelet transform structure is arranged in the encoder and the decoder, two multi-band feature enhancement modules, a multi-band decomposition attention module and a multi-band selection feature fusion module are arranged in the sub-network, and the multi-band decomposition attention module is arranged between the two multi-band feature enhancement modules and the multi-band feature enhancement module.

[0015] As a further improvement, the multi-scale cascade discrete wavelet transform structure introduces discrete wavelet transform and inverse discrete wavelet transform, inserts an unpooled Conv+ReLU convolution block after each level of discrete wavelet transform convolution block, and uses Conv to predict the residual result for the last layer of the last discrete wavelet transform convolution block.

[0016] As a further improvement, in step B, the discrete wavelet transform convolution block decomposes the original image as a noise image into a combination of low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands and diagonal high-frequency sub-bands, so that the inverse discrete wavelet transform convolution block can completely reconstruct the target image.

[0017] As a further improvement, the method for decomposing the original image is: decomposing the input original image Y by a Haar wavelet composed of a low-pass filter and a high-pass filter;

[0018]

[0019] Among them, f LL is a low-pass filter, f LH 、f HL and f HH are three high-pass filters;

[0020] The method for reconstructing the target image is:

[0021] F DWT (Y)=[(f LL *Y) ↓2 ,(f LH *Y) ↓2 ,(f HL *Y) ↓2 ,(f HH *Y) ↓2 ]

[0022] =[Y LL ,Y LH ,Y HL ,Y HH ];

[0023] Y=F IDWT (Y LL ,Y LH ,Y HL ,Y HH );

[0024] Among them, F DWT (·) represents the discrete wavelet transform convolution block calculation, F IDWT (·) represents the inverse discrete wavelet transform convolution block calculation, * is the convolution operation, and ↓2 is the standard downsampling with a sampling factor of 2.

[0025] As a further improvement, the multi-band feature enhancement module is a dual-branch four-layer residual dense block, the output of each previous layer is connected to the output of the current layer using a jump connection and input to the next layer; in the high-frequency sub-band branch of the multi-band feature enhancement module, a small convolution kernel is used; in the low-frequency sub-band branch of the multi-band feature enhancement module, a large convolution kernel is used; batch normalization and pooling are removed from each convolution layer, and only the activation function ReLU is retained.

[0026] As a further improvement, in step C, the specific method is:

[0027] The features of high-frequency sub-band and low-frequency sub-band are input into the multi-band feature enhancement module.

[0028] The high-frequency sub-band feature map is processed by Conv and ReLU to obtain a high-frequency feature map, and the high-frequency feature map is fused to obtain a high-frequency local feature map. The high-frequency local feature map and the features of the high-frequency sub-band input are then added pixel by pixel and output.

[0029] Similarly, the low-frequency sub-band feature map is obtained after Conv and ReLU operations, and the low-frequency feature map is fused to obtain the low-frequency local feature map; then the low-frequency local feature map and the features of the low-frequency sub-band input are added pixel by pixel and output.

[0030] As a further improvement, in step D,

[0031] The multi-band decomposition attention module takes the absolute values ​​of the horizontal, vertical and diagonal components in the high-frequency sub-band enhanced by the multi-band feature enhancement module as weights, and adds the enhanced low-frequency sub-band components to obtain new features. By capturing the directional information along the horizontal, vertical and diagonal directions, three texture features are constructed.

[0032] As a further improvement, the step E specifically includes:

[0033] The multi-band selection feature fusion module receives inputs from multiple sub-bands carrying information of different scales and combines these multi-scale features with a pixel-level sum as L;

[0034] Then, global average pooling is performed on the spatial dimension of L to obtain the channel element feature vector S; the channel element feature vector S passes through the convolution layer to generate a feature vector Z; the feature vector Z passes through four parallel channel amplification convolution layers to obtain multi-scale feature descriptors V1, V2, V3 and V4, each with a dimension of 1×1×C; the application function generates attention activation operators S1, S2, S3 and S4 corresponding to V1, V2, V3 and V4, which are used to adaptively recalibrate the multi-scale features L2, L3 and L4; feature recalibration and aggregation to obtain output feature map O f .

[0035] At the same time, the present invention solves the above problems through the following technical solutions:

[0036] A cascaded discrete wavelet multi-band decomposition attention image denoising device, characterized in that it includes a multi-scale cascaded discrete wavelet transform structure, a multi-band feature enhancement module, a multi-band decomposition attention module and a multi-band selection feature fusion module, wherein the multi-band feature enhancement module includes two, namely a first multi-band feature enhancement module and a second multi-band feature enhancement module, and the first multi-band feature enhancement module, the multi-band decomposition attention module, the second multi-band feature enhancement module and the multi-band selection feature fusion module are arranged in sequence;

[0037] The multi-scale cascade discrete wavelet transform structure is used to receive a noisy image, perform a multi-scale cascade discrete wavelet transform and a convolution block operation on the input noisy image to extract high and low frequency band feature maps at multiple scales; and obtain a residual image by performing a convolution block and an inverse discrete wavelet transform on the high and low frequency band feature maps fused by the multi-band selection feature fusion module, and add the residual image to the input noise image to obtain a denoised image;

[0038] The first multi-band feature enhancement module is connected to the multi-scale cascade discrete wavelet transform structure, and is used to send the high and low frequency band feature maps at multiple scales extracted by the multi-scale cascade discrete wavelet transform structure to each level of the sub-network of the image denoising network, so as to enhance the global and local key feature information of the high and low frequency band feature maps and suppress noise;

[0039] The multi-band decomposition attention module is connected to the first multi-band feature enhancement module, and is used to evaluate the importance of texture detail components through multi-band decomposition attention on the high and low frequency band feature maps enhanced by the first multi-band feature enhancement module, and weight the noise features of different frequency bands;

[0040] The second multi-band feature enhancement module is connected to the multi-band decomposition attention module, and is used to enhance the selective features of the high and low frequency band feature maps processed by the multi-band decomposition attention module through multi-band selection feature fusion, so as to improve the ability to remove noise of different scales;

[0041] The multi-band selection feature fusion module is connected to the second multi-band feature enhancement module and is used to fuse the high and low frequency band feature maps processed by the second multi-band feature enhancement module.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] The image denoising method of the present invention mines high-frequency information more deeply, pays attention to the global and local information of the image, and retains more text details while removing noise, so that its average PSNR result is significantly better than the currently used image denoising method, can produce clearer texture details without blurring, and the restored image retains more texture detail information; it can also reduce noise while maintaining the characteristic texture of the image, and retain edge information, so as to have a better visual effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural schematic diagram of the CDW-MDA image denoising network of the present invention;

[0045] Figure 2 It is a schematic diagram of a multi-band feature enhancement module of the present invention;

[0046] Figure 3 A schematic diagram of a multi-band decomposition attention module of the present invention;

[0047] Figure 4 It is a schematic diagram of a multi-band selection feature fusion module of the present invention;

[0048] Figure 5 Schematic diagram of denoising results of different methods of the present invention on grayscale image dataset Set12;

[0049] Figure 6 It is a schematic diagram of denoising results of different methods of the present invention on the color image dataset CBSD68;

[0050] Figure 7 It is a schematic diagram of denoising results of different methods of the present invention on a real noisy image dataset SIDD. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example

[0053] A cascaded discrete wavelet multi-band decomposition attention image denoising method, comprising the following steps:

[0054] Step 1: Build a cascaded discrete wavelet multi-band decomposition attention image denoising network (CDW-MDA);

[0055] like Figure 1, which is a schematic diagram of the structure of the CDW-MDA image denoising network of this embodiment. The CDW-MDA image denoising network adopts an encoder-subnetwork-decoder structure.

[0056] The cascaded discrete wavelet multi-band decomposition attention image denoising network (CDW-MDA) includes a multi-scale cascaded discrete wavelet transform structure (MC-DWT), a multi-band feature enhancement (Mb-FE) module, a multi-band decomposition attention (Mb-DA) module and a multi-band selective feature fusion (Mb-SFF) module.

[0057] Among them, a multi-scale cascade discrete wavelet transform structure is set in the encoder and the decoder, two multi-band feature enhancement modules, a multi-band decomposition attention module and a multi-band selection feature fusion module are set in the sub-network, and the multi-band decomposition attention module is set between the two multi-band feature enhancement modules.

[0058] Different from the general encoder structure that uses pooling operation for downsampling operation, the multi-scale cascade discrete wavelet transform structure of the image denoising network introduces discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT) to reduce the high-frequency information loss caused by the downsampling operation. The decomposition level of DWT is 4; in the sub-network stage, a multi-band feature enhancement (Mb-FE) module, a multi-band decomposition attention (Mb-DA) module and a multi-band selection feature fusion (Mb-SFF) module are designed. The low noise characteristics of the multi-band feature enhancement (Mb-FE) module can make the multi-band decomposition attention (Mb-DA) attention ... (Mb-DA) attention attention (Mb-DA) attention (Mb-DA) attention attention (Mb-DA) attention (Mb-DA) attention attention (Mb-DA) attention The de-attention (Mb-DA) module better captures the high and low frequency features of the image. The high and low frequency feature attention of the multi-band decomposition attention (Mb-DA) module can help the multi-band feature enhancement (Mb-FE) module better distinguish between noise and edge details, and a multi-band feature enhancement (Mb-FE) module is superimposed after each level of the multi-band decomposition attention (Mb-DA) module to construct a sub-network. The multi-band selective feature fusion (Mb-SFF) module fuses multi-scale high and low frequency band features to enhance the selective features and improve the model's ability to remove noise of different scales.

[0059] In the encoder stage, the input noisy image I in ∈R H×W×C Cascaded discrete wavelet transform (DWT) convolution block operations are performed to obtain high and low frequency band features at different scales. The mathematical model expression of the entire encoder process is shown in (Equation 1)-(Equation 3):

[0060]

[0061]

[0062]

[0063]

[0064] Among them, P n Represents the extracted n-th level high-frequency sub-band and low-frequency sub-band feature map, I in ∈R H×W×C Represents the input noisy image, H, W and C represent the height, width and number of channels of the input image respectively. and They represent the n-th level DWT output and the convolution block output respectively. According to experimental results, in this embodiment, n is 4.

[0065] Then, the frequency band features P1, P2, P3 and P4 of different scales extracted from the encoder are respectively sent to the multi-band feature enhancement (Mb-FE) module at each level in the sub-network to enhance the global-local high- and low-frequency key feature information and suppress the influence of noise; the enhanced high-frequency sub-band and low-frequency sub-band feature maps are then sent to the multi-band decomposition attention (Mb-DA) module to capture the key details in the decomposed high-frequency sub-band and low-frequency sub-band, so as to fully improve the redundancy and richness of the spatial domain features, thereby effectively retaining important image edge detail features. Since the low noise characteristics of the multi-band feature enhancement (Mb-FE) module can enable the multi-band decomposition attention (Mb-DA) module to better capture the high and low frequency features of the image, and the high and low frequency feature attention of the multi-band decomposition attention (Mb-DA) module can help the multi-band feature enhancement (Mb-FE) module better distinguish between noise and edge details. Therefore, a multi-band feature enhancement (Mb-FE) module is superimposed after each level of the multi-band decomposition attention (Mb-DA) module to construct a sub-network and generate feature maps L1, L2, L3 and L4. Finally, L1, L2, L3 and L4 are sent to the multi-band selection feature fusion (Mb-SFF) module to fuse the multi-scale high and low frequency band features to obtain the feature map O f The mathematical model of the entire sub-network is expressed as (Formula 4)-(Formula 7):

[0066]

[0067]

[0068]

[0069]

[0070] O f =F Mb-SFF (L1,...,L4) (Formula 7);

[0071] Among them, F' Mb-FE (·) and F” Mb-FE(·) represents the output of the first and second Mb-FE of each stage, represents the output of the nth level Mb-DA, F Mb-SFF (·) indicates Mb-SFF operation, O f is the output feature map.

[0072] In the decoder, O f The prediction residual image I is obtained by the convolution block and the inverse discrete wavelet transform (IDWT) output res , will I res and I in Add together to get the denoised image I out The mathematical model of the entire decoder is expressed as (Equation 8)-(Equation 9):

[0073]

[0074] I out =I res +I in (Formula 9);

[0075] in, and Represent the n-th level convolution block and IDWT operation, I res is the residual image obtained after cascade convolution and IDWT, I out is the output image after denoising.

[0076] Step 2: The noise image is input into the encoder part of the multi-scale cascade discrete wavelet transform structure (MC-DWT), and multi-scale cascade discrete wavelet transform and convolution block operations are performed on the input noise image to extract high and low frequency band feature maps at multiple scales.

[0077] The multi-scale cascade discrete wavelet transform structure (MC-DWT) transforms a complex CNN training image denoising problem into a simple CNN training problem of multiple scale wavelet sub-bands. A 3×3Conv+ReLU convolution block is inserted after each level of DWT, which can reduce the number of channels of the feature map and improve the inter-band independence of the feature map. There is no pooling. For the last layer of the last convolution block, only 3×3Conv is used to predict the residual result.

[0078] In this embodiment, the scale is four, and the two-dimensional DWT can decompose the original image Y into low-frequency sub-bands Y LL , horizontal high frequency sub-band Y LH , vertical high frequency sub-band Y HL , diagonal high frequency sub-band Y HH The present invention uses a Haar wavelet composed of a low-pass filter and a high-pass filter to decompose the input image Y, as shown in (Formula 10):

[0079]

[0080] Among them, f LL is a low-pass filter, f LH 、f HL and f HH There are three high-pass filters.

[0081] Due to the orthogonality of the four filters, no information loss occurs during the downsampling process, which means that the target image can be completely reconstructed using IDWT. The calculation steps of two-dimensional DWT and IDWT are shown in (Equation 11)-(Equation 12):

[0082] F DWT (Y)=[(f LL *Y) ↓2 ,(f LH *Y) ↓2 ,(f HL *Y) ↓2 ,(f HH *Y) ↓2 ]

[0083] =[Y LL ,Y LH ,Y HL ,Y HH ] (Formula 11);

[0084] Y=F IDWT (Y LL ,Y LH ,Y HL ,Y HH ) (Formula 12);

[0085] Among them, F DWT (·) indicates DWT calculation, F IDWT (·) indicates IDWT calculation, * is the convolution operation, and ↓2 is the standard downsampling with a sampling factor of 2.

[0086] Figure 1 It shows that inserting a 3×3Conv+ReLU convolution block after each wavelet transform can reduce the number of channels of the feature map and improve the inter-band independence of the feature map. The number of channels is marked on the right side of each convolution block, and there is no pooling. For the last layer of the last convolution block, only 3×3Conv is used to predict the residual result.

[0087] Step 3: The high and low frequency band feature maps at multiple scales are sent to the multi-band feature enhancement (Mb-FE) module to enhance the global and local key feature information of the high and low frequency band feature maps and suppress the influence of noise.

[0088] In each sub-network stage of CDW-MDA, the Mb-FE module is used alternately twice. Specifically, the Mb-FE module can be regarded as a dual-branch four-layer residual dense block (RDB). The output of each previous layer is connected to the output of the current layer using a skip connection and input to the next layer to improve the performance of the network. In the Mb-FE high-frequency sub-band branch, a small 3×3 convolution kernel is used to better preserve the image detail information, while in the Mb-FE low-frequency sub-band branch, a larger 5×5 convolution kernel is used to obtain more contextual information, thereby better enhancing the low-frequency features. In addition, batch normalization (BN) and pooling are removed from each convolution layer, and only the activation function ReLU is retained to better preserve the detail features.

[0089] Figure 2 The figure schematically shows a multi-band feature enhancement (Mb-FE) module according to an embodiment of the present invention.

[0090] like Figure 2 As shown, let the feature inputs of the high-frequency sub-band and the low-frequency sub-band in the Mb-FE module be and The feature maps of the high-frequency sub-band after 3 layers of 3×3Conv and ReLU operations are and Fusion The local feature map after Such as (Formula 13):

[0091]

[0092] in, Indicates that the feature map Channel concatenation is performed, Conv represents a 1×1 convolution operation, which is used to fuse local features. The final output is the fused local feature map and high frequency subband input features Perform pixel-by-pixel addition, as shown in (Equation 14):

[0093]

[0094] Similarly, the feature maps of the low-frequency subband after 3 layers of 5×5Conv and ReLU operations are and Fusion The local feature map after Such as (Formula 15):

[0095]

[0096] in, Indicates that the feature map Channel concatenation is performed, Conv represents a 1×1 convolution operation, which is used to fuse local features. The final output is the fused local feature map and low frequency subband input features Perform pixel-by-pixel addition, as shown in (Equation 16):

[0097]

[0098] Step 4: The enhanced high and low frequency band feature maps are passed through the multi-band decomposition attention (Mb-DA) module to evaluate the importance of texture detail components and weight the noise features of different frequency bands.

[0099] The multi-band decomposition attention (Mb-DA) module takes the absolute values ​​of the horizontal (LH), vertical (HL) and diagonal (HH) components in the high-frequency sub-band enhanced by the Mb-FE module as weights, and adds the enhanced low-frequency sub-band component (LL) to obtain new features. By capturing the directional information along the LH, HL and HH directions, three texture features are constructed.

[0100] The high frequency sub-band enhanced by the Mb-FE module The absolute values ​​of the horizontal (LH), vertical (HL) and diagonal (HH) components in are used as weights and added to the enhanced low-frequency subband The new features are obtained by combining the LH, HL, and HH components. Three texture features are constructed by capturing the directional information along the LH, HL, and HH directions. These three texture features are represented as M, including and Specifically as (Formula 17):

[0101]

[0102] This paper designs a multi-band decomposition attention mechanism to evaluate each texture component ( and ) and obtain texture features.

[0103] Figure 3 A schematic diagram of a multi-band decomposition attention (Mb-DA) module according to an embodiment of the present invention is shown schematically.

[0104] like Figure 3 Based on Global Average Pooling (GAP), this paper adopts shared fully connected (FC), FC: c→c / r→ReLU→c to evaluate and The weights of the components, where c is the dimension of the feature vector and r is the reduction ratio. Softmax is used to normalize the weights of the three texture components. The specific multi-band decomposition attention mechanism is as follows (Equation 18):

[0105] MbD_att=Softmax([FC(GAP(M[i,:,:,:]))]) i=1,2,3 (Equation 18);

[0106] Obviously, MbD_att is the attention weight of M. The final texture feature is as follows (Formula 19):

[0107]

[0108] Among them, ⊙ is the dot multiplication operation.

[0109] Finally, the original feature LL and the generated texture feature are concatenated to obtain a more comprehensive and discriminative output feature representation, as shown in (Equation 20):

[0110] Output features=Concat(Textural features,LL) (Equation 20);

[0111] Step 5: Use the multi-band selective feature fusion (Mb-SFF) module to enhance the selective features of the processed high and low frequency band feature maps, improve the model's ability to remove noise of different scales, and then fuse the high and low frequency band feature maps;

[0112] The Multi-band Selective Feature Fusion (Mb-SFF) module receives inputs from four sub-band features with different scales to enhance the selective features.

[0113] Figure 4 The figure schematically shows a schematic diagram of a multi-band selective feature fusion (Mb-SFF) module according to an embodiment of the present invention.

[0114] like Figure 4 , Mb-SFF receives inputs from 4 sub-bands with different scale information, and first combines these multi-scale features with a pixel-level sum as formula (21):

[0115] L=L1+L2+L3+L4 (21)

[0116] Then, global average pooling (GAP) is performed on the spatial dimension of L to obtain the channel element feature vector S. S passes through a 1×1Conv and ReLU convolution layer to generate a compact feature vector Z. Z passes through four parallel channel magnification convolution layers (one for each scale) to obtain four feature descriptors V1, V2, V3 and V4, each with a dimension of 1×1×C. The Softmax function is applied to generate attention activation operators S1, S2, S3 and S4 corresponding to V1, V2, V3 and V4, which are used to adaptively recalibrate the multi-scale features L1, L2, L3 and L4. Features are recalibrated and aggregated to obtain the output feature O f The calculation process is as follows:

[0117] O f =S1·L1+S2·L2+S3·L3+S4·L4 (22)

[0118] Step 6: The fused high- and low-frequency band feature maps are sent to the decoder to obtain a residual image after convolution block and inverse discrete wavelet transform. The residual image is added to the input noise image to obtain the denoised image.

[0119] The fused feature map is passed through convolution blocks and IDWT to obtain a residual image, and the residual image is added to the input noise image to obtain a denoised image, forming a CDW-MDA image denoising network, which can reduce the loss of high-frequency information while effectively maintaining image detail information, showing better visual effects.

[0120] The fused feature map O f The predicted residual image I is obtained through the convolution block and IDWT output res , will I res And the input noise image I in Add together to get the denoised image I out , as in step 1, formula (8)-(9).

[0121] The experimental results of this embodiment are compared and analyzed with the currently used image denoising method:

[0122] The CDW-MDA image denoising network in this embodiment uses the deep learning framework of Pytorch1.8 and Cuda11.1. All experiments are carried out under the conditions of Ubuntu 18.04.5 version and two NVIIDIA GTX3080 GPUs. The batch size of real noise training and synthetic noise training is 16, and the patch size is 256. During the training process, the learning rate is initially set to 1e-4, and the cosine annealing strategy is used to steadily reduce the learning rate, and the L2 loss and Adam optimizer (β1=0.9, β2=0.999) are used to optimize the network parameters. The CDW-MDA image denoising network uses four different feature scales, with the number of channels being 64, 128, 256 and 512 respectively, the DWT decomposition level being 4, and the number of Mb-FE and Mb-DA at each level being 2 and 1 respectively.

[0123] Table 1 shows the average PSNR results of these methods on the Set12 and BSD68 grayscale image datasets. The bold and underlined indicators represent the first and second ranked indicators among the listed methods. It can be seen that the quantitative results of this method are mostly optimal or suboptimal. Under complex noise, the image denoising method of the present invention is superior to the current image denoising methods, mainly because the CDW-MDA network architecture is designed to dig deeper into high-frequency information.

[0124] Table 1 Comparison of average PSNR (dB) results of different methods on grayscale image datasets

[0125]

[0126]

[0127] Figure 5 A schematic diagram shows denoising results on the grayscale image dataset Set12 using different methods according to embodiments of the present invention.

[0128] Figure 5 The image "Barbara" from the Set12 dataset is compared with the visual denoising results of different methods under σ = 50. The numbers in the subtitles of the figures represent PSNR (dB). In comparison, this method can produce clearer texture details without blurring, and the restored image retains more texture detail information.

[0129] Table 2 shows the average PSNR results of these methods on the CBSD68 and McMaster color image datasets. It can be seen that this method achieves the best results on the McMaster dataset, with PSNR gains of 0.06dB, 0.09dB, and 0.05dB compared to DeamNet at three different noise levels. In general, this design method can achieve the highest PSNR in most cases, because the CDW-MDA network architecture focuses on both global and local information of the image.

[0130] Table 2 Comparison of average PSNR (dB) results of different methods on color image datasets

[0131]

[0132]

[0133] Figure 6 The figure schematically shows the denoising results of different methods according to the embodiments of the present invention on the color image dataset CBSD68.

[0134] Figure 6 The image from the CBSD68 dataset is compared with the visual denoising results of different methods when σ = 50. The numbers in the subtitles of the figures represent PSNR (dB). It can be seen from the denoised image and the enlarged local image that this method can reduce noise while maintaining the characteristic texture of the image.

[0135] Table 3 shows the quantitative results on the SIDD real noise dataset. It can be seen that this method achieves the highest PSNR and SSIM values ​​on RIDNet, HI-GAN, and VDIR, with PSNR gains of 0.64dB, 0.88db, and 0.06dB, respectively, and SSIM values ​​increased by 0.013, 0.018, and 0.01, respectively.

[0136] Table 3. Comparison of denoising performance of different methods on SIDD real noise dataset

[0137] method PSNR(dB) SSIM BM3D 25.65 0.685 DnCNN 23.66 0.583 FFDNet 29.20 0.594 CBDNet 30.78 0.801 RIDNet 38.71 0.905 HI-GAN 38.47 0.900 VDIR 39.29 0.908 This design method 39.35 0.918

[0138] Figure 7 The following is a schematic diagram showing denoising results of different methods according to embodiments of the present invention on a real noisy image dataset SIDD.

[0139] Figure 7The comparison of visual denoising results of different methods from the images in the SIDD dataset is shown. The numbers in the subtitles of the figures represent the PSNR (dB) / SSIM values. The visual effect of the traditional algorithm BM3D is slightly better than that of DnCNN, but the denoising effect and edge information retention capabilities of both are limited. The denoising effect of CBDNet and RIDNet is improved, but there are residual noise and artifacts. In contrast, this design method can retain more text details while removing noise, so it has better visual effects.

[0140] The image denoising method of the present invention mines high-frequency information more deeply, pays attention to the global and local information of the image, and retains more text details while removing noise, so that its average PSNR result is significantly better than the currently used image denoising method, can produce clearer texture details without blurring, and the restored image retains more texture detail information; it can also reduce noise while maintaining the characteristic texture of the image, and retain edge information, so as to have a better visual effect.

[0141] Although the present invention is described herein with reference to the illustrative embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A cascaded discrete wavelet multi-band decomposition attention image denoising method, characterized in that: The following steps are involved: Step A: building a cascaded discrete wavelet multi-band decomposition attention image denoising network, wherein the image denoising network adopts an encoder-subnetwork-decoder structure; Step B: inputting the noisy image into the encoder of the image denoising network, and the encoder performs multi-scale cascaded discrete wavelet transform and convolution block operations on the input noisy image to extract high and low frequency band feature maps at multiple scales; Step C: sending the high and low frequency band feature maps at multiple scales extracted from the encoder to each level of the subnetwork of the image denoising network to enhance the global and local key feature information of the high and low frequency band feature maps and suppress noise; Step D: Decompose the high and low frequency band feature maps enhanced in step C through multi-band attention to evaluate the importance of texture detail components, and weight the noise characteristics of different frequency bands; Step E: enhancing the selective features of the high and low frequency band feature maps processed in step D by fusion of multi-band selection features, improving the ability to remove noise of different scales, and then fusing the high and low frequency band feature maps; Step F: Send the fused high and low frequency band feature maps to the decoder to obtain a residual image after convolution block and inverse discrete wavelet transform, and add the residual image to the input noise image to obtain the denoised target image.

2. According to claim 1, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: The image denoising network includes a multi-scale cascade discrete wavelet transform structure, a multi-band feature enhancement module, a multi-band decomposition attention module and a multi-band selection feature fusion module, wherein a multi-scale cascade discrete wavelet transform structure is arranged in the encoder and the decoder, two multi-band feature enhancement modules, a multi-band decomposition attention module and a multi-band selection feature fusion module are arranged in the sub-network, and the multi-band decomposition attention module is arranged between the two multi-band feature enhancement modules and the multi-band feature enhancement module.

3. According to claim 2, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: The multi-scale cascade discrete wavelet transform structure introduces discrete wavelet transform and inverse discrete wavelet transform, inserts an unpooled Conv+ReLU convolution block after each level of discrete wavelet transform convolution block, and uses Conv to predict the residual result for the last layer of the last discrete wavelet transform convolution block.

4. According to claim 3, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: In step B, the discrete wavelet transform convolution block decomposes the original image as the noise image into a combination of low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands and diagonal high-frequency sub-bands, so that the inverse discrete wavelet transform convolution block can completely reconstruct the target image.

5. According to claim 4, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: The method for decomposing the original image is: decomposing the input original image Y by using a Haar wavelet composed of a low-pass filter and a high-pass filter; Among them, f LL is a low-pass filter, f LH 、f HL and f HH are three high-pass filters; The method for reconstructing the target image is: Y=F IDWT (AND LL ,AND LH ,AND HL ,AND HH ); Among them, F DWT (·) represents the discrete wavelet transform convolution block calculation, F IDWT (·) represents the inverse discrete wavelet transform convolution block calculation, * is the convolution operation, and ↓2 is the standard downsampling with a sampling factor of 2.

6. According to claim 2, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: The multi-band feature enhancement module is a dual-branch four-layer residual dense block. The output of each previous layer is connected to the output of the current layer using a jump connection and input to the next layer. A small convolution kernel is used in the high-frequency sub-band branch of the multi-band feature enhancement module; a large convolution kernel is used in the low-frequency sub-band branch of the multi-band feature enhancement module; batch normalization and pooling are removed from each convolution layer, and only the activation function ReLU is retained.

7. According to claim 6, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: In the step C, the specific method is: The features of high-frequency sub-band and low-frequency sub-band are input into the multi-band feature enhancement module. The high-frequency sub-band feature map is processed by Conv and ReLU to obtain a high-frequency feature map, and the high-frequency feature map is fused to obtain a high-frequency local feature map. The high-frequency local feature map and the features of the high-frequency sub-band input are then added pixel by pixel and output. Similarly, the low-frequency sub-band feature map is obtained after Conv and ReLU operations, and the low-frequency feature map is fused to obtain the low-frequency local feature map; then the low-frequency local feature map and the features of the low-frequency sub-band input are added pixel by pixel and output.

8. According to claim 2, a cascaded discrete wavelet multi-band decomposition attention image denoising method is characterized in that: In the step D, The multi-band decomposition attention module takes the absolute values ​​of the horizontal, vertical and diagonal components in the high-frequency sub-band enhanced by the multi-band feature enhancement module as weights, and adds the enhanced low-frequency sub-band components to obtain new features. By capturing the directional information along the horizontal, vertical and diagonal directions, three texture features are constructed.

9. A cascaded discrete wavelet multi-band decomposition attention image denoising method according to claim 2, characterized in that: The step E specifically includes: The multi-band selection feature fusion module receives inputs from multiple sub-bands carrying information of different scales and combines these multi-scale features with a pixel-level sum as L; Then, global average pooling is performed on the spatial dimension of L to obtain a channel element feature vector S; the channel element feature vector S is passed through a convolution layer to generate a feature vector Z; the feature vector Z is passed through four parallel channel amplification convolution layers to obtain multi-scale feature descriptors V1, V2, V3 and V4, each of which has a dimension of 1×1×C; the application function generates attention activation operators S1, S2, S3 and S4 corresponding to V1, V2, V3 and V4, which are used to adaptively recalibrate the multi-scale features L1, L2, L3 and L4; the features are recalibrated and aggregated to obtain the output feature map O f .

10. A cascaded discrete wavelet multi-band decomposition attention image denoising device, characterized in that: It includes a multi-scale cascade discrete wavelet transform structure, a multi-band feature enhancement module, a multi-band decomposition attention module and a multi-band selection feature fusion module, wherein the multi-band feature enhancement module includes two, namely a first multi-band feature enhancement module and a second multi-band feature enhancement module, and the first multi-band feature enhancement module, the multi-band decomposition attention module, the second multi-band feature enhancement module and the multi-band selection feature fusion module are arranged in sequence; The multi-scale cascade discrete wavelet transform structure is used to receive a noisy image, perform a multi-scale cascade discrete wavelet transform and a convolution block operation on the input noisy image to extract high and low frequency band feature maps at multiple scales; and obtain a residual image by performing a convolution block and an inverse discrete wavelet transform on the high and low frequency band feature maps fused by the multi-band selection feature fusion module, and add the residual image to the input noise image to obtain a denoised image; The first multi-band feature enhancement module is connected to the multi-scale cascade discrete wavelet transform structure, and is used to send the high and low frequency band feature maps at multiple scales extracted by the multi-scale cascade discrete wavelet transform structure to each level of the sub-network of the image denoising network, so as to enhance the global and local key feature information of the high and low frequency band feature maps and suppress noise; The multi-band decomposition attention module is connected to the first multi-band feature enhancement module, and is used to evaluate the importance of texture detail components through multi-band decomposition attention on the high and low frequency band feature maps enhanced by the first multi-band feature enhancement module, and weight the noise features of different frequency bands; The second multi-band feature enhancement module is connected to the multi-band decomposition attention module, and is used to enhance the selective features of the high and low frequency band feature maps processed by the multi-band decomposition attention module through multi-band selection feature fusion, so as to improve the ability to remove noise of different scales; The multi-band selection feature fusion module is connected to the second multi-band feature enhancement module and is used to fuse the high and low frequency band feature maps processed by the second multi-band feature enhancement module.

Images