Image denoising method and system based on multi-scale gating attention mechanism

Abstract

This invention relates to an image denoising method and system based on a multi-scale gated attention mechanism, belonging to the field of computer vision and image processing technology. The method includes: acquiring noisy-clean image pairs and constructing a training dataset; building a denoising network with an encoder-decoder U-Net structure, embedding a multi-scale gated feature extraction module in the encoder-decoder to adaptively fuse multi-scale features, designing a selective attention feedforward module at bottlenecks to perform global context modeling on some channels to reduce computational load, and adding a gated feature modulation module at skip connections to suppress redundant information and noise residue; after optimizing the model through a progressive training strategy, the input noisy image can output the denoising result. This invention achieves a PSNR of 39.57 dB and an SSIM of 0.957 on the SIDD dataset, significantly reducing computational overhead while ensuring denoising performance and detail recovery capabilities, achieving an effective balance between denoising accuracy and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to an image denoising method and system based on a multi-scale gating attention mechanism. Background Technology

[0002] Image denoising aims to recover a clear image that closely resembles the real scene from a noisy image, and is a fundamental problem in computer vision and image processing. Because images are easily affected by sensor noise, environmental interference, and compression errors during acquisition, transmission, and storage, varying degrees of degradation often occur, impacting the accuracy of subsequent tasks such as recognition, detection, and segmentation. Therefore, effectively removing noise while preserving edge, texture, and structural information as much as possible has always been an important research direction in the field of image denoising.

[0003] Existing image denoising methods mainly include traditional methods and deep learning methods. Traditional methods typically rely on priors such as filtering and sparse representations for denoising. While these methods offer some interpretability, they often suffer from details loss and edge blurring in complex noisy scenes. In recent years, deep learning methods have significantly improved image denoising performance through convolutional neural networks, attention mechanisms, and Transformer structures. However, to achieve higher restoration quality, existing methods usually employ deeper networks or more complex structures, leading to a significant increase in the number of parameters and computational costs.

[0004] Furthermore, while multi-scale feature extraction helps enhance the network's ability to express texture and structural information at different scales, existing methods often lack effective adaptive control mechanisms during multi-scale feature fusion, making it difficult to fully leverage the complementary effects between features at different scales. On the other hand, although traditional self-attention mechanisms can model global dependencies, their computational complexity is high, leading to significant computational and storage burdens in high-resolution image processing. Meanwhile, while skip connections between the encoder and decoder help preserve details, they may also directly transmit noise residues and redundant information to the reconstruction stage, affecting the final denoising effect. Summary of the Invention

[0005] The purpose of this invention is to provide an image denoising method and system based on a multi-scale gated attention mechanism, in order to solve the problems of existing image denoising networks lacking adaptive control during multi-scale feature fusion, having high computational complexity in global dependency modeling, and having easy transmission of noise redundancy information in encoder-decoder skip connections, thereby improving image detail recovery capability and denoising performance while reducing model computational overhead.

[0006] To achieve the above objectives, the technical solution of the present invention is: an image denoising method based on a multi-scale gated attention mechanism, comprising:

[0007] Multiple pairs of noisy and clean images are obtained, and data preprocessing and data augmentation are performed to form a training dataset.

[0008] A denoising network model based on a multi-scale gated attention mechanism is constructed. The denoising network model adopts an encoder-decoder U-Net structure. The encoder and decoder are designed with multi-scale gated feature extraction modules to extract features at different scales and perform adaptive fusion. A selective attention feedforward module is designed at the bottleneck to perform global context modeling on some channels. A gated feature modulation module is designed at the skip connection to modulate the skip connection features to suppress redundant information.

[0009] Set the training parameters and train the denoising network model using the training dataset;

[0010] The noisy image is input into the trained denoising network model, and the corresponding denoised image is output.

[0011] Furthermore, the implementation method of the denoising network model is as follows:

[0012] First, input image Shallow feature extraction is performed using a 3×3 convolutional layer to obtain the initial feature map. Where H, W, and C represent the height, width, and initial number of channels of the feature map, respectively; then the initial feature map F0 is input into the first-level encoded multi-scale gated feature extraction module to obtain the first-level encoded output feature map. Next, the feature map F1 is downsampled for the first time, halving its size and expanding the number of channels to 2C, resulting in the first downsampled output feature map. The feature map F2 is then input into the second-level encoded multi-scale gated feature extraction module to obtain the second-level encoded output feature map. Subsequently, feature map F3 is downsampled a second time, halving its size and expanding the number of channels to 4C, resulting in the second downsampled output feature map. Then, the feature map F4 is input into the selective attention feedforward module for global context modeling, and the output feature map of the selective attention feedforward module is obtained through residual connections. Then, feature map F5 is upsampled for the first time, increasing its size to twice its original size and reducing the number of channels to 2C, resulting in the first upsampled output feature map. Simultaneously, the second-level encoded output feature map F3 is input into the second-level gated feature modulation module to obtain the gated enhanced feature map G2, which is then fused with feature map F6 element-wise to obtain the fused output feature map. The feature map F7 is then input into the second-level decoding multi-scale gated feature extraction module to obtain the second-level decoding output feature map. Then, the feature map F8 is upsampled a second time to restore the feature map size to the input resolution and reduce the number of channels to C, resulting in the second upsampled output feature map. Simultaneously, the first-level encoded output feature map F1 is input into the first-level gated feature modulation module to obtain the gated enhanced feature map G1, which is then fused with feature map F9 element by element to obtain the fused output feature map. Next, the feature map F 10 Inputting the first-level decoding multi-scale gated feature extraction module yields the first-level decoding output feature map. Finally, the feature map F is processed by a 3×3 convolutional layer. 11 Mapping to image space yields the residual mapping map. And the residual mapping graph F is connected through global residual join. 12 With input image I noisy By adding elements one by one, the final denoised image I is obtained. denoised ,Right now .

[0013] Furthermore, the first-level coding multi-scale gated feature extraction module, the second-level coding multi-scale gated feature extraction module, the first-level decoding multi-scale gated feature extraction module, and the second-level decoding multi-scale gated feature extraction module have the same structure and are all implemented using a unified multi-scale gated feature extraction module.

[0014] Furthermore, the implementation method of the multi-scale gated feature extraction module is as follows:

[0015] The input features X are processed by layer normalization and a two-branch gated convolution module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability in both cases.

[0016]

[0017] Where LN() represents processing through layer normalization, DGCM() represents processing through a dual-branch gated convolution module, and GFEM() represents processing through a gated feedforward enhancement module. , , , , All are intermediate features, and X'' represents the output feature of the multi-scale gated feature extraction module.

[0018] Furthermore, the implementation method of the dual-branch gated convolution module is as follows:

[0019] The input feature X1 is channel-expanded through a 1×1 convolution to obtain the intermediate feature X. expand :

[0020]

[0021] Subsequently, deep convolutional modeling at different scales is performed on both branches, and the feature response is further enhanced by combining simplified gating and simplified channel attention mechanisms:

[0022]

[0023]

[0024] Where SG() represents a simplified gating mechanism, and SCA() represents a simplified channel attention mechanism. , These represent 3×3 depthwise convolution and 5×5 depthwise convolution, respectively.

[0025] Then, the two-branch features are subjected to learnable gating parameters. , Perform adaptive fusion:

[0026]

[0027] Where σ is the Sigmoid activation function, g is the learned gating parameter, and W is the gating weight generated by the Sigmoid activation function;

[0028] The simplified gating mechanism SG is implemented as follows:

[0029] Let the input features be It is divided into two parts, H1 and H2, along the channel dimension:

[0030]

[0031] in, Split represents a channel splitting operation;

[0032] Gating is achieved through element-wise multiplication to obtain the output features:

[0033]

[0034] Where ⊙ represents element-wise multiplication;

[0035] The implementation method of the Simplified Channel Attention (SCA) mechanism is as follows:

[0036] Let the input features be First, channel statistics are obtained through global average pooling:

[0037]

[0038] in, ;

[0039] Then, channel weights are generated using 1×1 convolution:

[0040]

[0041] Finally, channel-weighted summation is applied to the input features to obtain the output features:

[0042]

[0043] The processing procedure of the gated feedforward enhancement module is as follows:

[0044] For the input feature X3, it is first expanded through a 1×1 convolution and then activated by the GELU function to obtain the intermediate feature U, which is then divided into two parts, U1 and U2, along the channel dimension:

[0045]

[0046]

[0047] One branch models the data using a 3×3 convolution and the GELU activation function, while the other branch remains untouched. The two features are then fused through element-wise multiplication and mapped back to the original channel dimensions via a 1×1 convolution.

[0048]

[0049]

[0050] Where X4 represents the output characteristic of the gated feedforward enhancement module.

[0051] Furthermore, the selective attention feedforward module is implemented as follows:

[0052] The input features Y are processed by layer normalization and a dynamic channel-aware attention module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability.

[0053]

[0054] Where Y'' represents the output feature of the selective attention feedforward module, LN() indicates processing through layer normalization, DCAAM() indicates processing through the dynamic channel perception attention module, and GFEM() indicates processing through the gated feedforward enhancement module. , , , , All are intermediate features.

[0055] Furthermore, the implementation method of the dynamic channel-aware attention module is as follows:

[0056] For the input feature Y1, channel adaptive weighting and channel partitioning are first performed through a channel attention mechanism to obtain the feature Y participating in the attention calculation. 1a and features Y that were not involved in attention calculation 1b ,Right now:

[0057]

[0058]

[0059] Wherein, CA represents channel attention mechanism, and Split represents channel splitting operation;

[0060] Then, the feature Y involved in the attention calculation 1a The global context is modeled using a multi-scale linear attention mechanism (MLA), resulting in attention-enhanced features Y. 1c ,Right now:

[0061]

[0062] For feature Y that is not involved in attention calculation 1b The feature Y is obtained by preserving the identity mapping. 1d ,Right now:

[0063]

[0064] Then, Y 1c With Y 1d The concatenation is performed along the channel dimension, and the channels are blended using a 1×1 convolution to obtain the transformed feature Y2:

[0065]

[0066] The channel attention mechanism is implemented as follows:

[0067] For the input feature Y1, global average pooling (GAP) is applied, followed by two 1×1 convolutional layers and then a sigmoid activation function. Generate channel weight coefficients:

[0068]

[0069] Next, the input feature Y1 is weighted channel by channel using the aforementioned channel weight coefficients:

[0070]

[0071] The implementation method of the multi-scale linear attention mechanism is as follows:

[0072] For input feature Y 1a Local location encoding enhancement is performed to obtain precoded features:

[0073]

[0074] This represents a 3×3 depthwise convolution;

[0075] Then, for feature Y 1h Local activation enhancement is performed using depthwise convolution and the GELU activation function to obtain the gated guidance feature Y. 1i :

[0076]

[0077] Next, basic query, key, and value features are generated through 1×1 convolution:

[0078]

[0079] Subsequently, multi-scale aggregation is performed on the basic query, key, and value features, and the original scale and features at each aggregated scale are concatenated along the channel dimension to obtain the multi-scale joint representation:

[0080]

[0081] This represents a 1×1 grouped convolution;

[0082] Then, feature Y 1k The input linear attention computation unit yields the query matrix Q, key matrix K, and value matrix V, respectively. After mapping through a kernel function, linear attention computation is performed to obtain the global context features.

[0083]

[0084] Where ϕ() represents the ReLU nonlinear activation function;

[0085] At the same time, local enhancements are performed from the value features:

[0086]

[0087] The global context feature O1 and the local detail feature O2 are added together to obtain the fused feature:

[0088]

[0089] Then, channel projection is performed using 1×1 convolution to obtain the projection features:

[0090]

[0091] Finally, the projected feature Y 1m With gated guidance feature Y 1i Perform element-wise multiplication and output the final result through a 1×1 convolution:

[0092] .

[0093] Furthermore, the implementation method of the gated feature modulation module is as follows:

[0094] For the input feature Z, the feature distribution is first standardized through layer normalization.

[0095]

[0096] LN() indicates processing through layer normalization;

[0097] Then, the process is divided into two paths. One path performs feature extraction, where the features are expanded through 1×1 convolutions and processed through depthwise separable convolutions at different scales. Subsequently, the features are concatenated along the channel dimension and fused using 1×1 convolutions to obtain the multi-scale modulated feature Z. ms :

[0098]

[0099]

[0100]

[0101]

[0102] Split represents the channel splitting operation. , These represent 3×3 depthwise convolution and 5×5 depthwise convolution, respectively.

[0103] Another approach spatially compresses the input feature Z1 using global average pooling (GAP) and generates channel weights G using 1×1 convolution and a sigmoid activation function, which are then modulated with the multi-scale feature Z. ms Element-wise multiplication suppresses redundancy features:

[0104]

[0105] Where ⊙ represents element-wise multiplication, and σ is the Sigmoid activation function;

[0106] After gating modulation, the features are remapped using 1×1 convolution, and residual connections are used to add the modulated features to the original input features to obtain the final output of the module.

[0107] .

[0108] Furthermore, the denoising network model is trained using a gradient optimization algorithm and the model parameters are updated using a learning rate decay strategy. The training process adopts a progressive training method, which improves the model's adaptability to noise at different scales by adjusting the input image patch size and batch size. The loss function includes pixel reconstruction loss and structure preservation loss for constraining image edge information.

[0109] The present invention also provides an image denoising system based on a multi-scale gated attention mechanism, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the method described above.

[0110] Compared with existing technologies, this invention has the following advantages: This invention provides an image denoising method and system based on a multi-scale gated attention mechanism. By constructing a denoising network model based on a multi-scale gated attention mechanism, a multi-scale gated feature extraction module is introduced into the encoder and decoder to achieve adaptive fusion of features at different scales. A dynamic channel-aware attention module is set at the network bottleneck, performing attention calculation only on some channels, thereby reducing computational complexity while maintaining global modeling capability. A gated feature modulation module is introduced at skip connections to effectively suppress redundant information and noise residue. This invention reduces computational overhead while ensuring feature representation capability, achieving an effective balance between image denoising performance and computational efficiency. Attached Figure Description

[0111] Figure 1 This is a flowchart illustrating the implementation of the image denoising method based on a multi-scale gating mechanism provided in this embodiment of the invention.

[0112] Figure 2 This is a structural diagram of the denoising network model based on a multi-scale gating mechanism in an embodiment of the present invention;

[0113] Figure 3 This is a structural diagram of the multi-scale gated feature extraction module in an embodiment of the present invention;

[0114] Figure 4 This is a structural diagram of the dual-branch gated convolution module in an embodiment of the present invention;

[0115] Figure 5 This is a structural diagram of the simplified gating mechanism SG in an embodiment of the present invention;

[0116] Figure 6 This is a structural diagram of the simplified channel attention mechanism (SCA) in an embodiment of the present invention;

[0117] Figure 7 This is a structural diagram of the gated feedforward enhancement module in an embodiment of the present invention;

[0118] Figure 8 This is a structural diagram of the selective attention feedforward module in an embodiment of the present invention;

[0119] Figure 9 This is a structural diagram of the dynamic channel perception attention module in an embodiment of the present invention.

[0120] Figure 10 This is a structural diagram of the channel attention mechanism in an embodiment of the present invention.

[0121] Figure 11 This is a structural diagram of the multi-scale linear attention mechanism in an embodiment of the present invention.

[0122] Figure 12 This is a structural diagram of the gated feature modulation module in an embodiment of the present invention.

[0123] Figure 13 This is a comparison image of the denoising results of real images in the embodiments of the present invention.

[0124] Figure 14 This is a comparison chart of the denoising performance and model complexity of real images in the embodiments of the present invention. Detailed Implementation

[0125] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0126] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0127] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0128] This invention provides an image denoising method based on a multi-scale gated attention mechanism, comprising:

[0129] Multiple pairs of noisy and clean images are obtained, and data preprocessing and data augmentation are performed to form a training dataset.

[0130] A denoising network model based on a multi-scale gated attention mechanism is constructed. The denoising network model adopts an encoder-decoder U-Net structure. The encoder and decoder are designed with multi-scale gated feature extraction modules to extract features at different scales and perform adaptive fusion. A selective attention feedforward module is designed at the bottleneck to perform global context modeling on some channels. A gated feature modulation module is designed at the skip connection to modulate the skip connection features to suppress redundant information.

[0131] Set the training parameters and train the denoising network model using the training dataset;

[0132] The noisy image is input into the trained denoising network model, and the corresponding denoised image is output.

[0133] The following are specific implementation examples of the present invention.

[0134] like Figure 1 As shown, this embodiment provides an image denoising method based on a multi-scale gated attention mechanism, including the following steps:

[0135] Step 1: Obtain multiple pairs of noisy-clean images, perform data preprocessing and data augmentation to form a training dataset.

[0136] In this embodiment, noisy-clean image pairs are obtained from the existing publicly available SIDD (Smartphone Image Denoising Dataset) dataset as training data. Then, data preprocessing and data augmentation are performed, including cropping the images into small patches and randomly flipping these patches horizontally and vertically to enhance the training data, ultimately forming the training dataset.

[0137] Step 2: Construct a denoising network model based on a multi-scale gated attention mechanism. The denoising network model adopts a U-Net structure, with multi-scale gated feature extraction modules designed at the encoder and decoder, a selective attention feedforward module designed at the bottleneck, and a gated feature modulation module designed at the skip connection.

[0138] The structure of the denoising network model is as follows: Figure 2 As shown, its specific implementation method is as follows:

[0139] First, input image Shallow feature extraction is performed using a 3×3 convolutional layer to obtain the initial feature map. Where H, W, and C represent the height, width, and initial number of channels of the feature map, respectively; then the initial feature map F0 is input into the first-level encoded multi-scale gated feature extraction module to obtain the first-level encoded output feature map. Next, the feature map F1 is downsampled for the first time, halving its size and expanding the number of channels to 2C, resulting in the first downsampled output feature map. The feature map F2 is then input into the second-level encoded multi-scale gated feature extraction module to obtain the second-level encoded output feature map. Subsequently, feature map F3 is downsampled a second time, halving its size and expanding the number of channels to 4C, resulting in the second downsampled output feature map. Then, the feature map F4 is input into the selective attention feedforward module for global context modeling, and the output feature map of the selective attention feedforward module is obtained through residual connections. Then, feature map F5 is upsampled for the first time, increasing its size to twice its original size and reducing the number of channels to 2C, resulting in the first upsampled output feature map. Simultaneously, the second-stage output feature map F3 from the encoding end is input into the second-stage gated feature modulation module to obtain the gated enhanced feature map G2, which is then fused with feature map F6 element-wise to obtain the fused output feature map. The feature map F7 is then input into the second-level decoding multi-scale gated feature extraction module to obtain the second-level decoding output feature map. Then, the feature map F8 is upsampled a second time to restore the feature map size to the input resolution and reduce the number of channels to C, resulting in the second upsampled output feature map. Simultaneously, the first-stage output feature map F1 from the encoding end is input into the first-stage gated feature modulation module to obtain the gated enhanced feature map G1, which is then fused with feature map F9 element by element to obtain the fused output feature map. Next, the feature map F 10 Inputting the first-level decoding multi-scale gated feature extraction module yields the first-level decoding output feature map. Finally, the feature map F is processed by a 3×3 convolutional layer. 11 Mapping to image space yields the residual mapping map. And the residual mapping graph F is connected through global residual join. 12 With input image I noisy By adding elements one by one, the final denoised image I is obtained. denoised ,Right now .

[0140] The first-level coding multi-scale gated feature extraction module, the second-level coding multi-scale gated feature extraction module, the first-level decoding multi-scale gated feature extraction module, and the second-level decoding multi-scale gated feature extraction module have the same structure and are all implemented using a unified multi-scale gated feature extraction module.

[0141] The structure of the multi-scale gated feature extraction module is as follows: Figure 3 As shown, its specific implementation method is as follows:

[0142] The input features X are processed by layer normalization and a two-branch gated convolution module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability in both cases.

[0143]

[0144] Where DGCM() represents processing through a dual-branch gated convolution module, GFEM() represents processing through a gated feedforward enhancement module, and X'' represents the output feature of the multi-scale gated feature extraction module.

[0145] like Figure 4 As shown, the processing procedure of the dual-branch gated convolution module is as follows:

[0146] The input feature X1 is channel-expanded using a 1×1 convolution to obtain the intermediate feature X. expand :

[0147]

[0148] Subsequently, deep convolutional modeling at different scales is performed on both branches, and the feature response is further enhanced by combining simplified gating and simplified channel attention mechanisms:

[0149]

[0150]

[0151] Here, SG() represents a simplified gating mechanism, and SCA() represents a simplified channel attention mechanism.

[0152] Then, the two branches of features are adaptively fused using learnable gating parameters:

[0153]

[0154] Where σ is the Sigmoid activation function, g is the learned gating parameter, and W is the gating weight generated by the Sigmoid activation function.

[0155] like Figure 5As shown, the implementation method of the simplified gating mechanism SG is as follows:

[0156] Let the input features be It is divided into two parts, H1 and H2, along the channel dimension:

[0157]

[0158] in, .

[0159] Gating is achieved through element-wise multiplication to obtain the output features:

[0160]

[0161] Here, ⊙ represents element-wise multiplication.

[0162] like Figure 6 As shown, the implementation method of the simplified channel attention mechanism (SCA) is as follows:

[0163] Let the input features be First, channel statistics are obtained through global average pooling:

[0164]

[0165] in, .

[0166] Then, channel weights are generated using 1×1 convolution:

[0167]

[0168] Finally, channel-weighted summation is applied to the input features to obtain the output features:

[0169]

[0170] Here, ⊙ represents element-wise multiplication.

[0171] like Figure 7 As shown, the processing procedure of the gated feedforward enhancement module is as follows:

[0172] For the input feature X3, it is first expanded through a 1×1 convolution and then activated by the GELU function to obtain the intermediate feature U, which is then divided into two parts, U1 and U2, along the channel dimension:

[0173]

[0174]

[0175] One branch models the data using a 3×3 convolution and the GELU activation function, while the other branch remains untouched. The two features are then fused through element-wise multiplication and mapped back to the original channel dimensions via a 1×1 convolution.

[0176]

[0177]

[0178] Where X4 represents the output feature of the gated feedforward enhancement module, GELU represents the activation function, U is the expanded intermediate feature, U1 and U2 represent two feature subspaces respectively, ⊙ represents element-wise multiplication, Split represents channel splitting operation, and Conv represents convolution operation.

[0179] The structure of the selective attention feedforward module is as follows: Figure 8 As shown, its specific implementation method is as follows:

[0180] The input features Y are processed by layer normalization and a dynamic channel-aware attention module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability.

[0181]

[0182] Where Y'' represents the output feature of the selective attention feedforward module, DCAAM() represents processing through the dynamic channel perception attention module, and GFEM() represents processing through the gated feedforward enhancement module. The specific implementation method is the same as that of the gated feedforward enhancement module in the aforementioned multi-scale gated feature extraction module, and will not be repeated here.

[0183] like Figure 9 As shown, the processing procedure of the dynamic channel perception attention module is as follows:

[0184] For the input feature Y1, channel adaptive weighting and channel partitioning are first performed through a channel attention mechanism to obtain the feature Y participating in the attention calculation. 1a and features Y that were not involved in attention calculation 1b ,Right now:

[0185]

[0186]

[0187] Then, the feature Y involved in the attention calculation 1a A multi-scale linear attention mechanism is used to model the global context, resulting in the attention-enhanced feature Y. 1c ,Right now:

[0188]

[0189] For feature Y that is not involved in attention calculation 1b The feature Y is obtained by preserving the identity mapping. 1d ,Right now:

[0190]

[0191] Subsequently, the attention-enhancing feature Y 1c Features Y that were not involved in attention calculation 1d The concatenation is performed along the channel dimension, and the channels are blended using a 1×1 convolution to obtain the transformed feature Y2:

[0192]

[0193] like Figure 10 As shown, the processing procedure of the channel attention mechanism is as follows:

[0194] For the input feature Y1, global average pooling is performed, followed by two 1×1 convolutional layers, and then channel weight coefficients are generated by the Sigmoid activation function.

[0195]

[0196] Next, the input feature Y1 is weighted channel by channel using the aforementioned channel weight coefficients:

[0197]

[0198] like Figure 11 As shown, the processing procedure of the multi-scale linear attention mechanism is as follows:

[0199] For input feature Y 1a Local location encoding enhancement is performed to obtain precoded features:

[0200]

[0201] Then, for feature Y 1h Local activation enhancement is performed using depthwise convolution and the GELU activation function to obtain the gated guidance feature Y. 1i :

[0202]

[0203] Next, basic query, key, and value features are generated through 1×1 convolution:

[0204]

[0205] Subsequently, multi-scale aggregation is performed on the basic query, key, and value features, and the original scale and features at each aggregated scale are concatenated along the channel dimension to obtain a multi-scale joint representation:

[0206]

[0207] Then, feature Y 1k The input linear attention computation unit yields the query matrix Q, key matrix K, and value matrix V, respectively. After mapping through a kernel function, linear attention computation is performed to obtain the global context features.

[0208]

[0209] Where ϕ() represents the ReLU nonlinear activation function.

[0210] At the same time, local enhancements are performed from the value features:

[0211]

[0212] The global context feature O1 and the local detail feature O2 are added together to obtain the fused feature:

[0213]

[0214] Then, channel projection is performed using 1×1 convolution to obtain the projection features:

[0215]

[0216] Finally, the projected feature Y 1m With gated guidance feature Y 1i Perform element-wise multiplication and output the final result through a 1×1 convolution:

[0217]

[0218] The structure of the gated feature modulation module is as follows: Figure 12 As shown, its specific implementation method is as follows:

[0219] For the input feature Z, the feature distribution is first standardized through layer normalization.

[0220]

[0221] Then, the process is divided into two paths. One path performs feature extraction, where the features are expanded through 1×1 convolutions and processed through depthwise separable convolutions at different scales. Subsequently, the features are concatenated along the channel dimension and fused using 1×1 convolutions to obtain the multi-scale modulated feature Z. ms :

[0222]

[0223]

[0224]

[0225]

[0226] Another approach spatially compresses the input feature Z1 using global average pooling, and generates channel weights G using 1×1 convolution and a sigmoid activation function, which are then modulated with the multi-scale feature Z. ms Element-wise multiplication suppresses redundancy features:

[0227]

[0228] After gating modulation, the features are remapped using 1×1 convolution, and residual connections are used to add the modulated features to the original input features to obtain the final output of the module.

[0229]

[0230] Step 3: Set the training parameters and train the denoising network model using the training dataset;

[0231] In this embodiment, the denoising network model is optimized using the AdamW algorithm, where parameters β1=0.9, β2=0.999, and the weight decay coefficient is 1×10⁻⁶. -4 The initial learning rate is set to 3×10. -4 The cosine annealing strategy was used to reduce the temperature to 1×10. -6 The denoising network model employs a progressive training method, with the batch size decreasing progressively according to {16, 8, 4} and the patch size increasing progressively according to {128, 176, 256} to adapt to noise at different scales and improve the robustness of the denoising network model. The loss function L of the denoising network model is expressed as follows:

[0232]

[0233] Among them, L char ( ) represents the Charbonnier loss function, L edge Denotes the marginal loss function; λ edge The weight coefficients representing the edge loss function are set to 0.1 in this embodiment based on experience; denoised I represents a denoised image. groundTruth This indicates a clean image.

[0234] Among them, the Charbonnier loss function L charThe calculation formula is as follows:

[0235]

[0236] Edge loss function L edge The calculation formula is as follows:

[0237]

[0238] in, Represents Laplace operation.

[0239] Step 4: Input the noisy image into the trained denoising network model to denoise the image and obtain the denoised image.

[0240] The image denoising method proposed in this invention will be verified from both quantitative and qualitative perspectives using a test set.

[0241] This embodiment performs quantitative error analysis on the test set images of SIDD. Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are used as the main objective evaluation indicators. PSNR is measured in decibels (dB); a higher PSNR value indicates better image denoising performance. SSIM values ​​range from 0 to 1; a value closer to 1 indicates higher overall similarity between two images. PSNR and SSIM are often used together as objective evaluation indicators, and their calculation formulas are as follows:

[0242]

[0243]

[0244]

[0245] Where m and n represent the number of rows and columns of the image, respectively; I(i,j) and K(i,j) represent the gray values ​​at pixel (i,j) in the original image and the denoised image, respectively; MAX represents the maximum possible value of a pixel (e.g., 255 for an 8-bit image); and MSE represents the mean square error. and These represent the mean values ​​(brightness) of the image at x and y, respectively. and These represent variance (contrast). This represents covariance (structural correlation). and Use constants to avoid zero denominators.

[0246] The experimental results obtained by testing the images in the test set using different methods are shown in Table 1:

[0247] Table 1. Comparison of PSNR and SSIM of different image denoising methods on the test set.

[0248]

[0249] As can be seen from the results in Table 1, the image denoising method proposed in this invention achieved a peak signal-to-noise ratio of 39.57 dB and a structural similarity of 0.957. Compared with the denoising algorithms in comparison, these values ​​indicate that the image denoising method proposed in this invention is more effective.

[0250] From a qualitative perspective, Figure 13 The visual effect comparison between the present invention and the contrasting denoising algorithm is shown. It can be seen that the present invention can preserve the overall structure of the image and restore relatively clear texture and edge details.

[0251] From the perspective of model complexity, Figure 14 The performance and model complexity of MSGAN and the comparison methods on the SIDD validation set are shown. It can be seen that although MSGAN has moderate model complexity, it significantly outperforms the other comparison methods in denoising, especially in balancing PSNR and runtime.

[0252] This embodiment also provides an image denoising system based on a multi-scale gated attention mechanism, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-described method.

[0253] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0254] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0255] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0256] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0257] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An image denoising method based on a multi-scale gated attention mechanism, characterized in that, include: Multiple pairs of noisy and clean images are obtained, and data preprocessing and data augmentation are performed to form a training dataset. A denoising network model based on a multi-scale gated attention mechanism is constructed. The denoising network model adopts an encoder-decoder U-Net structure. The encoder and decoder are designed with multi-scale gated feature extraction modules to extract features at different scales and perform adaptive fusion. A selective attention feedforward module is designed at the bottleneck to perform global context modeling on some channels. A gated feature modulation module is designed at the skip connection to modulate the skip connection features to suppress redundant information. Set the training parameters and train the denoising network model using the training dataset; The noisy image is input into the trained denoising network model, and the corresponding denoised image is output.

2. The image denoising method based on multi-scale gated attention mechanism according to claim 1, characterized in that, The implementation method of the denoising network model is as follows: First, input image Shallow feature extraction is performed using a 3×3 convolutional layer to obtain the initial feature map. Where H, W, and C represent the height, width, and initial number of channels of the feature map, respectively; then the initial feature map F0 is input into the first-level encoded multi-scale gated feature extraction module to obtain the first-level encoded output feature map. Next, the feature map F1 is downsampled for the first time, halving its size and expanding the number of channels to 2C, resulting in the first downsampled output feature map. The feature map F2 is then input into the second-level encoded multi-scale gated feature extraction module to obtain the second-level encoded output feature map. Subsequently, feature map F3 is downsampled a second time, halving its size and expanding the number of channels to 4C, resulting in the second downsampled output feature map. Then, the feature map F4 is input into the selective attention feedforward module for global context modeling, and the output feature map of the selective attention feedforward module is obtained through residual connections. Then, feature map F5 is upsampled for the first time, increasing its size to twice its original size and reducing the number of channels to 2C, resulting in the first upsampled output feature map. Simultaneously, the second-level encoded output feature map F3 is input into the second-level gated feature modulation module to obtain the gated enhanced feature map G2, which is then fused with feature map F6 element-wise to obtain the fused output feature map. The feature map F7 is then input into the second-level decoding multi-scale gated feature extraction module to obtain the second-level decoding output feature map. Then, the feature map F8 is upsampled a second time to restore the feature map size to the input resolution and reduce the number of channels to C, resulting in the second upsampled output feature map. Simultaneously, the first-level encoded output feature map F1 is input into the first-level gated feature modulation module to obtain the gated enhanced feature map G1, which is then fused with feature map F9 element by element to obtain the fused output feature map. Next, the feature map F 10 Inputting the first-level decoding multi-scale gated feature extraction module yields the first-level decoding output feature map. Finally, the feature map F is processed by a 3×3 convolutional layer. 11 Mapping to image space yields the residual mapping map. And the residual mapping graph F is connected through global residual join. 12 With input image I noisy By adding elements one by one, the final denoised image I is obtained. denoised ,Right now .

3. The image denoising method based on multi-scale gated attention mechanism according to claim 2, characterized in that, The first-level coding multi-scale gated feature extraction module, the second-level coding multi-scale gated feature extraction module, the first-level decoding multi-scale gated feature extraction module, and the second-level decoding multi-scale gated feature extraction module have the same structure and are all implemented using a unified multi-scale gated feature extraction module.

4. The image denoising method based on multi-scale gated attention mechanism according to claim 3, characterized in that, The implementation method of the multi-scale gated feature extraction module is as follows: The input features X are processed by layer normalization and a two-branch gated convolution module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability in both cases. Where LN() represents processing through layer normalization, DGCM() represents processing through a dual-branch gated convolution module, and GFEM() represents processing through a gated feedforward enhancement module. , , , , All are intermediate features, and X'' represents the output feature of the multi-scale gated feature extraction module.

5. The image denoising method based on multi-scale gated attention mechanism according to claim 4, characterized in that, The implementation method of the dual-branch gated convolution module is as follows: The input feature X1 is channel-expanded through a 1×1 convolution to obtain the intermediate feature X. expand : Subsequently, deep convolutional modeling at different scales is performed on both branches, and the feature response is further enhanced by combining simplified gating and simplified channel attention mechanisms: Where SG() represents a simplified gating mechanism, and SCA() represents a simplified channel attention mechanism. , These represent 3×3 depthwise convolution and 5×5 depthwise convolution, respectively. Then, the two-branch features are subjected to learnable gating parameters. , Perform adaptive fusion: Where σ is the Sigmoid activation function, g is the learned gating parameter, and W is the gating weight generated by the Sigmoid activation function; The simplified gating mechanism SG is implemented as follows: Let the input features be It is divided into two parts, H1 and H2, along the channel dimension: in, Split represents a channel splitting operation; Gating is achieved through element-wise multiplication to obtain the output features: Where ⊙ represents element-wise multiplication; The implementation method of the Simplified Channel Attention (SCA) mechanism is as follows: Let the input features be First, channel statistics are obtained through global average pooling: in, ; Then, channel weights are generated using 1×1 convolution: Finally, channel-weighted summation is applied to the input features to obtain the output features: The processing procedure of the gated feedforward enhancement module is as follows: For the input feature X3, it is first expanded through a 1×1 convolution and then activated by the GELU function to obtain the intermediate feature U, which is then divided into two parts, U1 and U2, along the channel dimension: One branch models the data using a 3×3 convolution and the GELU activation function, while the other branch remains untouched. The two features are then fused through element-wise multiplication and mapped back to the original channel dimensions via a 1×1 convolution. Where X4 represents the output characteristic of the gated feedforward enhancement module.

6. The image denoising method based on multi-scale gated attention mechanism according to claim 1, characterized in that, The selective attention feedforward module is implemented as follows: The input features Y are processed by layer normalization and a dynamic channel-aware attention module, and the output features are processed by layer normalization and a gated feedforward enhancement module. Residual connections are used to improve training stability. Where Y'' represents the output feature of the selective attention feedforward module, LN() indicates processing through layer normalization, DCAAM() indicates processing through the dynamic channel perception attention module, and GFEM() indicates processing through the gated feedforward enhancement module. , , , , All are intermediate features.

7. The image denoising method based on multi-scale gated attention mechanism according to claim 6, characterized in that, The implementation method of the dynamic channel-aware attention module is as follows: For the input feature Y1, channel adaptive weighting and channel partitioning are first performed through a channel attention mechanism to obtain the feature Y participating in the attention calculation. 1a and features Y that were not involved in attention calculation 1b ,Right now: Where CA represents channel attention mechanism and Split represents channel splitting operation; Then, the feature Y involved in the attention calculation 1a The global context is modeled using a multi-scale linear attention mechanism (MLA), resulting in attention-enhanced features Y. 1c ,Right now: For feature Y that is not involved in attention calculation 1b The feature Y is obtained by preserving the identity mapping. 1d ,Right now: Then, Y 1c With Y 1d The concatenation is performed along the channel dimension, and the channels are blended using a 1×1 convolution to obtain the transformed feature Y2: The channel attention mechanism is implemented as follows: For the input feature Y1, global average pooling (GAP) is applied, followed by two 1×1 convolutional layers and then a sigmoid activation function. Generate channel weight coefficients: Next, the input feature Y1 is weighted channel by channel using the aforementioned channel weight coefficients: The implementation method of the multi-scale linear attention mechanism is as follows: For input feature Y 1a Local location encoding enhancement is performed to obtain precoded features: This represents a 3×3 depthwise convolution; Then, for feature Y 1h Local activation enhancement is performed using depthwise convolution and the GELU activation function to obtain the gated guidance feature Y. 1i : Next, basic query, key, and value features are generated through 1×1 convolution: Subsequently, multi-scale aggregation is performed on the basic query, key, and value features, and the original scale and features at each aggregated scale are concatenated along the channel dimension to obtain the multi-scale joint representation: This represents a 1×1 grouped convolution; Then, feature Y 1k The input linear attention computation unit yields the query matrix Q, key matrix K, and value matrix V, respectively. After mapping through a kernel function, linear attention computation is performed to obtain the global context features. Where ϕ() represents the ReLU nonlinear activation function; At the same time, local enhancements are performed from the value features: The global context feature O1 and the local detail feature O2 are added together to obtain the fused feature: Then, channel projection is performed using 1×1 convolution to obtain the projection features: Finally, the projected feature Y 1m With gated guidance feature Y 1i Perform element-wise multiplication and output the final result through a 1×1 convolution: 。 8. The image denoising method based on multi-scale gated attention mechanism according to claim 1, characterized in that, The implementation method of the gated feature modulation module is as follows: For the input feature Z, the feature distribution is first standardized through layer normalization. LN() indicates processing through layer normalization; Then, the process is divided into two paths. One path performs feature extraction, where the features are expanded through 1×1 convolutions and processed through depthwise separable convolutions at different scales. Subsequently, the features are concatenated along the channel dimension and fused using 1×1 convolutions to obtain the multi-scale modulated feature Z. ms : Split represents the channel splitting operation. , These represent 3×3 depthwise convolution and 5×5 depthwise convolution, respectively. Another approach spatially compresses the input feature Z1 using global average pooling (GAP) and generates channel weights G using 1×1 convolution and a sigmoid activation function, which are then modulated with the multi-scale feature Z. ms Element-wise multiplication suppresses redundancy features: Where ⊙ represents element-wise multiplication, and σ is the Sigmoid activation function; After gating modulation, the features are remapped using 1×1 convolution, and residual connections are used to add the modulated features to the original input features to obtain the final output of the module. 。 9. The image denoising method based on multi-scale gated attention mechanism according to claim 1, characterized in that, The denoising network model is trained using a gradient optimization algorithm and its parameters are updated using a learning rate decay strategy. The training process employs a progressive training method, adjusting the input image patch size and batch size to improve the model's adaptability to noise at different scales. The loss function includes pixel reconstruction loss and structure preservation loss for constraining image edge information.

10. An image denoising system based on a multi-scale gated attention mechanism, characterized in that, It includes a memory, a processor, and computer program instructions stored in the memory and executable by the processor, wherein when the processor executes the computer program instructions, it can implement the method as described in any one of claims 1-9.

Images