Image denoising method based on efficient information distillation and multi-scale feature fusion

Abstract

The present invention provides an image denoising method based on efficient information distillation and multi-scale feature fusion. The method constructs an image denoising network, including an encoder and a decoder, with skip connections between the encoder and decoder. A multi-focus distillation module is introduced at each stage of the encoder and decoder. The image denoising method uses the encoder to extract image features from the preprocessed image, and uses the decoder to restore image details layer by layer, combining the encoder features through skip connections. Before downsampling at each encoder stage and after upsampling at each decoder stage, the multi-focus distillation module is used to sequentially perform information distillation, multi-scale feature extraction, layer normalization, and image information fusion using a high-frequency and contrast attention mechanism to achieve image denoising. The present invention can maintain image details and quality in complex environments, thereby improving the quality of subsequent image processing after image denoising.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to an image denoising method based on efficient information distillation and multi-scale feature fusion. Background Art

[0002] Image denoising is a fundamental and critical task in image processing. Its goal is to remove noise from corrupted images while preserving the details and structure of the original image as much as possible. With the widespread use of digital images in various applications such as medical imaging, satellite image interpretation, video communications, and digital photography, efficient and accurate image denoising techniques have become increasingly important.

[0003] Traditional image denoising methods, such as Gaussian filtering, median filtering, and wavelet transforms, rely on predefined mathematical models. These methods perform well with simple noise models, but their effectiveness is often limited when faced with complex or unknown noise types. This is particularly true for high ISO noise, salt and pepper noise, and noise introduced by image sensors and transmission processes. These traditional methods struggle to achieve satisfactory denoising results.

[0004] In recent years, deep learning technology has brought new possibilities to image denoising. Methods based on convolutional neural networks (CNNs), in particular, have shown great potential in automatically learning complex patterns in images. However, while existing deep learning image denoising methods can initially achieve image denoising, they are ineffective in complex noisy environments and struggle to preserve image detail and quality in these environments, thus impacting the quality of subsequent image processing. Furthermore, deep learning image denoising methods also suffer from high computational costs, strong reliance on training data, and insufficient ability to handle multi-scale noise features. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies in the prior art and provide an image denoising method based on efficient information distillation and multi-scale feature fusion. This image denoising method solves the problem that traditional image denoising methods are ineffective in complex noisy environments. It can maintain the details and quality of the image in complex environments, thereby improving the subsequent image processing quality of image denoising.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an image denoising method based on efficient information distillation and multi-scale feature fusion, characterized by: constructing an image denoising network, the network structure including an encoder and a decoder, with skip connections between the encoder and the decoder to connect the encoder's feature map with the corresponding layer of the decoder to restore the lost spatial information; introducing a multi-focus distillation module at each stage of the encoder and decoder;

[0007] This image denoising method uses an encoder to extract image features from the preprocessed image, a decoder to restore image details layer by layer, and combines the encoder features through jump connections to achieve image reconstruction; before downsampling at each stage of the encoder and after upsampling at each stage of the decoder, a multi-focus distillation module is used to perform information distillation, multi-scale feature extraction, layer normalization processing on the image information in sequence, and image information fusion through high-frequency and contrast attention mechanisms to achieve image denoising.

[0008] The multi-focus distillation module includes an information distillation unit, a multi-scale feature extraction unit, a layer normalization processing unit and a high-frequency and contrast attention mechanism unit.

[0009] The information distillation unit first uses 3x3 convolution to extract preliminary features from the image. It then performs channel segmentation on these extracted features to produce 12-channel and 36-channel feature vectors. It then uses 1x1 and 3x3 convolution kernels in parallel to extract finer features from these 12-channel and 36-channel feature vectors. Finally, it concatenates these features to produce the distilled features. By combining multiple convolution operations and activation functions, this information distillation unit deeply processes the extracted features, enhancing the image denoising network's ability to learn and reconstruct details.

[0010] The following formula is used to connect the obtained features:

[0011] D(F)=torch.cat([F1(x),F2(x),…,F n (x)], dim=1);

[0012] Where D(F) represents the characteristics after distillation, F i (x) is the feature of layer i, and torch.cat represents the concatenation operation along a specific dimension.

[0013] The multi-scale feature extraction unit uses three dilated convolution kernels with different step sizes to extract scale information from the distilled features and obtain multi-scale fused features:

[0014]

[0015] Among them, it represents the features after multi-scale fusion of M(D), Conv k×k represents a k-step dilated convolution operation, where K is 1, 2, and 4. represents the feature fusion operation, and D is the feature after distillation.

[0016] In order to improve the image denoising network's ability to capture features of different scales, the present invention performs multi-scale feature extraction after information distillation, which can simultaneously perceive the fine-grained details and macroscopic structure of the image, thereby retaining more image information during denoising.

[0017] The layer normalization processing unit normalizes the multi-scale fused features to obtain normalized features:

[0018]

[0019] Among them, LN(M) is the normalized feature, M is the multi-scale fusion feature, μ and σ 2 are the mean and variance of the features, γ and the learnable scale and translation parameters, and β is a small constant.

[0020] The high-frequency and contrast attention mechanism unit enhances the high-frequency components of the normalized features using the following formula:

[0021] A(LN)=σ(Conv(ReLU(Conv(HF(LN)+C(LN)))));

[0022] Among them, A(LN) represents the output of the attention mechanism, HF(LN) and C(LN) represent high-frequency and contrast features respectively, σ is the sigmoid activation function, and the corresponding formulas of HF(LN) and C(LN) are as follows:

[0023] HF(LN)=|Conv HF (LN)|;

[0024]

[0025] Among them, Conv represents the convolution operation, |*| represents taking the absolute value, N represents the total number of pixels considered when calculating the contrast, and μ represents the average value of the pixels in the neighborhood.

[0026] In order to better preserve the high-frequency information of the image and improve the contrast during the denoising process, the present invention introduces a high-frequency and contrast attention mechanism after layer normalization. This high-frequency and contrast attention mechanism focuses on the high-frequency components of the image. By strengthening these components, the network's ability to reconstruct image details such as edges and textures is enhanced.

[0027] In each stage of the encoder, the features obtained after processing the image information using the multi-focal distillation module are downsampled using a 2x2 maximum pooling operation.

[0028] In each stage of the decoder, upsampling is performed using the following formula:

[0029] U(S)=WT *S+b;

[0030] Among them, U(S) is the upsampled feature map, W T is the transposed convolution kernel, b is the bias term, * represents the convolution operation, and S is the output feature map of the encoder.

[0031] It also includes convolution of the preprocessed image as the input feature map of the encoder:

[0032] C(x)=W*x+b;

[0033] Where x is the preprocessed image, W is the convolution kernel, b is the bias term, and * represents the convolution operation;

[0034] The ReLU activation function is used to introduce nonlinearity. The formula is:

[0035] ReLU(C(x))=max(0,C(x));

[0036] It also includes an output layer connected to the decoder output, which performs a 3x3 convolution operation on the output features of the decoder to obtain a denoised image I;

[0037] The denoised image I is denormalized to obtain the final denoised image, which is expressed as:

[0038] IS=I*255;

[0039] Where I is the denoised image, * is the multiplication sign, and IS is the final denoised image.

[0040] The image denoising method based on efficient information distillation and multi-scale feature fusion has the following advantages:

[0041] 1. Improved denoising efficiency and effectiveness: By integrating efficient information distillation with multi-scale feature extraction techniques, this paper significantly improves the efficiency and effectiveness of image denoising. This method can more comprehensively handle different types and levels of noise while preserving or even enhancing the original image details and quality.

[0042] 2. Enhanced feature expression capabilities: The application of information distillation technology enables the network to more effectively extract and refine key image features, which is particularly important when processing highly complex images, such as in medical imaging and satellite image analysis.

[0043] 3. Improved multi-scale processing capabilities: Multi-scale feature extraction enables the present invention to adapt to images of different resolutions, thereby effectively removing noise without sacrificing details.

[0044] 4. Enhance the stability and generalization ability of the network: By introducing layer normalization processing, such as the LayerNorm normalization layer, the present invention can maintain consistent high performance on images of different types and sources, reducing excessive dependence on specific datasets.

[0045] 5. Focus on key visual information: The introduction of high-frequency and contrast attention mechanisms enables the present invention to more accurately retain and enhance key visual information in the image, such as edges and texture details, while removing noise.

[0046] 6. Adaptability and flexibility: The network structure design of the present invention is highly adaptable and can be adjusted according to different denoising requirements and application scenarios, making it suitable for a wide range of image denoising tasks.

[0047] 7. Computational efficiency: Although the present invention adopts a complex neural network structure, its design focuses on computational efficiency, so that it can maintain reasonable processing speed and low latency when running on modern computing hardware.

[0048] In general, the present invention not only achieves significant technical improvements to image denoising methods, but also provides a powerful tool suitable for a variety of practical application scenarios, with broad market application prospects and social value.

[0049] The image denoising method based on efficient information distillation and multi-scale feature fusion can be applied to multiple application scenarios, such as:

[0050] 1. Medical imaging denoising: In medical imaging, such as MRI or CT image denoising, the network parameters are adjusted to adapt to the characteristics of the medical image and the noise type.

[0051] 2. Remote sensing image denoising: For satellite remote sensing images, the network structure and training strategy are adjusted based on their large size and multi-spectral characteristics.

[0052] 3. Video denoising: In video denoising applications, in addition to single-frame image denoising, information in the time dimension can also be considered to improve the denoising effect.

[0053] Compared with the existing technology, the present invention has the following advantages and beneficial effects: The image denoising method based on efficient information distillation and multi-scale feature fusion solves the problem that traditional image denoising methods are not effective in complex noise environments, and can maintain the details and quality of the image in complex environments, thereby improving the subsequent image processing quality of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a schematic diagram of the overall network framework of the image denoising method based on efficient information distillation and multi-scale feature fusion in the present invention; wherein 1 is the multi-focus distillation module;

[0055] Figure 2 Schematic diagram of the structure of the multi-focus distillation module in the image denoising method based on efficient information distillation and multi-scale feature fusion of the present invention;

[0056] Figure 3 Schematic diagram of the structure of the information distillation unit in the image denoising method based on efficient information distillation and multi-scale feature fusion of the present invention;

[0057] Figure 4 Schematic diagram of the structure of the multi-scale feature extraction unit in the image denoising method based on efficient information distillation and multi-scale feature fusion of the present invention;

[0058] Figure 5 This is a structural diagram of the high-frequency and contrast attention units in the image denoising method based on efficient information distillation and multi-scale feature fusion of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example

[0061] like Figures 1 to 5 As shown, the present invention first constructs an image denoising network based on an image denoising method based on efficient information distillation and multi-scale feature fusion. The network structure includes an encoder and a decoder. A jump connection is used between the encoder and the decoder to connect the feature map of the encoder with the corresponding layer of the decoder to restore the lost spatial information; a multi-focus distillation module 1 is introduced into each stage of the encoder and the decoder. The multi-focus distillation module 1 includes an information distillation unit, a multi-scale feature extraction unit, a layer normalization processing unit and a high-frequency and contrast attention mechanism unit.

[0062] This image denoising method uses an encoder to extract image features from the preprocessed image, uses a decoder to restore image details layer by layer, and combines the features of the encoder through jump connections to achieve image reconstruction; before downsampling at each stage of the encoder and after upsampling at each stage of the decoder, a multi-focus distillation module 1 is used to perform information distillation, multi-scale feature extraction, layer normalization processing on the image information in sequence, and image information fusion through high-frequency and contrast attention mechanisms to achieve image denoising.

[0063] Specifically, the information distillation unit first uses 3x3 convolution to extract preliminary features from the image. It then performs channel segmentation on these extracted preliminary features to obtain 12-channel and 36-channel feature vectors. It then uses 1x1 and 3x3 convolution kernels in parallel to extract finer features from these 12-channel and 36-channel feature vectors. Finally, it concatenates these features to produce the distilled features. By combining multiple convolution operations and activation functions, the information distillation unit deeply processes the extracted features, enhancing the image denoising network's ability to learn and reconstruct details.

[0064] The following formula is used to connect the obtained features:

[0065] D(F)=torch.cat([F1(x),F2(x),…,F n (x)], dim=1);

[0066] Where D(F) represents the characteristics after distillation, F i (x) is the feature of layer i, and torch.cat represents the concatenation operation along a specific dimension.

[0067] The multi-scale feature extraction unit uses three dilated convolution kernels with different step sizes to extract scale information from the distilled features and obtain multi-scale fused features:

[0068]

[0069] Among them, it represents the features after multi-scale fusion of M(D), Conv k×k represents a k-step dilated convolution operation, where K is 1, 2, and 4. represents the feature fusion operation, and D is the feature after distillation.

[0070] In order to improve the image denoising network's ability to capture features of different scales, the present invention performs multi-scale feature extraction after information distillation, which can simultaneously perceive the fine-grained details and macroscopic structure of the image, thereby retaining more image information during denoising.

[0071] The layer normalization processing unit normalizes the multi-scale fused features to obtain normalized features:

[0072]

[0073] Among them, LN(M) is the normalized feature, M is the multi-scale fusion feature, μ and σ 2 are the mean and variance of the features, γ and the learnable scale and translation parameters, and β is a small constant.

[0074] The high-frequency and contrast attention mechanism unit enhances the high-frequency components of the normalized features using the following formula:

[0075] A(LN)=σ(Conv(ReLU(Conv(HF(LN)+C(LN)))));

[0076] Among them, A(LN) represents the output of the attention mechanism, HF(LN) and C(LN) represent high-frequency and contrast features respectively, σ is the sigmoid activation function, and the corresponding formulas of HF(LN) and C(LN) are as follows:

[0077] HF(LN)=|Conv HF (LN)|;

[0078]

[0079] Among them, Conv represents the convolution operation, |*| represents taking the absolute value, N represents the total number of pixels considered when calculating the contrast, and μ represents the average value of the pixels in the neighborhood.

[0080] In order to better preserve the high-frequency information of the image and improve the contrast during the denoising process, the present invention introduces a high-frequency and contrast attention mechanism after layer normalization. This high-frequency and contrast attention mechanism focuses on the high-frequency components of the image. By strengthening these components, the network's ability to reconstruct image details such as edges and textures is enhanced.

[0081] In each stage of the encoder, the features obtained after processing the image information using the multi-focal distillation module are downsampled using a 2x2 maximum pooling operation.

[0082] In each stage of the decoder, upsampling is performed using the following formula:

[0083] U(S)=W T *S+b;

[0084] Among them, U(S) is the upsampled feature map, W T is the transposed convolution kernel, b is the bias term, * represents the convolution operation, and S is the output feature map of the encoder.

[0085] It also includes convolution of the preprocessed image as the input feature map of the encoder:

[0086] C(x)=W*x+b;

[0087] Where x is the preprocessed image, W is the convolution kernel, b is the bias term, and * represents the convolution operation;

[0088] The ReLU activation function is used to introduce nonlinearity. The formula is:

[0089] ReLU(C(x))=max(0,C(x));

[0090] It also includes an output layer connected to the decoder output, which performs a 3x3 convolution operation on the output features of the decoder to obtain a denoised image I;

[0091] The denoised image I is denormalized to obtain the final denoised image, which is expressed as:

[0092] IS=I*255;

[0093] Where I is the denoised image, * is the multiplication sign, and IS is the final denoised image.

[0094] The image denoising method based on efficient information distillation and multi-scale feature fusion can be applied to multiple application scenarios, such as:

[0095] 1. Medical imaging denoising: In medical imaging, such as MRI or CT image denoising, the network parameters are adjusted to adapt to the characteristics of the medical image and the noise type.

[0096] 2. Remote sensing image denoising: For satellite remote sensing images, the network structure and training strategy are adjusted based on their large size and multi-spectral characteristics.

[0097] 3. Video denoising: In video denoising applications, in addition to single-frame image denoising, information in the time dimension can also be considered to improve the denoising effect.

[0098] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An image denoising method based on efficient information distillation and multi-scale feature fusion, characterized by: Construct an image denoising network. The network structure includes an encoder and a decoder. The encoder and decoder use skip connections to connect the encoder's feature map with the corresponding layer of the decoder to restore the lost spatial information. Multiple focal distillation modules are introduced in each stage of encoder and decoder; This image denoising method uses an encoder to extract image features from preprocessed images, a decoder to restore image details layer by layer, and combines encoder features through skip connections to achieve image reconstruction. Before downsampling at each encoder stage and after upsampling at each decoder stage, a multi-focus distillation module is used to perform information distillation, multi-scale feature extraction, layer normalization, and image information fusion through a high-frequency and contrast attention mechanism to achieve image denoising. The multi-focus distillation module includes an information distillation unit, a multi-scale feature extraction unit, a layer normalization processing unit, and a high-frequency and contrast attention mechanism unit; The information distillation unit first uses 3x3 convolution to perform preliminary feature extraction on the image information; then it performs channel segmentation on the extracted preliminary features to obtain 12-channel feature vectors and 36-channel feature vectors, and uses 1x1 convolution kernel and 3x3 convolution kernel in parallel to perform fine feature extraction on the 12-channel feature vectors and 36-channel feature vectors; finally, the obtained features are concatenated to obtain the distilled features.

2. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 1, characterized in that: The following formula is used to connect the obtained features: D=torch.cat([F1(x’),F2(x’),…,F n (x’)],dim=1); Among them, D represents the characteristics after distillation, F i (x') is the feature of layer i, and torch.cat represents the concatenation operation along a specific dimension.

3. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 1, characterized in that: The multi-scale feature extraction unit uses three dilated convolution kernels with different step sizes to extract scale information from the distilled features and obtain multi-scale fused features: Among them, M represents the multi-scale fusion feature, Conv k×k represents a k-step dilated convolution operation, where K is 1, 2, and 4. represents the feature fusion operation, and D is the feature after distillation.

4. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 3, characterized in that: The layer normalization processing unit normalizes the multi-scale fused features to obtain normalized features: Among them, LN is the normalized feature, M is the multi-scale fusion feature, μ and σ 2 are the mean and variance of the features, γ is a learnable scale and translation parameter, and β is a small constant.

5. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 4, characterized in that: The high-frequency and contrast attention mechanism unit enhances the high-frequency components of the normalized features using the following formula: A(LN)=σ(Conv(ReLU(Conv(HF(LN)+C(LN))))); Among them, A(LN) represents the output of the attention mechanism, HF(LN) and C(LN) represent high-frequency and contrast features respectively, σ is the sigmoid activation function, and the corresponding formulas of HF(LN) and C(LN) are as follows: HF(LN)=|Conv HF (LN)|; Among them, Conv represents the convolution operation, |*| represents taking the absolute value, N represents the total number of pixels considered when calculating the contrast, and μ' represents the average value of the pixel values ​​in the neighborhood.

6. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 1, characterized in that: In each stage of the encoder, the features obtained after processing the image information using the multi-focal distillation module are downsampled using a 2x2 maximum pooling operation.

7. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 1, characterized in that: In each stage of the decoder, upsampling is performed using the following formula: U(S)=W T *S+b; Among them, U(S) is the upsampled feature map, W T is the transposed convolution kernel, b is the bias term, * represents the convolution operation, and S is the output feature map of the encoder.

8. The image denoising method based on efficient information distillation and multi-scale feature fusion according to claim 1, characterized in that: It also includes convolution of the preprocessed image as the input feature map of the encoder: C(x)=W*x+b; Where x is the preprocessed image, W is the convolution kernel, b is the bias term, and * represents the convolution operation; The ReLU activation function is used to introduce nonlinearity. The formula is: ReLU(C(x))=max(0,C(x)); It also includes an output layer connected to the decoder output, which performs a 3x3 convolution operation on the output features of the decoder to obtain a denoised image I; Perform an inverse normalization operation on the denoised image I to obtain the final denoised image, which is expressed as: IS=I*255; Where I is the denoised image, * is the multiplication sign, and IS is the final denoised image.

Images