Image denoising model and method based on parallel fusion network
By using a parallel fusion network image denoising model that combines CNN and Swing Transformer to extract local and global features, the problem of poor denoising performance in existing technologies is solved, and efficient image denoising in complex noisy environments is achieved.
Patent Information
- Application Number
- CN202511466801.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-14
- Publication Date
- 2026-01-13
AI Technical Summary
Existing image denoising techniques often lose important details when faced with complex noise, and as network depth increases, the number of parameters and inference time also increase, affecting practical applications.
An image denoising model based on parallel fusion network is adopted, which includes a first sub-network and a second sub-network. The first sub-network uses an encoder and decoder composed of CNN to extract local features, and the second sub-network uses the Swing Transformer module to extract global features. Image denoising is achieved through a data fusion module and residual connections.
It effectively balances the processing of features at different levels of the image, improves the denoising effect, has strong adaptability, and performs exceptionally well in complex noisy environments and real-world scene images, making it highly promising for industrial applications.
Smart Images
Figure CN121329809A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and in particular relates to an image denoising model and method based on parallel fusion networks. Background Technology
[0002] With the proliferation of data repositories and technological advancements, image usage has permeated various fields. From medical imaging and autonomous driving to surveillance systems, images have become an indispensable part of modern life. However, real-world images often suffer quality degradation due to noise, which in turn weakens the effectiveness of image algorithms. Noise in an image can be viewed as irregular structures obscuring visual content. Image denoising is a crucial preprocessing step that not only removes noise and improves image quality but also enhances the accuracy and reliability of image-based systems while preserving visual appeal. Therefore, the demand for image denoising and efficient noise reduction continues to grow.
[0003] Traditional denoising techniques include mean filtering, median filtering, wavelet transform, Wiener filtering, and block-matched 3D filtering. These methods achieve noise removal through hand-designed filters. While effective for certain types of noise, traditional methods often lose important details when dealing with unknown and complex noise, affecting their practicality.
[0004] The emergence of deep neural networks (DNNs) has enabled denoising models to overcome the limitations of traditional methods. DNN-based models have fewer hyperparameters, faster inference speeds, and superior denoising performance. Deep denoising networks include convolutional networks and residual networks. Convolutional networks map noisy images to sharp images, while residual networks utilize skip connections to learn residual mappings, thereby improving efficiency. Most existing denoising networks combine convolutional and residual networks, but are limited by local receptive fields, which may lead to color shifts or texture loss. Furthermore, as the network deepens, the increase in the number of parameters and inference time hinders its application in practical denoising scenarios. Some dual-convolutional neural networks utilize parallel branches to extract complementary features, enhancing detail recovery in low-level visual tasks. However, treating all features equally in parallel branches may limit denoising performance. Summary of the Invention
[0005] The purpose of this invention is to provide an image denoising model and method based on parallel fusion networks, aiming to solve the problems mentioned in the background art.
[0006] The present invention is implemented as follows: an image denoising model based on a parallel fusion network includes a first sub-network, a second sub-network, and a data fusion module; The first sub-network uses an encoder and decoder composed of CNNs to extract and process local features; The second sub-network uses the Swin Transformer module (STB module) to extract global features; The data fusion module is used to perform channel fusion of the local features extracted by the first sub-network and the global features extracted by the second sub-network, and obtain a clean image after denoising through residual connection.
[0007] In a further technical solution, the encoder of the first sub-network includes two CAB modules, the decoder includes three CAB modules and a 3×3 convolutional layer, and the encoder and decoder are connected through three Skip connections.
[0008] In a further technical solution, the second sub-network consists of a 4×4 convolutional layer, two PM+STB combined modules, an STB module, an STB+PE combined module, an STB+PE+DualUP combined module, and a 3×3 convolutional layer.
[0009] Another objective of this invention is to provide an image denoising method based on a parallel fusion network, which, based on the above-described model, includes the following steps: Step 1: Process the noisy input image A 3×3 convolutional layer is used to extract low-level features. ,in and These are the height and width of the input image pixels, respectively; Step 2: Local feature maps are extracted through multiple CBA modules and spatial attention mechanism in the first sub-network. The calculation process is as follows: ; in, , , , and The feature map generated during the calculation process. This is the CAB module, used to extract local spatial information from the feature map; Downsampling is used to extract important features and reduce the resolution of the feature map. Upsampling is used to restore the resolution and details of the feature map; It is a 3×3 convolutional layer; Step 3: After patching and linear embedding operations, a token is obtained that is convenient for STB module operations. , The global feature map is extracted by the second sub-network, which is mainly composed of STB modules. The calculation process is as follows: ; in, , , , and The feature map generated during the calculation process; This is the STB module, used to extract global interaction features from the feature map; This is a patch merging operation, used to merge small patches, reduce feature map resolution, balance feature map dimensions, extract multi-scale features, and improve computational efficiency and feature quality. This is a patch expanding operation used to restore feature map resolution and reduce feature map channel dimensions; Dual upsampling is used to restore the feature map resolution, reduce the occurrence of checkerboard artifacts, and improve feature quality. Step 4: Extract the local feature map from the first sub-network Global feature map extracted by the second sub-network Perform a fusion operation to obtain the final feature representation. : ; in, For feature fusion function; Step 5: Finally, obtain the denoised clean image through residual mapping. : .
[0010] This invention provides an image denoising model and method based on a parallel fusion network. The model employs a parallel fusion network structure, comprising two sub-networks. The first sub-network consists of a CNN and a spatial attention mechanism connected in series, focusing on extracting local features. The second sub-network introduces a Swing Transformer, which models the global contextual information of the image through a hierarchical window attention mechanism, complementing the local feature extraction capabilities of the CNN. This organic combination of local and global features allows the model to better balance the processing of features at different levels of the image, effectively addressing multi-scale and multi-modal noise. Finally, the network achieves image denoising by fusing the two sub-networks through channels and combining residual learning. This model design has significant practicality and broad application value, especially in complex noisy environments and real-world images, effectively improving denoising performance. It is highly adaptable and has high potential for industrial applications. Attached Figure Description
[0011] Figure 1 This is a schematic diagram of the structure of an image denoising model based on a parallel fusion network provided in an embodiment of the present invention; Figure 2 This is a structural diagram of an important module in an image denoising model based on a parallel fusion network, provided in an embodiment of the present invention. Figure 3 Visual comparison of the denoising effects of different models on the cc dataset.
[0012] In the attached diagram: First subnetwork 1; Second subnetwork 2; Data fusion module 3. Detailed Implementation
[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0014] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0015] like Figure 1 As shown, an image denoising model based on a parallel fusion network is provided in one embodiment of the present invention, including a first sub-network 1, a second sub-network 2, and a data fusion module 3; The first sub-network 1 uses a CNN-based encoder and decoder to extract and process local features. Specifically, the encoder of the first sub-network includes two CAB modules, and the decoder includes three CAB modules and a 3×3 convolutional layer. The encoder and decoder are connected via three skip connections. The CAB modules are as follows: Figure 2 As shown in a.
[0016] The second sub-network 2 mainly employs the Swin Transformer module (STB module), primarily used for extracting global features. Specifically, the second sub-network 2 consists of a 4×4 convolutional layer, two PM+STB combined modules, one STB module, one STB+PE combined module, one STB+PE+DualUP combined module, and a 3×3 convolutional layer. The STB module is as follows... Figure 2 As shown in c; The data fusion module 3 is used to perform channel fusion of the local features extracted by the first sub-network 1 and the global features extracted by the second sub-network 2, and obtain a clean image after denoising through residual connection.
[0017] In this embodiment of the invention, the device includes two sub-networks. The first sub-network 1 is composed of a CNN and a spatial attention mechanism (SAB) connected in series, focusing on extracting local features. The spatial attention mechanism (SAB) is as follows: Figure 2 As shown in b, the second sub-network 2 introduces the Swin Transformer, which models the global contextual information of the image through a hierarchical window attention mechanism, complementing the local feature extraction capability of CNN. The organic combination of local and global features allows the denoiser to better balance the processing of features at different levels of the image, thus effectively dealing with multi-scale and multi-modal noise. Finally, the two sub-networks are fused through the data fusion module 3, and image denoising is achieved by combining residual learning. This denoiser design has significant practicality and wide application value, especially in the face of complex noisy environments and real-world images, effectively improving denoising performance, exhibiting strong adaptability, and possessing high potential for industrial applications.
[0018] The overall process of the device is as follows: Figure 1 As shown. In the first sub-network 1, a CNN-based encoder and decoder are used to extract and process local information, as follows: Figure 1 The upper half of the network. The second sub-network, part 2, primarily uses the SwinTransformer module to extract global features, such as... Figure 1 The lower half of the text.
[0019] As a preferred embodiment of the present invention, the device has some improvements based on the prior art: (1) In the first sub-network 1, traditional convolution tends to ignore the correlation of contextual information in feature extraction. Therefore, a spatial attention mechanism is introduced to dynamically adjust the spatial feature weights to highlight key regions, thereby enhancing the model's denoising ability.
[0020] (2) Add residual connections, use 3×3 convolution kernels to achieve feature fusion in the channel dimension, and add residual connections to construct a clean image through residual learning to obtain the denoised image.
[0021] (3) Using the fusion loss function, mean squared error loss is used for the removal of synthetic noise; for the removal of real noise, the common Charbonnier loss and reconstruction loss function are optimized and combined to obtain the total loss function for network training, so that the constructed model has good generalization ability for different types of problems.
[0022] In practice, the noisy image data is first randomly cropped to obtain training samples for the model. For grayscale image denoising, the cropped color image patches are converted to grayscale. Then, the cropped image patches are subjected to data augmentation through random rotation and flipping. Finally, tensor transformation is performed on the augmented noisy image patches.
[0023] Then, the model parameters are initialized, and the model hyperparameters are set. The model parameters are initialized using an orthogonal matrix initialization method, and the Batch Normalization (BN) layer uses a uniform distribution initialization method. For the specific denoising task, the initial learning rate and learning rate decay strategy, the number of training epochs, the batch size of training samples, the DropRate used in each Swin-Transformer module, the sliding window size in the STB module, and the query dimension for each attention head are set.
[0024] The model parameters are then optimized using the Adam optimizer, with hyperparameter settings following the default settings of the Adam optimizer.
[0025] During training, for a specific denoising task, the model is validated with a validation set every certain number of iterations, and the parameters of the best-performing model are saved.
[0026] After the model is trained, the image data that needs to be denoised is patched and then input into the model to obtain the corresponding output. The patched output is then reverse-patched to restore the clean image data.
[0027] Another embodiment of the present invention provides an image denoising method based on a parallel fusion network, which, based on the above model, includes the following steps: Step 1: Process the noisy input image A 3×3 convolutional layer is used to extract low-level features. ,in and These are the height and width of the input image pixels, respectively.
[0028] Step 2: Local feature maps are extracted through multiple CBA modules and spatial attention mechanism in the first sub-network 1. The calculation process is as follows: ; in, , , , and The feature map generated during the calculation process. The CAB module is used to extract local spatial information from feature maps, such as... Figure 2 As shown in a; Downsampling is used to extract important features and reduce the resolution of the feature map. Upsampling is used to restore the resolution and details of the feature map; It is a 3×3 convolutional layer.
[0029] Step 3: After patching and linear embedding operations, a token is obtained that is convenient for STB module operations. , The global feature map is extracted by the second subnetwork 2, which is mainly composed of STB modules. The calculation process is as follows: ; in, , , , and The feature map generated during the calculation process; The STB module is used to extract global interaction features from the feature map, such as... Figure 2 As shown in c; This is a patch merging operation, used to merge small patches, reduce feature map resolution, balance feature map dimensions, extract multi-scale features, and improve computational efficiency and feature quality. This is a patch expanding operation used to restore feature map resolution and reduce feature map channel dimensions; Dual upsampling is used to restore feature map resolution, reduce checkerboard artifacts, and improve feature quality, such as... Figure 2 As shown in d.
[0030] Step 4: Extract the local feature map from the first sub-network 1 Global feature map extracted by sub-network 2 Perform a fusion operation to obtain the final feature representation. : ; in, This is the feature fusion function.
[0031] Step 5: Finally, obtain the denoised clean image through residual mapping. : .
[0032] The following simulation experiments based on this model are used to verify the effectiveness of this method.
[0033] Table 1 below uses the cc dataset as an example. The models in the table are the most advanced models currently available. They are models based on convolutional neural networks and Swin Transformers, respectively. CSPFNet refers to the model proposed in this invention.
[0034] Table 1 Comparison of PSNR results for different models
[0035] Table 1 compares the PSNR (Peak Signal-to-Noise Ratio) results of different models on the cc dataset. The results show that CSPFNet achieved the best results in 12 out of 15 scenarios, the second best in 2 scenarios, and the fourth best in 1 scenario. In the comparison of average PSNR results, CSPFNet also achieved the best results, with an improvement of at least 2.44 dB compared to other methods. This indicates that CSPFNet can achieve better performance than existing methods.
[0036] Figure 3 Visual comparison of the denoising performance of different models on the cc dataset shows that CSPFNet not only achieves the highest PSNR, but also achieves more outstanding results in terms of both overall structure and detail.
[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An image denoising model based on parallel fusion networks, characterized in that, It includes a first sub-network, a second sub-network, and a data fusion module; The first sub-network uses an encoder and decoder composed of CNNs to extract and process local features; The second sub-network uses the STB module to extract global features; The data fusion module is used to perform channel fusion of the local features extracted by the first sub-network and the global features extracted by the second sub-network, and obtain a clean image after denoising through residual connection.
2. The image denoising model based on parallel fusion networks according to claim 1, characterized in that, The encoder of the first sub-network includes two CAB modules, and the decoder includes three CAB modules and a 3×3 convolutional layer. The encoder and decoder are connected by three Skip connections.
3. The image denoising model based on parallel fusion networks according to claim 1, characterized in that, The second sub-network consists of a 4×4 convolutional layer, two PM+STB combined modules, an STB module, an STB+PE combined module, an STB+PE+DualUP combined module, and a 3×3 convolutional layer.
4. An image denoising method based on a parallel fusion network, based on the image denoising model based on a parallel fusion network as described in any one of claims 1-3, characterized in that, Includes the following steps: Step 1: Process the noisy input image A 3×3 convolutional layer is used to extract low-level features. ,in and These are the height and width of the input image pixels, respectively; Step 2: Local feature maps are extracted through multiple CBA modules and spatial attention mechanism in the first sub-network. The calculation process is as follows: ； in, , , , and The feature map generated during the calculation process. This is the CAB module, used to extract local spatial information from the feature map; This is for downsampling, used to extract important features and reduce the resolution of the feature map; Upsampling is used to restore the resolution and details of the feature map; It is a 3×3 convolutional layer; Step 3: After patching and linear embedding operations, a token is obtained that is convenient for STB module operations. , The global feature map is extracted by the second sub-network, which is mainly composed of STB modules. The calculation process is as follows: ； in, , , , and The feature map generated during the calculation process; This is the STB module, used to extract global interaction features from the feature map; For patch merging operations; This is a patch expansion operation used to restore feature map resolution and reduce feature map channel dimensions; Dual upsampling is used to restore the feature map resolution and reduce the occurrence of chessboard artifacts; Step 4: Extract the local feature map from the first sub-network Global feature map extracted by the second sub-network Perform a fusion operation to obtain the final feature representation. : ； in, For feature fusion function; Step 5: Finally, obtain the denoised clean image through residual mapping. : 。