A joint denoising training method combining a noise-free image and a noisy image

Abstract

The application discloses a kind of combined noise-free image and joint denoising training method of noisy image, S1.image denoising model training: using noise-free image and corresponding noisy image as training data to image denoising model is trained;S2.data enhancement: to input image is carried out multiple enhancement operations, and enhancement operation includes at least one in rotation, flip, crop, color change;S3.in model training process, using hybrid loss function is optimized to network;S4.during training process, dynamic optimization strategy is implemented: according to current denoising effect, the learning rate of model is dynamically adjusted, to further improve the convergence speed and denoising performance of model;S5.adopt adaptive optimizer to update parameter in model optimization process.The method based on the application can further improve the general denoising performance of image denoising model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a joint denoising training method that combines noise-free images and noisy images. Background Technology

[0002] Image denoising is a fundamental step in image processing and a basis for advanced computer vision tasks such as object detection and image segmentation. It aims to recover high-quality images from noisy images. During acquisition and transmission, images are often corrupted by noise, which severely degrades the visual quality of the acquired images. Various types of noise may exist during image acquisition and processing, including Gaussian noise, Poisson noise, and JPEG compression noise, making image denoising a very challenging task. Because the image degradation process is very complex, it is difficult to accurately characterize noise information, thus restoring a clean image is a difficult task. In the field of computer vision, image denoising has been a research hotspot since the 1990s.

[0003] Most traditional denoising methods are algorithm-based. With the development of deep learning, Convolutional Neural Networks (CNNs) have become mainstream in the field of image denoising, such as DnCNN, which has achieved significant performance improvements in image denoising. Nevertheless, CNN-based methods still have some limitations. Since the convolution kernel is usually fixed, using the same kernel to process different image regions may not be optimal. Furthermore, the convolution kernel can be viewed as a small window for capturing local information, meaning that global information is easily lost when attempting to model relationships between distant pixels.

[0004] On the other hand, the introduction of VIT pioneered the introduction of the Transformer into the field of computer vision. Due to its ability to effectively capture global contextual information in images and adaptively focus on pixels at different locations, Transformer-based methods have surpassed convolutional neural network (CNN)-based methods in a growing number of computer vision tasks, including image segmentation, object detection, reconstruction, and super-resolution, reaching state-of-the-art levels. However, training a Transformer model requires a large number of parameters, especially when processing high-resolution data such as images, and boundary pixels in images are difficult to fully utilize in the Transformer, which may lead to inaccurate feature extraction near boundaries. Therefore, how to apply the Transformer, especially in low-level computer vision tasks, remains a challenge. Summary of the Invention

[0005] The purpose of this invention is to provide a joint denoising training method that combines noiseless images and noisy images, and to further improve the denoising performance of the image denoising model by using a hybrid loss function.

[0006] A joint denoising training method combining noise-free and noisy images, characterized by the following steps:

[0007] S1. Training of the image denoising model:

[0008] The image denoising model is trained using noise-free images and corresponding noisy images as training data.

[0009] S2. Data Augmentation:

[0010] The input image is subjected to various enhancement operations, including at least one of rotation, flipping, cropping, and color change. The enhanced input image is then fed into the image denoising model for training, thereby increasing the robustness and generalization ability of the image denoising model.

[0011] S3. During model training, a hybrid loss function is used to optimize the network;

[0012] The hybrid loss function consists of two parts: a first loss function composed of the pixel-level loss of the denoised noisy image and the real clean image; and a second loss function composed of the pixel-level loss of the denoised noiseless image and the real clean image.

[0013] S4. Implement dynamic optimization strategies during training:

[0014] Based on the current denoising effect, dynamically adjust the model's learning rate to further improve the model's convergence speed and denoising performance.

[0015] S5. An adaptive optimizer is used to update the parameters during model optimization:

[0016] After each training iteration, the model performance is monitored using validation set performance metrics (such as peak signal-to-noise ratio PSNR). The training process terminates when the validation set performance metrics stabilize after multiple iterations and no longer improve significantly, or when the preset number of training rounds is reached.

[0017] In step S1, the training process takes noiseless images and noisy images with added random Gaussian noise as input, and uses the corresponding real clean images as labels to optimize the image denoising model through supervised learning.

[0018] In step S1, the image denoising model is a graph denoising model called SUNet (Swin Transformer UNet).

[0019] Like a denoising model;

[0020] SUNet is an image denoising model that combines the Swin Transformer with the UNet architecture; the model consists of three modules: a shallow feature extraction module, a UNet feature extraction module, and an image reconstruction module.

[0021] (1) Shallow feature extraction module:

[0022] This module adjusts the feature dimensions of the input image using a 3×3 convolutional layer and extracts low-level features; this process can be represented by the following formula:

[0023] F shallow =M SFE (I)

[0024] in, This represents the image to be denoised, where H and W represent the height and width of the image, respectively, and 3 represents the number of channels (i.e., RGB channels); M SFE (·) indicates a shallow feature extraction layer, which is the process of extracting features from an image through convolution operations;

[0025] C represents the number of channels in the shallow features (in this invention, it is set to 96; 96 channels refer to the depth of the feature map generated by the convolution operation. Choosing 96 channels aims to balance computational complexity and feature representation ability, ensuring that enough low-level features can be effectively extracted for subsequent processing).

[0026] (2) UNet Feature Extraction Module:

[0027] This module adopts an encoder-decoder structure as its core.

[0028] The encoder gradually reduces the spatial resolution of the image and extracts deep features through multiple Swing Transformer blocks and downsampling layers (such as PatchMerging layers); the decoder gradually restores the spatial resolution of the image through multiple Swing Transformer blocks and upsampling layers, while retaining shallow feature information by combining skip connections.

[0029] The Swin Transformer is a transformer model based on a sliding window attention mechanism that can effectively capture local and global feature information in an image. Each Swin Transformer block consists of 8 Swin Transformer layers. Each layer performs self-attention computation locally through a sliding window mechanism, while gradually increasing the receptive field using a hierarchical structure to capture multi-scale contextual information.

[0030] Will Input to UNet feature extraction M VFE In (·), the process of extracting high-level and multi-scale deep features can be represented as:

[0031] F deep =M VFE (F shallow )

[0032] Among them, deep features Consistent with the shallow feature dimension, C represents the number of channels (96 in this invention).

[0033] (3) Image Reconstruction Module

[0034] This module uses a 3×3 convolutional layer M R (·), generating a denoised image based on deep features; the final image reconstruction process is represented by the following formula:

[0035]

[0036] in, This represents the final output of the model, where H and W represent the height and width of the image, respectively, and 3 represents the number of image channels (i.e., RGB channels). M R (·) represents the image reconstruction layer, which is the layer that extracts deep features F through convolution operations. deep Reconstruct the denoised image in the middle;

[0037] It's worth noting that the convolutional layers in the shallow feature extraction module are used to extract shallow features (i.e., low-level features of the image), while the convolutional layers in the reconstruction module are used to generate the final denoised image from deep features. Although these two convolutional layers have the same structure—both are 3×3 convolutional layers—their input and output dimensions differ. The former processes low-level features, while the latter generates the final denoised image. Their convolutional operations and implementations are similar, but their functions differ: the former aims to extract features, while the latter is used to reconstruct the image.

[0038] In step S2, the data augmentation operation includes image rotation angle, contrast adjustment, brightness change, and random cropping. The augmented image is used to further improve the generalization ability of the model.

[0039] In step S3, the hybrid loss function is composed of the first loss function and the second loss function:

[0040] First loss function: The noisy image is input into the denoising model to generate the denoised image, and then the pixel values ​​of the real sample image are combined to calculate the first loss value:

[0041]

[0042] in, For the denoised image output by the model, I $％ea( The image is a clean, true image; ||·| refers to the L1 norm operation, which calculates the absolute error at each pixel location and sums all these errors to obtain the image loss value.

[0043] Second loss function: The clean image is input into the denoised image generated by the model, and then the second loss is calculated by combining the pixel values ​​of the real sample image:

[0044]

[0045] in, For the clean image after denoising, I $％ea( For a truly clean image;

[0046] The expression for the mixed loss function is:

[0047]

[0048] Where λ1 and λ ρ These are hyperparameters used to control the weights of the two parts of the loss function. In this invention, λ1 and λ... ρ Set all to 1.

[0049] In step S4, the dynamic optimization strategy refers to adjusting the learning rate based on the denoising effect of the model during training, in order to avoid overfitting and further improve the stability of the model.

[0050] First, a warm-up strategy is employed; during the warm-up phase, the learning rate starts from η. . (An initial small value) is gradually increased until the preset initial learning rate η is reached. 0a12345 The learning rate is maintained at this value until the end of the warm-up phase. The learning rate adjustment formula is:

[0051]

[0052] Where η6 is the learning rate of the current training round t steps, η wa9:3；p The target learning rate at the end of the warm-up phase is set to 2e-4 in this invention, where η is the learning rate.. This is the learning rate at the start of the warm-up phase, typically set to 1e-6; T 0a12345 This is the total number of training rounds during the warm-up phase, which is set to 3 in this invention.

[0053] After the warm-up phase, the learning rate enters the cosine annealing phase, where it gradually decreases according to a cosine function as training progresses. The learning rate adjustment formula for the cosine annealing strategy is:

[0054]

[0055] Where η6 is the learning rate at step t of the current training round, η 2i( η is the minimum learning rate during training (typically set to 1e-6). wa9:3；p It is the target learning rate at the end of the warm-up phase; T HIHa％ T is the total number of training rounds. 0a12345 This refers to the total number of training rounds during the warm-up phase, which are set to 200 and 3 respectively in this invention. The cosine annealing strategy will cause the learning rate to decrease from η... wa9:3；p Initially, it gradually decreases to η. 2i( This helps the optimizer to fine-tune parameters as it approaches the optimal solution.

[0056] In step S5, the adaptive optimizer is the Adam optimizer, which uses a dynamic learning rate adjustment method combined with a weight decay strategy to further improve the training efficiency and denoising effect of the model. The update formula of the Adam optimizer is as follows:

[0057]

[0058] Where θ6 represents the model parameters at the current training epoch (step t), θ6 represents the parameters in the network that need to be optimized (such as the convolutional kernel weights in the shallow feature extraction module and image reconstruction module, the weight matrix and bias terms of the SwinTransformer self-attention layer in the UNet feature extraction module, etc.), and η6 represents the current learning rate. and ε is the first and second moment estimate of the gradient, and ε is a minimal constant to prevent division by zero error. ε is usually taken as 1e-8.

[0059] First-order moment estimation and second-order moment estimation The update rules are as follows:

[0060] m6=β1m 631 +(1-β1)g6

[0061] v6 = β ρ v 631 +(1-β ρ g6 ρ

[0062]

[0063] Here, g6 represents the gradient of the loss function relative to the model parameters at training epoch t. This gradient reflects the direction and magnitude of the model parameters' changes in the current epoch and is a crucial basis for the optimization algorithm (Adam optimizer) to update the model parameters. Specifically, as the training epochs progress, the model parameters (such as the convolutional kernel weights in the shallow feature extraction module and image reconstruction module, and the weight matrix and bias terms of the SwinTransformer self-attention layer in the UNet feature extraction module) are dynamically adjusted to gradually minimize the loss function. The gradient of the loss function relative to the model parameters is calculated using the backpropagation algorithm and is used to indicate how each parameter should be adjusted in the current epoch to reduce training error, thereby improving model performance. β1 and β c The attenuation rate hyperparameters for estimating the first and second moments are typically taken as β1 = 0.9 and β2 = 0.9. ρ =0.999.

[0064] The backpropagation algorithm continuously updates model parameters using an optimizer, iteratively optimizing the model until the best denoising effect is achieved. Furthermore, model regularization techniques (such as L2 regularization and Dropout) can be used during training to further prevent overfitting, making the model more robust and reliable on test data.

[0065] Beneficial effects:

[0066] This invention combines a joint denoising training method using noise-free and noisy images. Specifically, it includes the following steps: S1. Training the image denoising model: Using noise-free images and corresponding noisy images as training data, the image denoising model is trained; S2. Data augmentation: Performing various augmentation operations on the input image and inputting the augmented image into the model for training increases the model's robustness and generalization ability; S3. Using a hybrid loss function: Incorporating pixel-level losses from the denoised noise-free image and the real clean image into the calculation of the hybrid loss function improves the model's denoising effect and detail restoration ability; S4. Dynamic optimization: Adjusting the learning rate based on the model's denoising effect during training to improve the denoising performance; S5. Model optimization: Using an adaptive optimizer to update model parameters and iteratively optimize until the optimal denoising effect is achieved. Based on existing image denoising models, this invention proposes a joint denoising training method that combines noiseless and noisy images, considering that the model may eliminate noise that is not present in the image. This method ensures that the proposed image denoising model can guarantee a reasonable convergence position of the output image in the image solution space, thereby obtaining a denoised image that is as close as possible to the real sample image.

[0067] Compared with the prior art, the present invention has the following advantages:

[0068] The purpose of this invention is to provide a joint denoising training method that combines noise-free and noisy images. Compared with existing technologies, this invention fully considers the impact of noise-free images on image denoising performance. By adding pixel-level losses of the denoised noise-free image and the real clean image to the loss function calculation process, it effectively avoids the model removing noise that was not originally present in the image during the denoising process, thereby reducing the suppression of irrelevant noise. This method can further improve the denoising performance of the image denoising model, enhance the model's robustness to real noise, and thus improve the quality of the denoised image.

[0069] Based on the method proposed in this invention, the general denoising performance of the image denoising model can be further improved. Attached image description:

[0070] Figure 1 This is a schematic diagram of the overall process of a joint denoising training method combining noiseless and noisy images provided by the present invention.

[0071] Figure 2 This is a flowchart of the method of the present invention.

[0072] Figure 3 This is a partial visualization of the denoising results of this invention. Detailed Implementation

[0073] The present invention will be further described below with reference to specific embodiments and accompanying drawings:

[0074] Example 1

[0075] Task definition

[0076] Image denoising aims to recover a clean image X from a noisy image Y that is as close as possible to the original. Specifically, given a noisy image Y, the task is to output a denoised image xQ = D(Y) using a denoising model D(·), such that the denoised image... It should be as similar as possible to the original clean image X.

[0077] The objective of this invention is to provide a joint denoising method that combines noise-free images and noisy images to further improve the visual quality of images.

[0078] Implementation steps

[0079] like Figures 1-2 As shown, the present invention provides a joint denoising method combining noise-free images and noisy images, comprising the following steps:

[0080] Step 1: Training the image denoising model:

[0081] The image denoising model is trained using noiseless images and corresponding noisy images as training data. The denoising model is SUNet (Swin Transformer UNet), proposed in the literature Chi-Mao Fan, Tsung-Jung Liu, and Kuan-Hsien Liu. Sunet: Swin Transformer UNet for image denoising. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pages 2333–2337. IEEE, 2022. Specifically, SUNet is an image denoising model that combines the Swin Transformer and UNet architectures. This model consists of three modules: a shallow feature extraction module, a UNet feature extraction module, and an image reconstruction module.

[0082] Shallow feature extraction module. This module adjusts the feature dimensions of the input image using a 3×3 convolutional layer and extracts low-level features. This process can be represented by the following formula:

[0083] F shallow =M SFE (I)

[0084] in, This represents the image to be denoised, where H and W represent the height and width of the image, respectively, and 3 represents the number of channels (i.e., RGB channels). C represents the number of channels in the shallow layer feature (set to 96 in this invention).

[0085] The UNet feature extraction module primarily employs an encoder-decoder structure. The encoder progressively reduces the image's spatial resolution and extracts deep features through multiple Swing Transformer blocks and downsampling layers (such as Patch Merging layers). The decoder, on the other hand, progressively restores the image's spatial resolution through multiple Swing Transformer blocks and upsampling layers, while simultaneously preserving shallow feature information using skip connections.

[0086] The Swin Transformer is a transformer model based on a sliding window attention mechanism that can effectively capture local and global feature information in an image. Each Swin Transformer block consists of 8 Swin Transformer layers. Each layer performs self-attention computation locally through a sliding window mechanism, while gradually increasing the receptive field using a hierarchical structure to capture multi-scale contextual information.

[0087] Will Input to UNet feature extraction M VFE In (·), the process of extracting high-level and multi-scale deep features can be represented as:

[0088] F deep =M VFE (F shallow )

[0089] in, Consistent with the shallow feature dimension, C represents the number of channels (96 in this invention).

[0090] Image reconstruction module. This module uses 3×3 convolutional layers to generate a denoised image from deep features. The final image reconstruction process is represented by the following formula:

[0091]

[0092] in, This represents the final output of the model. H and W represent the height and width of the image, respectively, and 3 represents the number of channels in the image (i.e., RGB channels).

[0093] It's worth noting that the convolutional layers in the shallow feature extraction module are used to extract shallow features (i.e., low-level features of the image), while the convolutional layers in the reconstruction module are used to generate the final denoised image from deep features. Although these two convolutional layers have the same structure—both are 3×3 convolutional layers—their input and output dimensions differ. The former processes low-level features, while the latter generates the final denoised image. Their convolutional operations and implementations are similar, but their functions differ: the former aims to extract features, while the latter is used to reconstruct the image.

[0094] Step 2. Data augmentation: Perform various augmentation operations on the input image:

[0095] During training, low-quality images (noisy images) and their corresponding high-quality images (clean images) are first padded to ensure that the image size matches the model input requirements and to avoid loss of edge information. Next, random cropping is performed to crop images from different regions, increasing the diversity of training samples and enabling the model to learn denoising features at different image locations. Furthermore, random geometric transformations, such as horizontal flipping and rotation, enhance the diversity of the training data, allowing the model to adapt to images from different directions and angles. Building on this, to simulate real-world Gaussian noise environments, noise of varying intensities (σ=5 to σ=50) is added to real clean images to create a noisy image set, ensuring the model can handle images with varying noise levels.

[0096] Step 3. During model training, optimize the network using a hybrid loss function:

[0097] First, a first loss value is determined based on the pixel values ​​of the denoised image generated by inputting the noisy image into the model, the pixel values ​​of the real clean image, and a first loss function. The first loss function is, for example, the mean absolute value error loss function, which can be expressed as:

[0098]

[0099] in, For the denoised image output by the model, I $％ea( For a true, clean image.

[0100] Then, a second loss value is determined based on the pixel values ​​of the denoised image generated by inputting the noiseless image into the model, the pixel values ​​of the real sample image, and the second loss function. The second loss function is, for example, the mean absolute value error loss function, which can be expressed as:

[0101]

[0102] in, For the noise-free image after denoising, I $％ea( For a true, clean image.

[0103] Next, the loss function value can be determined based on the first loss value and the second loss value. In this embodiment of the invention, the first loss value and the second loss value are summed to obtain the loss function value:

[0104]

[0105] Where λ1 and λ ρ These are hyperparameters (both are set to 1 in this example), used to control the weights of the two parts of the loss function.

[0106] Step 4: Dynamic optimization. Based on the denoising performance of the model during training, dynamically adjust the model's learning rate.

[0107] To improve stability in the early stages of training and gradually increase the learning rate, a warm-up strategy was adopted in the early stages of training, gradually increasing the learning rate to a predetermined value; in the later stages of training, a cosine annealing strategy was adopted, gradually decreasing the learning rate to smooth the convergence process of the model and avoid overfitting.

[0108] Specifically, a warm-up strategy is first employed. During the warm-up phase, the learning rate is gradually increased from a small initial value until it reaches a preset initial learning rate η. 0a12345 The learning rate is maintained at this value until the end of the warm-up phase. The learning rate adjustment formula is:

[0109]

[0110] Where η6 is the learning rate of the current training round t steps, η wa9:3；p The target learning rate at the end of the warm-up phase is set to 2e-4 in this invention, where η is the learning rate. . This is the learning rate at the start of the warm-up phase, typically set to 1e-6. 0a12345 This is the total number of training rounds during the warm-up phase, which is set to 3 in this invention.

[0111] After the warm-up phase, the learning rate enters the cosine annealing phase, gradually decreasing as training progresses according to a cosine function. The learning rate adjustment formula for the cosine annealing strategy is:

[0112]

[0113] Where η6 is the learning rate at step t of the current training round, η 2i( The minimum learning rate during training is typically set to 1e-6, where η is the minimum learning rate. wa9:3；p It is the target learning rate at the end of the warm-up phase; T HIHa％ T is the total number of training rounds. 0a12345 This refers to the total number of training rounds during the warm-up phase, which are set to 200 and 3 respectively in this invention. The cosine annealing strategy will cause the learning rate to decrease from η... wa9:3；p Initially, it gradually decreases to η. 2i( This helps the optimizer to fine-tune parameters as it approaches the optimal solution.

[0114] S5. Model optimization: Update model parameters using an adaptive optimizer.

[0115] First, an optimizer based on the Adam optimization algorithm is selected. This algorithm is a widely used adaptive optimization method that can automatically adjust the step size of parameter updates during model training, thereby improving training efficiency and convergence performance. Specifically, the update formula of the Adam optimizer is as follows:

[0116]

[0117] Where θ6 represents the model parameters at the current training epoch (step t), and η6 represents the current learning rate. and ε is the first and second moment estimate of the gradient, and ε is a minimal constant to prevent division by zero error, usually taken as 1e-8.

[0118] First-order moment estimation and second-order moment estimation The update rules are as follows:

[0119] m6=β1m 631 +(1-β6)g6

[0120] v6 = β ρ v 631 +(1-β ρ g6 ρ

[0121]

[0122] Where g6 is the gradient of the current training round t, β1 and β ρ The attenuation rate hyperparameters for estimating the first and second moments are typically taken as β1 = 0.9 and β2 = 0.9. ρ =0.999.

[0123] In each training round, the optimizer calculates the gradient using backpropagation based on the model's loss function and updates the model's weight parameters. As training progresses, the learning rate is gradually adjusted to achieve finer optimization and further improve image denoising performance. Through these optimization processes, the model gradually approaches the optimal state, ultimately outputting the best image denoising model.

[0124] Experimental results

[0125] 1. Dataset

[0126] To evaluate the effectiveness of the method, the denoising model was trained end-to-end on the DIV2K dataset, and the performance of image denoising was evaluated on two commonly used datasets, Kodak and BSD.

[0127] The DIV2K dataset is a popular dataset for image processing research. It contains high-resolution images from various fields, typically 2K (2048x1080 pixels) or higher. The images cover a wide range of scenes, including natural landscapes, cityscapes, and indoor photographs. This diversity makes DIV2K suitable for a variety of image processing tasks.

[0128] The Kodak dataset, released by Eastman Kodak in 1994, contains 24 standard test images used to evaluate image processing and compression algorithms. These images cover various scenes and themes, including people, natural landscapes, and indoor scenes, with each image having a resolution of 768×512 pixels.

[0129] The BSD dataset is a commonly used dataset for image segmentation and computer vision tasks. It was originally created by the Computer Vision Research Group at the University of California, Berkeley (UC Berkeley) to provide a wide range of natural scene images for research and evaluation in image segmentation and vision-related tasks.

[0130] 2. Evaluation Indicators

[0131] Evaluation metrics in image denoising are used to quantitatively assess the performance of denoising algorithms. Below are two common image denoising evaluation metrics and their calculation formulas. PSNR is one of the metrics for measuring image quality, used to represent the quality of an image. It is the ratio of the maximum possible peak value of the original image to the mean square error (MSE); a higher value is better.

[0132]

[0133] Where MAX is the maximum pixel value of the image, and PSNR is in dB.

[0134] SSIM measures the structural similarity between two images, including brightness, contrast, and structure. SSIM values ​​range from -1 to 1, with values ​​closer to 1 indicating higher image quality.

[0135]

[0136] Where μ represents the mean, σ represents the standard deviation, and σ xy c is the covariance of x and y, and c1 and c2 are constants that maintain stability, usually (0.01 × L). 2 and (0.03×L) 2 , where L is the dynamic range of pixel values.

[0137] 3. Performance Comparison

[0138] This section compares the performance of the model of this invention, the baseline model, and classic image denoising algorithms on two datasets:

[0139] CBM3D [Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on image processing, 16(8):2080–2095, 2007.]: An image denoising method based on block matching and 3D transform filtering, which performs collaborative denoising through similar image blocks.

[0140] DNCNN [Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian de-noiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.]: An end-to-end image denoising method based on convolutional neural networks, which efficiently extracts noise features through residual learning and batch normalization, and achieves robust denoising for various noise levels.

[0141] FFDNet [Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9): 4608–4622, 2018]: An efficient and flexible image denoising method that combines denoising performance and processing speed by introducing an adjustable noise level map and a multi-scale architecture, suitable for denoising tasks with different noise intensities and image resolutions.

[0142] IRCNN [Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 126–135, 2017]: An iteratively optimized image denoising method that combines deep convolutional networks and image priors to progressively reduce noise through residual learning, achieving efficient image restoration.

[0143] SUNet (Baseline) [Chi-Mao Fan, Tsung-Jung Liu, and Kuan-Hsien Liu. Sunet: Swin Transformer Union for Image Denoising. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pages 2333–2337. IEEE, 2022.]: A method for image denoising using SwinTransformer combined with UNet architecture.

[0144] Table 1. Performance comparison of different methods on the CBSD68 dataset.

[0145]

[0146] Table 2. Performance comparison of different methods on the Kodak24 dataset.

[0147]

[0148]

[0149] In the CBSD68 dataset, when the noise level σ = 15, the proposed method improves the PSNR by 0.33 dB compared to the SUNet baseline model. At a noise level σ = 50, the proposed method maintains its superiority, achieving a PSNR of 28.00 dB and an SSIM of 0.8012, representing improvements of 0.24 dB and 0.0087 respectively compared to the baseline model. Furthermore, experimental results on the Kodak24 dataset further validate the effectiveness of the proposed method. At all noise levels, the proposed method outperforms traditional denoising models, CNN methods, and the SUNet baseline model, fully demonstrating the advantages of image denoising training based on a hybrid loss function.

[0150] 4. Qualitative Analysis

[0151] The visualization of some denoising results of this invention on the CBSD test set shows the denoising results after denoising, such as... Figure 3 As shown, this invention effectively suppresses artifacts and produces sharper image edges.

[0152] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A joint denoising training method combining noise-free images and noisy images, characterized in that, Includes the following steps: S1. Training the image denoising model: The image denoising model is trained using noise-free images and corresponding noisy images as training data. S2. Data Augmentation: The input image is subjected to various enhancement operations, including at least one of rotation, flipping, cropping, and color change. The enhanced input image is then fed into the image denoising model for training, thereby increasing the robustness and generalization ability of the image denoising model. S3. During model training, a hybrid loss function is used to optimize the network; The hybrid loss function consists of two parts: a first loss function composed of the pixel-level loss of the denoised noisy image and the real clean image; and a second loss function composed of the pixel-level loss of the denoised noiseless image and the real clean image. S4. Implement dynamic optimization strategies during training: Based on the current denoising effect, dynamically adjust the model's learning rate to further improve the model's convergence speed and denoising performance. S5. An adaptive optimizer is used to update the parameters during model optimization: After each training iteration, the model performance is monitored using validation set performance metrics. The training process terminates when the validation set performance metrics stabilize after multiple iterations and no longer improve significantly, or when the preset number of training rounds is reached.

2. The joint denoising training method combining noise-free and noisy images according to claim 1, characterized in that, In step S1, the training process takes noiseless images and noisy images with added random Gaussian noise as input, and uses the corresponding real clean images as labels to optimize the image denoising model through supervised learning.

3. The joint denoising training method combining noise-free images and noisy images according to claim 1, characterized in that, In step S1, the image denoising model is an image denoising model called SUNet; SUNet is an image denoising model that combines the Swin Transformer with the UNet architecture; the model consists of three modules: a shallow feature extraction module, a UNet feature extraction module, and an image reconstruction module. Shallow feature extraction module: This module adjusts the feature dimensions of the input image using a 3×3 convolutional layer and extracts low-level features; this process can be represented by the following formula: ； in, This represents the image to be denoised, where H and W represent the height and width of the image, respectively, and 3 represents the number of channels in the image. This represents a shallow feature extraction layer, which is the process of extracting features from an image through convolution operations; This represents shallow features, where C represents the number of channels in the shallow features. UNet feature extraction module: This module adopts an encoder-decoder structure as the main component. The encoder gradually reduces the spatial resolution of the image and extracts deep features through multiple Swing Transformer blocks and downsampling layers. The decoder gradually restores the spatial resolution of the image through multiple Swing Transformer blocks and upsampling layers, while retaining shallow feature information by combining skip connections. Image Reconstruction Module This module uses 3×3 convolutional layers. The denoised image is generated based on deep features; the final image reconstruction process is represented by the following formula: ； in, This represents the final output of the model, where H and W represent the height and width of the image, respectively, and 3 represents the number of channels in the image. This represents the image reconstruction layer, which extracts deep features through convolutional operations. The denoised image is reconstructed.

4. The joint denoising training method combining noise-free and noisy images according to claim 1, characterized in that, In step S2, the data augmentation operation includes image rotation angle, contrast adjustment, brightness change, and random cropping. The augmented image is used to further improve the generalization ability of the model.

5. The joint denoising training method combining noise-free and noisy images according to claim 1, characterized in that, In step S3, the hybrid loss function is composed of the first loss function and the second loss function: First loss function: The noisy image is input into the denoising model to generate the denoised image, and then the pixel values ​​of the real clean image are combined to calculate the first loss value: ； in, The denoised image output by the model. For a truly clean image; L1 norm operation refers to calculating the absolute error of the image at each pixel location and summing all these errors to obtain the image loss value. The second loss function is calculated by inputting the noiseless image into the denoised image generated by the model, and then combining the pixel values ​​of the real clean image. ； in, The image after denoising is a noise-free image. For a truly clean image; The expression for the hybrid loss function is: + ； in, and These are hyperparameters used to control the weights of the two parts of the loss function.

6. The joint denoising training method combining noise-free and noisy images according to claim 1, characterized in that, In step S4, the dynamic optimization strategy refers to adjusting the learning rate based on the denoising effect of the model during training. First, a warm-up strategy is employed; during the warm-up phase, the learning rate starts from... Gradually increase the learning rate until the preset target learning rate is reached. The learning rate is maintained at this value until the end of the warm-up phase. The learning rate adjustment formula is: ； in, It is the learning rate of the current training round t steps. This represents the target learning rate at the end of the warm-up phase. It is the learning rate at the start of the warm-up phase; It is the total number of training rounds during the warm-up phase; After the warm-up phase, the learning rate enters the cosine annealing phase, where it gradually decreases according to a cosine function as training progresses. The learning rate adjustment formula for the cosine annealing strategy is: ； in, It is the learning rate at step t of the current training round. This is the minimum learning rate during the training process. It is the target learning rate at the end of the warm-up phase; This is the total number of training rounds. It represents the total number of training rounds during the warm-up phase.

7. The joint denoising training method combining noise-free and noisy images according to claim 1, characterized in that, In step S5, the adaptive optimizer is the Adam optimizer, which uses a dynamic learning rate adjustment method combined with a weight decay strategy to further improve the training efficiency and denoising effect of the model. The update formula of the Adam optimizer is as follows: ； in, These are the model parameters for the current training epoch, step t. The current learning rate, and For the estimation of the first and second moments of the gradient, A very small constant to prevent division by zero errors; First-order moment estimation and second-order moment estimation The update rules are as follows: ； ； ； in, This represents the gradient of the loss function with respect to the model parameters when the training epoch is t. and The decay rate hyperparameters are estimated for the first and second moments.

Images