Mixed domain double-branch network for image denoising

Abstract

The invention discloses a mixed domain double-branch network for image denoising, which comprises a spatial domain feature extraction block, a frequency domain feature extraction block and a double-domain interaction block, and is characterized in that a double-branch network follows a U-Net architecture and comprises an upper branch network and a lower branch network, and information exchange between an encoder and a decoder is realized through jump connection; the upper branch network is a spatial domain feature extraction block used for extracting spatial domain features, the lower branch network is a frequency domain feature extraction block used for extracting frequency domain features, the double-domain interaction block is used for realizing interactive link between the two branch networks, and contour information of the spatial domain feature extraction block is introduced into the lower branch network. The edge information of the frequency domain feature extraction block is integrated into the feature map of the upper branch network. A spatial domain feature extraction block and a frequency domain feature extraction block are developed, and the noise in a spatial domain and a frequency domain is effectively solved; meanwhile, a double-domain interaction block is designed to enhance effective integration of features from different domain branches, and the denoising performance is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image denoising processing, and in particular relates to a hybrid domain double-branch network for image denoising. Background Art

[0002] Image denoising is a fundamental challenge in low-level vision and image processing, which aims to recover clear, noise-free content from corrupted images. With the rapid advancement of deep learning techniques, a large number of single image denoising methods based on deep learning have emerged, which develop complex network architectures, extract detailed feature representations from images, and utilize large training datasets to achieve effective denoising. Although deep learning-based methods generally exhibit excellent learning capabilities, they may also bring challenges, including a higher number of parameters and longer processing time, which may limit their practical applications in real-world denoising tasks.

[0003] Inspired by the latest progress in image restoration technology that integrates frequency domain insights, this application proposes an innovative dual-branch denoising network HDDNet. This network model is able to process spatial and frequency domain features differently across branches, effectively reducing computational requirements while maintaining denoising performance without significantly increasing the number of parameters; a dual-domain interaction block (DIB) is designed to promote the effective integration of features in different domains, enabling the network to handle noise in complex scenes and enhance denoising capabilities while maintaining low computational complexity. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a hybrid domain dual-branch network for image denoising, which combines spatial domain and frequency domain information to fully utilize the image properties of the two domains, and achieves the optimal balance between denoising performance and computational efficiency by integrating these data types into the network.

[0005] In order to solve the above technical problems, the present invention is implemented in the following ways:

[0006] A hybrid domain dual-branch network for image denoising includes a spatial domain feature extraction block (SDFEB), a frequency domain feature extraction block (FDFEB) and a dual-domain interaction block (DIB), wherein the dual-branch network follows the U-Net architecture, includes an upper branch network and a lower branch network, and realizes information exchange between an encoder and a decoder through a jump connection; the upper branch network is a spatial domain feature extraction block for extracting spatial domain features, the lower branch network is a frequency domain feature extraction block for extracting frequency domain features, the dual-domain interaction block is used to realize an interactive link between the two branch networks, the contour information of the spatial domain feature extraction block is introduced into the lower branch network, and the edge information of the frequency domain feature extraction block is integrated into the feature map of the upper branch network; the hybrid domain dual-branch network model expression is as follows:

[0007]

[0008] Where X represents the noisy input image, represents the denoised output image.

[0009] Furthermore, the specific method of the hybrid domain dual-branch network is as follows:

[0010] In any input image X∈R Cin×H×W , where Cin, H and W represent the number of input channels, height and width respectively. A 3×3 convolutional layer is used to process the noisy input image, extract the shallow feature F0, and send the shallow feature F0 to two symmetrical upper and lower branch networks with four-level encoding and decoding stages to generate deep features F respectively. spa and F fre ∈R C×H×W ,The upper branch network contains a spatial domain feature extraction block, which uses a series of convolution, pooling, and activation operations to capture local details and global semantic information.,In the lower branch network, the frequency domain feature extraction module uses Fourier transform to convert the image from the spatial domain to the frequency domain, and extracts,features related to the frequency distribution;

[0011] In the encoding stage, the input features undergo continuous convolution and pooling operations, gradually reducing the spatial resolution while increasing the number of feature channels; in the decoding stage, PixelShuffle upsampling is used to restore the low-resolution features to their original size, retaining feature clarity and details, and skip connections are used to merge the upsampled features with their corresponding encoder features; the downsampled features from the encoder and the upsampled features from the decoder are processed by the dual-domain interaction block and added element-wise to the corresponding features of the opposite branch to obtain the updated features F′ spa and F′ fre , connected and passed through a 3×3 convolutional layer to produce a residual image X r ∈R Cin×H×W , and then generate the restored image

[0012] Furthermore, the spatial domain feature extraction block (SDFEB) uses a 3×3 deep convolution layer, and applies a 1×1 convolution layer before the 3×3 deep convolution layer to increase the number of channels in the feature map, and then applies a GELU activation function layer, and then a 1×1 convolution layer to reduce the number of channels; then it passes through a feed-forward network (FFN), which is composed of a normalization layer, a 1×1 convolution layer, a GELU activation function layer, and a 1×1 convolution layer in sequence. The output of the lth block is represented by X L , the calculation process expression of SDFEB is as follows:

[0013] X′ L =X L-1+Conv1×1(GELU(Conv3×3(Conv1×1(LN(X L-1 )))))

[0014] X L =X′ L +Conv1×1(GELU(Conv1×1(LN(X′ L ))))

[0015] Among them, LN represents layer normalization, GELU represents activation function, Conv3×3 represents 3×3 depth convolution, Conv1×1(·) represents convolution operation using 1×1 filter, and X′ L represents the intermediate variable, X L-1 Represents the output of the L-1th SDFEB block.

[0016] Furthermore, the frequency domain feature extraction block uses a 3×3 depth convolution layer, and applies a 1×1 convolution layer operation before the 3×3 depth convolution layer to obtain W q , W k and W v Features, use fast Fourier transform (FFT) to get W in the frequency domain q , W k and W v The self-attention weight V fre , which is expressed as follows:

[0017]

[0018] Where F(·) represents the two-dimensional discrete Fourier transform that transforms the image signal into the frequency domain. represents the complex conjugate transpose operation;

[0019] Apply spatial projection F in the spatial domain v To capture this ignored information, obtain the corrected feature, and combine the feature with the frequency domain self-attention, the expression is as follows:

[0020] V spa =L(F -1 (V fer )·F(W v ))

[0021] Among them, F -1 (·) denotes inverse FFT, L(·) denotes layer normalization;

[0022] Finally, the expression for generating the FDFEB output feature is as follows:

[0023] X out =X+Conv1×1(V spa )

[0024] Among them, Conv1×1(·) indicates the convolution operation using a 1×1 filter, and X represents the input feature of FDFEB.

[0025] Furthermore, the dual-domain interaction block consists of a 7×7 deep convolutional layer and a 7×7 convolutional layer with two large kernel convolutional layers, which aims to enhance the frequency domain features to reduce the computational cost and refine the spatial domain features to promote information exchange between spatial domains.

[0026] The loss function plays a vital role in determining the performance of the model. Preferably, a weighted combination of spatial domain loss and frequency domain loss is used to guide network training, which is expressed as follows:

[0027]

[0028] Among them, L spa represents the spatial domain loss function, L fre represents the frequency domain loss function;

[0029] By introducing the hyperparameter λ, the above two loss functions are combined to obtain the total loss function of HDDNet, which is expressed as follows:

[0030] L total =L spa +λL fre

[0031] Among them, λ represents the ratio between different loss functions of the balance weights, and we set λ = 0.1 through experiments.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The HDDNet proposed in this application is an innovative network designed for the task of blind image denoising, and shows a high level of proficiency in processing synthetic noise and real noise. For the two branches of the model, a spatial domain feature extraction module (SDFEB) and a frequency domain feature extraction module (FDFEB) are developed, focusing on effectively solving the noise in the spatial and frequency domains. At the same time, a dual-domain interaction block (DIB) is designed to enhance the effective integration of features from different domain branches and improve the denoising performance. The performance of HDDNet is comparable to that of many mainstream denoising methods, and even shows stronger competitiveness in some aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the HDDNet structure of the present invention;

[0035] Figure 2 It is a schematic diagram of a spatial domain feature extraction block of the present invention;

[0036] Figure 3 A schematic diagram of a frequency domain feature extraction block of the present invention;

[0037] Figure 4 A schematic diagram showing the comparison of grayscale image denoising on the Set12 dataset with a noise level of σ=50 according to the present invention;

[0038] Figure 5 A schematic diagram showing the comparison of color image denoising on the McMaster dataset with a noise level of σ=50 according to the present invention;

[0039] Figure 6 This is a schematic diagram of the comparison of real image denoising on the SIDD dataset based on the present invention. DETAILED DESCRIPTION

[0040] The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and specific examples.

[0041] like Figure 1 As shown, a hybrid domain dual-branch network for image denoising includes a spatial domain feature extraction block (SDFEB), a frequency domain feature extraction block (FDFEB) and a dual-domain interaction block (DIB), wherein the dual-branch network follows the U-Net architecture, which includes an upper branch network and a lower branch network, and realizes information exchange between the encoder and the decoder through jump connections; the upper branch network is a spatial domain feature extraction block for extracting spatial domain features, and the lower branch network is a frequency domain feature extraction block for extracting frequency domain features. The dual-domain interaction block is used to realize interactive links between the two branch networks, and the contour information of the spatial domain feature extraction block is introduced into the lower branch network, and the edge information of the frequency domain feature extraction block is integrated into the feature map of the upper branch network; the hybrid domain dual-branch network model expression is as follows:

[0042]

[0043] Where X represents the noisy input image, represents the denoised output image.

[0044] Furthermore, the specific method of the hybrid domain dual-branch network is as follows:

[0045] In any input image X∈R Cin×H×W , where Cin, H and W represent the number of input channels, height and width respectively. A 3×3 convolutional layer is used to process the noisy input image, extract the shallow feature F0, and send the shallow feature F0 to two symmetrical upper and lower branch networks with four-level encoding and decoding stages to generate deep features F respectively. spa and F fre ∈R C×H×W,The upper branch network contains a spatial domain feature extraction block, which uses a series of convolution, pooling, and activation operations to capture local details and global semantic information.,In the lower branch network, the frequency domain feature extraction module uses Fourier transform to convert the image from the spatial domain to the frequency domain, and extracts,features related to the frequency distribution;

[0046] In the encoding stage, the input features undergo continuous convolution and pooling operations, gradually reducing the spatial resolution while increasing the number of feature channels. This dimensionality reduction reduces computational complexity while retaining essential information. In the decoding stage, PixelShuffle upsampling is used to restore low-resolution features to their original size, retaining feature clarity and details. In order to further enhance the restored output, a jump connection is used to merge the upsampled features with their corresponding encoder features. This integration compensates for the loss of details due to downsampling, thereby improving the quality of the restored image. The downsampled features from the encoder and the upsampled features from the decoder are processed by the dual-domain interaction block and added element-wise to the corresponding features of the opposite branch to obtain the updated features F′. spa and F′ fre , connected and passed through a 3×3 convolutional layer to produce a residual image X r ∈R Cin×H×W , represents the difference between the original image and the restored image, and then generates the restored image

[0047] Convolutional layers are crucial in CNN because of their powerful feature extraction capabilities. As the network depth increases, traditional convolution operations will cause the computational requirements to increase rapidly. To solve this problem, Figure 2 As shown in FIG. 1 , the spatial domain feature extraction block (SDFEB) uses a 3×3 deep convolution layer to reduce the computational complexity, and applies a 1×1 convolution layer before the 3×3 deep convolution layer to increase the number of channels in the feature map, followed by a GELU activation function layer, and then a 1×1 convolution layer to reduce the number of channels, improve computational efficiency and reduce model parameters; then it passes through a feed-forward network (FFN), which is composed of a normalization layer, a 1×1 convolution layer, a GELU activation function layer and a 1×1 convolution layer in sequence, to support further feature extraction and enhance model performance. The output of the lth block is represented by X L , the calculation process expression of SDFEB is as follows:

[0048] X′ L =X L-1 +Conv1×1(GELU(Conv3×3(Conv1×1(LN(X L-1 )))))

[0049] X L =X′ L+Conv1×1(GELU(Conv1×1(LN(X′ L ))))

[0050] Among them, LN represents layer normalization, GELU represents activation function, Conv3×3 represents 3×3 depth convolution, Conv1×1(·) represents convolution operation using 1×1 filter, and X′ L represents the intermediate variable, X L-1 Represents the output of the L-1th SDFEB block.

[0051] The main goal of designing the basic module FDFEB for the frequency domain branch is to analyze the input image in the complex domain, capture various frequency components and extract local frequency domain features. Similar to the SDFEB block, it is followed by an identical feed-forward network (FFN) to further extract features and enhance model performance. Figure 3 As shown, the frequency domain feature extraction block uses a 3×3 depth convolution layer, and applies a 1×1 convolution layer operation before the 3×3 depth convolution layer to obtain W q , W k and W v Features, use fast Fourier transform (FFT) to get W in the frequency domain q , W k and W v The self-attention weight V fre , which is expressed as follows:

[0052]

[0053] Where F(·) represents the two-dimensional discrete Fourier transform that transforms the image signal into the frequency domain. represents the complex conjugate transpose operation;

[0054] Calculating self-attention in the frequency domain may ignore some spatial information. To solve this problem, a spatial projection F is applied in the spatial domain. v To capture the ignored information (the noise information that is processed incorrectly in the frequency domain or the new noise information introduced in the frequency domain), obtain the corrected features, and combine the features with the frequency domain self-attention. The expression is as follows:

[0055] V spa =L(F -1 (V fre )·F(W v ))

[0056] Among them, F -1 (·) denotes inverse FFT, L(·) denotes layer normalization;

[0057] Finally, the expression for generating the FDFEB output feature is as follows:

[0058] X out =X+Conv1×1(V spa )

[0059] Among them, Conv1×1(·) indicates the convolution operation using a 1×1 filter, and X represents the input feature of FDFEB.

[0060] The dual-branch structure allows the network to capture both spatial and frequency domain information. The basic tandem approach is difficult to fully exploit the collaborative advantages of cross-domain branches. To address this problem, a dual-domain interaction block (DIB) is integrated to effectively combine deep features into their respective feedforward sequences, thereby enhancing feature complementarity. Experimental results demonstrate the key role of DIB in improving information integration.

[0061] Specifically, DIB implements interactive links between the two branches to promote feature enhancement. To achieve this goal, the dual-domain interaction block combines two large-kernel convolutional layers: a 7×7 deep convolutional layer designed to enhance frequency domain features and reduce computational cost; and a 7×7 convolutional layer designed to refine spatial domain features and promote information exchange between spatial domains.

[0062] The loss function plays a vital role in determining the performance of the model. Preferably, a weighted combination of spatial domain loss and frequency domain loss is used to guide network training, which is expressed as follows:

[0063]

[0064] Among them, L spa represents the spatial domain loss function, L fre represents the frequency domain loss function;

[0065] By introducing the hyperparameter λ, the above two loss functions are combined to obtain the total loss function of HDDNet, which is expressed as follows:

[0066] L total =L spa +λL fre

[0067] Among them, λ represents the ratio between different loss functions of the balance weights, and we set λ = 0.1 through experiments.

[0068] Example:

[0069] The training dataset is divided into two subsets: a Gaussian synthetic noise image training set and a real noise image training set; for the synthetic noise image training set, the following four public datasets are selected: the BSD dataset containing 432 natural images, the DIV2K dataset containing 800 natural images, the Flickr2K dataset containing 2,650 natural images, and the WED dataset containing 4,744 natural images. Together, these datasets constitute a large synthetic noise training set consisting of 8,626 natural images. In order to further enrich the training dataset and improve the generalization ability of the model, each image is randomly cropped, resulting in a total of 71,580 image patches. In order to train the real noise image denoising model, we select HR noise images and their corresponding near-noise images from the SIDD dataset as the training set. In order to maximize the use of these images, we randomly crop them into 256×256 patches.

[0070] The test dataset is also divided into two parts: synthetic noise image test set and real noise image test set. The synthetic noise image test set includes public datasets BSD68, Set12, CBSD68, Kodak24 and McMaster for evaluating grayscale and color Gaussian noise denoising models. In addition, SIDD dataset and PolyU dataset were selected for testing.

[0071] 1) Experimental setup

[0072] The parameter configuration for training the HDDNet denoiser is as follows: AdamW optimizer, hyperparameters β1 and β2 are set to 0.9, the initial learning rate is 0.001, and it is dynamically adjusted using a cosine annealing scheme. Throughout the training process, image patches of 256 × 256 pixels are used as input, and the batch size is 32, which helps to effectively capture local features and improve training efficiency. The experiments were conducted on Ubuntu 18.04, using PyTorch 1.11.0 and Python 3.9.18. To accelerate training, an Nvidia GeForce GTX 3090 GPU with Nvidia CUDA 11.3 was used.

[0073] 2) Ablation experiment

[0074] All models are trained for 40,000 iterations on the real noisy image denoising task and evaluated using the SIDD dataset.

[0075] Impact of different DIB configurations:

[0076] As shown in Table 1, ID1 (which does not include any DIB components) is used as a reference point for comparison, integrating various DIB configurations and observing the resulting performance changes. ID 2 uses standard convolutional layers for both domain conversions, showing significant improvements over the baseline; ID 3 exclusively utilizes deep convolutions for frequency-to-spatial conversion, demonstrating particularly effective enhancements, indicating its advantage in capturing essential features in the frequency domain; ID 4 uses deep convolutions for spatial-frequency transformation and uses standard convolutions in contrast, providing balanced improvements and emphasizing the importance of selective use of deep operations; ID 5 shows only moderate enhancements, indicating that not all DIB configurations produce equally significant advantages. The strategic use of deep convolutions, especially in frequency-to-spatial conversion, can significantly improve performance, and combined with the DIB mechanism enhances the model's ability to effectively process and integrate spatial and frequency domain information, resulting in excellent image reconstruction results.

[0077] Table 1 Impact of different DIB configurations

[0078]

[0079] The impact of different DIB convolution kernel sizes:

[0080] ID 3, which performed best in the ablation experiments with different DIB configurations, is used as the baseline, and the effect of kernel size on performance is further explored. As shown in Table 2, various combinations of standard convolution (Conv) and depthwise convolution (DWconv) kernels with sizes of 3×3, 5×5, and 7×7 are explored, and the results are measured using PSNR (dB) and SSIM metrics. From the results, increasing the kernel size from 3×3 to 7×7 in the Conv and DWconv layers improves the performance, which indicates that larger convolution kernels are more effective in capturing complex image features, resulting in better overall image quality.

[0081] Table 2 Effect of different DIB convolution kernel sizes

[0082]

[0083]

[0084] The impact of different modules:

[0085] As shown in Table 3, ablation studies are performed to investigate the contributions of different modules in the model: spatial domain feature extraction module (SDFEB), frequency domain feature extraction module (FDFEB), and dual feature extraction module (FDFEB), dual domain interaction block (DIB). When only SDFEB is used, the model performs well; replacing SDFEB with FDFEB brings significant improvements, as FDFEB better captures and fuses deep features, thereby improving the overall performance. When SDFEB and FDFEB are used in combination, the model obtains further performance gains, indicating that these modules work together to extract and process features more effectively. The most substantial progress is observed when all three components including DIB are incorporated, and the inclusion of DIB improves the feature representation ability of the model, demonstrating its importance in capturing multi-scale information. This comprehensive feature extraction and fusion mechanism provided by SDFEB, FDFEB, and DIB jointly enhances the model's ability to produce high-quality image reconstructions.

[0086] Table 3 Impact of different blocks

[0087]

[0088] 3) Gaussian grayscale image denoising results

[0089] The performance of the proposed blind denoising method HDDNet is evaluated on two widely used grey Gaussian denoising benchmark datasets: Set12 and BSD68, and HDDNet is compared with a series of representative denoising methods, including BM3D, TNRD, DnCNN, BUIFD, IRCNN, FFDNet, DSNetB, ADNet, AINDNet, NIFBGDNet, NLED, S2S, R2R, CDNet, DudeNet, MWDCNN and DRANet.

[0090] Quantitative comparison:

[0091] Table 4 shows the average PSNR performance of various denoising methods under different noise levels (σ=15, 25, 50) on the Set12 dataset. The table shows that HDDNet achieves higher PSNR values ​​at all noise levels, demonstrating excellent performance. That is, when σ=15, the average PSNR of HDDNet is 33.07dB, which is 0.07dB better than the second best method DRANet; when σ=25, the average PSNR of HDDNet reaches 30.77dB, which exceeds DRANet by 0.08dB; when σ=50, the average PSNR of HDDNet is 27.71dB, which exceeds DRANet by 0.09dB.

[0092] Table 4 Average PSNR (dB) of various methods on Set12 dataset when the noise level is 15, 25 and 50

[0093]

[0094]

[0095] Similarly, Table 5 shows the average PSNR performance of various denoising methods at different noise levels (σ=15, 25, 50) on the BSD68 dataset. The table shows that the method proposed in this study, HDDNet, consistently achieves higher PSNR values ​​at all noise levels, indicating superior performance. When σ=15, the average PSNR of HDDNet is 31.85dB, which is 0.06dB higher than DRANet; when σ=25, HDDNet reaches 29.42dB, which is 0.06dB better than DRANet; when σ=50, HDDNet reaches 26.53dB, which is 0.06dB better than DRANet.

[0096] Table 5 Average PSNR (dB) of various methods on the BSD68 dataset when the noise level is 15, 25 and 50

[0097]

[0098]

[0099] Qualitative results and analysis:

[0100] like Figure 4 As shown in Figure 1, the results of the Set12 dataset show a visual comparison of grayscale image denoising in several challenging situations with a noise level of 50. It is observed that DnCNN and IRCNN are largely ineffective in removing noise, while FFDNet produces blurry and over-smoothed results, and although DRANet successfully removes noise, it also loses a lot of details. In contrast, the proposed HDDNet achieves high-quality visual results while maintaining a higher PSNR value.

[0101] 4) Gaussian color image denoising results

[0102] The performance of the proposed blind denoising method HDDNet is evaluated on three widely used color Gaussian denoising benchmark datasets: CBSD68, Kodak24 and McMaster. HDDNet is compared with several representative denoising methods, including BM3D, DnCNN, BUIFD, IRCNN, FFDNet, DSNetB, RIDNet, VDN, BRDNet, ADNet, DudeNet, AirNet, MWDCNN and DRANet.

[0103] Quantitative comparison:

[0104] Table 6 shows the average PSNR performance of various denoising methods at different noise levels (σ=15, 25, 50) for the CBSD68, Kodak24, and McMaster color image datasets. HDDNet consistently achieves higher PSNR values ​​than other methods at all noise levels, indicating its superior denoising capabilities. For the CBSD68 dataset, HDDNet has average PSNR values ​​of 34.23dB, 31.61dB, and 28.42dB, respectively, outperforming the second-best method DRANet by 0.05dB at each noise level. On the Kodak24 dataset, HDDNet has average PSNR values ​​of 35.15dB, 32.71dB, and 29.65dB, outperforming DRANet by 0.13dB, 0.12dB, and 0.15dB, respectively. For the McMaster dataset, the average PSNR values ​​of HDDNet are 35.13dB, 32.89dB, and 29.85dB, which are 0.04dB, 0.05dB, and 0.08dB higher than those of DRANet, respectively.

[0105] Table 6 Average PSNR (dB) of various methods on color images with noise levels of 15, 25 and 50

[0106]

[0107]

[0108] Qualitative results and analysis:

[0109] like Figure 5 As shown in Figure 2, the results on the McMaster dataset show a visual comparison of color image denoising in several challenging situations with a noise level of 50. DnCNN and IRCNN perform poorly in removing real noise, while FFDNet causes image blur and over-smoothing, and although DRANet successfully removes noise, it loses a lot of details. The proposed HDDNet achieves superior visual results and higher PSNR values.

[0110] 5) Real image denoising results

[0111] In addition to synthetic noise, the effectiveness of HDDNet in denoising real image noise is evaluated. The performance of the blind denoising method HDDNet is evaluated on two widely used real image denoising benchmark datasets: SIDD and PolyU. HDDNet is compared with several representative denoising methods, including BM3D, TWSC, MCWNNM, DnCNN, CBDNet, RIDNet, AINDNet, VDN, SADNet, DANet+, DeamNet, VDIR, CycleISP, HI-GAN, LIGN, MPRNet, MIRNet, NAFNet, Uformer, CFNet and DRANet.

[0112] Quantitative comparison:

[0113] Table 7 shows the PSNR and SSIM performance indicators of real image denoising on the SIDD dataset. HDDNet achieved the highest scores in both PSNR and SSIM, indicating its strong denoising ability. Specifically, HDDNet's PSNR score is 39.59dB, which is 0.09dB higher than the second best method DRANet; in terms of SSIM, HDDNet's score is 0.958, which is 0.01 higher than DRANet.

[0114] Table 7 Real image denoising results on the SIDD dataset

[0115]

[0116] Table 8 lists the PSNR and SSIM performance indicators for real image denoising on the PolyU dataset. HDDNet achieved the highest result on the SSIM indicator, indicating its excellent denoising performance. Specifically, for SSIM, HDDNet scored 0.979, exceeding the second best method CFNet by 0.027. For PSNR, HDDNet scored 37.09dB, which is not the highest, but still reflects a higher signal-to-noise ratio compared to other advanced methods.

[0117] Table 8 Real image denoising results on the PolyU dataset

[0118]

[0119]

[0120] Qualitative results and analysis:

[0121] like Figure 6As shown in Figure 2, the results of the SIDD dataset test show a visual comparison of real image denoising for several challenging cases. CBDNet and RIDNet perform poorly in removing real noise, while CycleISP tends to produce blurry and over-smoothed images. Although DRANet successfully removes noise, a lot of details are lost in the process. The HDDNet of the present invention shows excellent visual results and achieves higher PSNR values.

[0122] 6) Model complexity analysis

[0123] To evaluate the processing speed of different denoising models when the noise level is 25, three sizes of synthetic images are randomly selected in different channels: 256 × 256, 512 × 512, and 1024 × 1024. The time values ​​in Table 9 represent the average execution time, which is calculated by 20 runs for each model.

[0124] Compared with BM3D, TNRD, MCWNNM, AINDNet, VDN, AirNet, VDIR, and DRANet, the HDDNet method shows a shorter inference time. However, the running time of HDDNet is longer than DnCNN, BUIFD, IRCNN, FFDNet, ADNet, SADNet, and DANet+; despite this, HDDNet still achieves superior denoising performance than these models. These results show that HDDNet has great practical value and potential in denoising applications.

[0125] Table 9 Inference time of various denoising methods

[0126]

[0127]

[0128] Table 10 illustrates the computational complexity of various denoising methods using ptflop calculation. The table shows that HDDNet exhibits significant performance improvements in denoising tasks, especially in terms of parameter count and computational complexity. HDDNet has only 3.33M parameters, which is much lower than other methods; the computational complexity of HDDNet is 12.71GMAC, which is significantly lower than most other methods, including MIRNet and MPRNet, indicating that HDDNet achieves efficient computational performance while maintaining a low number of parameters.

[0129] Table 10 Complexity of various noise reduction methods

[0130]

[0131] The above description is only an implementation mode of the present invention. It is stated again that for ordinary technicians in this technical field, several improvements can be made to the present invention without departing from the principle of the present invention. These improvements are also included in the protection scope of the claims of the present invention.

Claims

1. A hybrid domain dual-branch network for image denoising, characterized in that: It includes a spatial domain feature extraction block, a frequency domain feature extraction block and a dual-domain interaction block, wherein the dual-branch network follows the U-Net architecture, which includes an upper branch network and a lower branch network, and realizes information exchange between the encoder and the decoder through jump connection; the upper branch network is a spatial domain feature extraction block for extracting spatial domain features, and the lower branch network is a frequency domain feature extraction block for extracting frequency domain features. The dual-domain interaction block is used to realize the interactive link between the two branch networks, and the contour information of the spatial domain feature extraction block is introduced into the lower branch network, and the edge information of the frequency domain feature extraction block is integrated into the feature map of the upper branch network; the mixed domain dual-branch network model expression is as follows: Where X represents the noisy input image, represents the denoised output image.

2. The hybrid domain dual-branch network for image denoising according to claim 1, characterized in that: The specific method of the hybrid domain dual-branch network is as follows: In any input image X∈R Cin×H×W , where Cin, H and W represent the number of input channels, height and width respectively. A 3×3 convolutional layer is used to process the noisy input image, extract the shallow feature F0, and send the shallow feature F0 to two symmetrical upper and lower branch networks with four-level encoding and decoding stages to generate deep features F respectively. spa and F fre ∈R C×H×W ,The upper branch network contains a spatial domain feature extraction block, which uses a series of convolution, pooling, and activation operations to capture local details and global semantic information.,In the lower branch network, the frequency domain feature extraction module uses Fourier transform to convert the image from the spatial domain to the frequency domain, and extracts,features related to the frequency distribution; In the encoding stage, the input features undergo continuous convolution and pooling operations, gradually reducing the spatial resolution while increasing the number of feature channels; in the decoding stage, PixelShuffle upsampling is used to restore the low-resolution features to their original size, retaining feature clarity and details, and skip connections are used to merge the upsampled features with their corresponding encoder features; the downsampled features from the encoder and the upsampled features from the decoder are processed by the dual-domain interaction block and added element-wise to the corresponding features of the opposite branch to obtain the updated features F′ spa and F′ fre , connected and passed through a 3×3 convolutional layer to produce a residual image X r ∈R Cin×H×W , and then generate the restored image 3. The hybrid domain dual-branch network for image denoising according to claim 1, characterized in that: The spatial domain feature extraction block uses a 3×3 deep convolutional layer, and applies a 1×1 convolutional layer before the 3×3 deep convolutional layer to increase the number of channels in the feature map, and then applies a GELU activation function layer, and then a 1×1 convolutional layer to reduce the number of channels; After that, it passes through a feed-forward network, which is composed of a normalization layer, a 1×1 convolution layer, a GELU activation function layer, and a 1×1 convolution layer. The output of the lth block is represented by X L , the calculation process expression of SDFEB is as follows: X ′ L =X L-1 +Conv1×1(GELU(Conv3×3(Conv1×1(LN(X L-1 ))))) X L =X ′ L +Conv1×1(GELU(Conv1×1(LN(X ′ L )))) Among them, LN represents layer normalization, GELU represents activation function, Conv3×3 represents 3×3 depth convolution, Conv1×1(·) represents convolution operation using 1×1 filter, and X ′ L represents the intermediate variable, X L-1 Represents the output of the L-1th SDFEB block.

4. The hybrid domain dual-branch network for image denoising according to claim 1, characterized in that: The frequency domain feature extraction block uses a 3×3 depth convolution layer, and applies a 1×1 convolution layer operation before the 3×3 depth convolution layer to obtain W q , W k and W v Features, use fast Fourier transform (FFT) to get W in the frequency domain q , W k and W v The self-attention weight V fre , which is expressed as follows: Where F(·) represents the two-dimensional discrete Fourier transform that transforms the image signal into the frequency domain. represents the complex conjugate transpose operation; Apply spatial projection F in the spatial domain v To capture this ignored information, obtain the corrected feature, and combine the feature with the frequency domain self-attention, the expression is as follows: V spa =L(F -1 (V fre )·F(W v )) Among them, F -1 (·) denotes inverse FFT, L(·) denotes layer normalization; Finally, the expression for generating the FDFEB output feature is as follows: X out =X+Conv1×1(V spa ) Among them, Conv1×1(·) indicates the convolution operation using a 1×1 filter, and X represents the input feature of FDFEB.

5. The hybrid domain dual-branch network for image denoising according to claim 1, characterized in that: A weighted combination of spatial domain loss and frequency domain loss is used to guide network training, which is expressed as follows: Among them, L spa represents the spatial domain loss function, L fre represents the frequency domain loss function; By introducing the hyperparameter λ, the above two loss functions are combined to obtain the total loss function of HDDNet, which is expressed as follows: THE total =L spa +λL fre Among them, λ represents the ratio between different loss functions with balanced weights.

Images