Image denoising method based on feature fusion complementary learning

Abstract

The invention discloses an image denoising method based on feature fusion complementary learning, and mainly solves the problems that an existing image denoising method is unstable in performance and a generated denoised image is poor in quality. According to the implementation scheme, the method comprises the following steps: constructing two predictors each comprising a multi-stage encoder and a multi-stage decoder, constructing a feature interaction module comprising a condition weight generator and a convolutional network, and constructing a fusion module comprising a convolutional layer; the output of each level of decoder of the two predictors is connected with the input of the next level of decoder after being subjected to weighted coupling through a feature interaction module, the last level of decoder of the first predictor outputs a first de-noised image, the last level of decoder of the second predictor outputs noise, and the noise is subtracted from the original noise image to obtain a second de-noised image; and the first denoising image and the second denoising image are connected in parallel according to channel dimensions and then input into a fusion module to obtain a final denoising image. According to the method, the image noise can be stably and effectively removed, the texture details of the image can be well restored, and the method can be used for target identification, image conversion and resource exploration.

Description

Technical Field

[0001] The invention belongs to the technical field of digital image processing, and in particular relates to an image denoising method, which can be used for target recognition, image conversion, and resource exploration. Background Art

[0002] Due to environmental and human factors as well as sensor defects, imaging devices inevitably introduce noise when acquiring, manually editing, and subsequently transmitting signals collected from imaging objects. Noisy images can have an adverse effect on many image processing applications. Therefore, image denoising is an important part of digital image processing, and its goal is to reduce or eliminate noise in images and improve image quality and usability. In the field of image denoising, image denoising methods based on deep learning have developed rapidly and have attracted widespread attention from researchers.

[0003] Wu, Yinhu et al. proposed a remote sensing hyperspectral image denoising method based on multi-cycle guidance in the paper "A Remote Sensing Hyperspectral Image Noise Removal Method Based on Multipriors Guidance". The method extracts image features from the hyperspectral image after variance stabilization transformation through a linear self-attention module, and then restores the extracted image features to a noise-free image. Although this method can stably remove noise from the image, the quality of the denoised image needs to be improved.

[0004] Huang, Yongqiang et al. proposed a method for denoising optical coherence tomography images based on disentangled representation in the paper “Noise-Powered Disentangled Representation for Unsupervised Speckle Reduction of Optical Coherence Tomography Images”. The method first extracts noise information from the noisy image through a noise encoder, and then subtracts the extracted noise information from the noisy image to obtain a denoised image. As a noise prediction-based method, this method can obtain a denoised image of good quality, but its performance is not robust enough, especially in challenging scenarios such as high noise intensity, where the denoising performance will be significantly reduced.

[0005] Therefore, how to design an image denoising method that can accurately denoise and well preserve image details has become an urgent problem to be solved in this field. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose an image denoising method based on feature fusion complementary learning convolutional neural network to improve the quality of denoised images and ensure the stability of denoising performance in high noise intensity scenes.

[0007] The technical idea of ​​the present invention is to combine the advantages of deep learning image denoising methods based on noise prediction and denoising content prediction, integrate the fusion of prediction results and the interaction of intermediate feature maps into a unified framework to achieve effective fusion of the two methods, and effectively improve the quality of denoised images.

[0008] According to the above ideas, the implementation scheme of the present invention includes the following:

[0009] 1. An image denoising method based on feature fusion complementary learning, characterized in that it comprises the following steps:

[0010] 1) Obtain a noisy data image set and a denoised data image set in pairs from a public data set, perform normalization preprocessing on them, and change the pixel value range of the image from [0, 255] to [-1, 1]; and divide the preprocessed data image set into a training set and a test set in a ratio of 8:2;

[0011] 2) Constructing a feature interactive complementary learning denoising network:

[0012] 2a) establishing two predictors each including an N-level encoder and an M-level decoder, for predicting the denoised image and the noise respectively;

[0013] 2b) establishing a feature interaction module including a conditional weight generator and a convolutional network for weighted coupling of the outputs of each level of the decoder of the two predictors;

[0014] 2c) establishing a fusion module including a convolutional layer, for fusing the denoised images predicted by the last-stage decoder outputs of the two predictors to obtain a fused denoised image;

[0015] 2d) The outputs of the 1st to M-1th level decoders of the two predictors are connected to the input of the next level decoder after weighted coupling by the feature interaction module, and the Mth level decoders of the two predictors respectively output a first denoised image and noise, and the noise is subtracted from the original noise image to obtain a second denoised image, and then the first denoised image and the second denoised image are connected in parallel according to the channel dimension and connected to the input end of the fusion module to form a feature interaction complementary learning denoising network;

[0016] 3) Design the loss function L of the feature interactive complementary learning denoising network as:

[0017] L=L fin +0.5L n +0.5L rsi

[0018] Among them, Lfin is the loss function of the denoised image output by the fusion module, Lrsi is the loss function of the first denoised image, and Ln is the loss function of the noise output by the predictor;

[0019] 4) Input the training set image into the feature interaction complementary learning denoising network, use the back propagation method to calculate the gradient of the loss function, and use the optimizer to optimize the parameters of the two predictors and the feature interaction module and the fusion module respectively with the goal of predicting the denoised image and the goal of predicting the noise, until the loss function tends to be stable to obtain the trained feature interaction complementary learning denoising network, which includes the trained denoised image predictor and the noise predictor;

[0020] 5) Input the noisy image in the test set into the trained feature interaction complementary learning denoising network to obtain the denoised image.

[0021] 2. An image denoising system based on feature fusion complementary learning, characterized by comprising: a denoising image predictor, a noise predictor, a feature interaction module and a fusion module;

[0022] The denoised image predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract features of image content in a noisy image and input the features to a first-level decoder, and outputs of the first to M-1-th levels of the M-level decoders are weightedly coupled through a feature interaction module and then input to a next-level decoder, and the M-th-level decoder outputs a preliminary denoised image;

[0023] The noise predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract the noise features in the noise image and input the features to the first-level decoder, and the outputs of the first to the M-1-level decoders are weighted coupled through a feature interaction module and then input to the next-level decoder, and the M-level decoder outputs the image noise, and then the image noise output by the predictor is subtracted from the original noise image to obtain a preliminary denoised image;

[0024] The feature interaction module includes a weight generator and a convolutional network layer, which is used to weight the output features of each level of decoders in the two predictors;

[0025] The fusion module includes a convolutional network layer, which is used to fuse the preliminary denoised images output by the last-stage decoders of the two predictors to obtain a fused final denoised image.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] First, the present invention effectively combines the advantages of deep learning denoising methods based on noise prediction and deep learning image denoising methods based on denoised image prediction, and constructs an image denoising network based on feature fusion complementary learning, thereby improving the quality of denoised images.

[0028] Second, the present invention integrates the feature interaction between predictors and the output results of the predictors into one system to achieve a full combination of the two methods, which not only effectively improves the denoising performance of the image, but also ensures the stability of the denoising performance in high noise intensity scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart for realizing the present invention;

[0030] Figure 2 A schematic diagram of the structure of the feature interactive complementary learning denoising network constructed in the present invention;

[0031] Figure 3 for Figure 2 The predictor structure diagram in ;

[0032] Figure 4 for Figure 2 The structure diagram of the feature interaction module in ;

[0033] Figure 5 The structural block diagram of the invented denoising system;

[0034] Figure 6 A comparison of the denoising effects of the present invention and five existing denoising methods in high-noise scenes. DETAILED DESCRIPTION

[0035] The embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0036] Reference Figure 1 , the implementation steps of the present invention include the following:

[0037] Step 1: Acquisition and preprocessing of data sets.

[0038] Download a public image denoising dataset from the public network and perform normalization preprocessing on it to change the pixel value range of the image from [0, 255] to [-1, 1];

[0039] The preprocessed data image set is divided into a training set and a test set in a ratio of 8:2.

[0040] Step 2: Construct an image denoising network based on feature fusion complementary learning.

[0041] 2.1) Two predictors, each including an N-level encoder and an M-level decoder, are established to predict the denoised image and the noise respectively:

[0042] like Figure 3As shown, each predictor includes a convolution layer Conv with a convolution kernel size of 4×4 and a stride of 2, a deconvolution layer DeConv with a convolution kernel size of 4×4 and a stride of 2, a batch normalization layer BN, a LeakRelu activation function, and a Tanh activation function, where:

[0043] The convolution layer is connected to the batch normalization layer to form the first-level encoder. The convolution layer, batch normalization layer, and LeakRelu activation function are connected in sequence to form other levels of encoders. The encoders at each level are then connected in sequence to form an N-level encoder.

[0044] The deconvolution layer, batch normalization layer and Tanh activation function are connected in sequence to form the last level of decoder, the deconvolution layer, batch normalization layer and LeakRelu activation function are connected in sequence to form other levels of decoders, and the decoders at all levels are connected in sequence to form M-level decoders;

[0045] Connect the Nth level encoder to the first level decoder, and connect each level output in the encoder to the corresponding level in the decoder to obtain a predictor;

[0046] 2.2) Construct a feature interaction module to weight the output of each decoder of the two predictors:

[0047] like Figure 4 As shown, the implementation of this step includes the following:

[0048] 2.2.1) Establish a conditional weight generator CWG consisting of an average pooling layer, a linear layer, and a sigmoid activation function cascade to generate weight parameters of the feature map;

[0049] 2.2.2) Create a convolution layer Conv with a convolution kernel size of 1×1 and a step size of 1 1×1 , used to perform weighted processing on each channel of the feature map;

[0050] 2.2.3) According to the weight generator CWG and the convolution layer Conv 1×1 The following interaction relationships constitute the feature interaction module:

[0051] The initial feature PF1 of the first predictor is compared with the PF1 processed by CWG and the PF1 processed by Conv 1×1 The processed initial features PF2 of the second predictor are multiplied to obtain the intermediate features MPF1 of the first predictor:

[0052] MPF1=PF1×CWG(PF1)×Conv 1×1 (PF2)

[0053] The initial feature PF2 of the second predictor is compared with the PF2 processed by CWG and the PF2 processed by Conv 1×1The processed initial feature PF1 of the first predictor is multiplied to obtain the intermediate feature MPF2 of the second predictor:

[0054] MPF2=PF2×CWG(PF2)×Conv 1×1 (PF1);

[0055] The first predictor coupling feature CPF1 is obtained by subtracting MPF1 from the initial feature PF1 of the first predictor and adding MPF2:

[0056] CPF1=PF1-MPF1+MPF2;

[0057] The second predictor coupling feature CPF2 is obtained by subtracting MPF2 from the initial feature PF2 of the second predictor and adding MPF1:

[0058] CPF2=PF2-MPF2+MPF1;

[0059] 2.3) A fusion module is established, which consists of a convolution layer with a convolution kernel size of 1×1 and a step size of 1, and is used to fuse the denoised images predicted by the last level decoder output of the two predictors to obtain a fused denoised image;

[0060] 2.4) The outputs of the 1st to M-1th level decoders in the two predictors are weighted coupled by the feature interaction module and connected to the input of the next level decoder. The Mth level decoders of the two predictors output a first denoised image and noise respectively. The noise is subtracted from the original noisy image to obtain a second denoised image. The first denoised image and the second denoised image are then connected in parallel according to the channel dimension and connected to the input of the fusion module to form an image denoising network based on feature fusion complementary learning.

[0061] Step 3: Construct the loss function of the image denoising network based on feature fusion complementary learning.

[0062] 3.1) Set the loss function L of the fusion module output denoised image fin :

[0063] L fin =L char +0.15L SSIM +0.2L edge

[0064] in is the pixel loss between the predicted result and the standard result, Pre f and gt i Represent the fused denoised image and the standard noise-free image in the training set, respectively, and ε is a constant of 10 -3 ;

[0065] L SSIM =1-SSIM(pref ,gt i ) is the SSIM loss of the predicted result and the standard result, and SSIM is the structural similarity between the predicted result and the standard result;

[0066] is the edge loss between the predicted result and the standard result, Δ is the Laplace operator;

[0067] 3.2) Set the first denoising image loss function L rsi :

[0068] L rsi =L' char +0.15L' SSIM +0.2L' edge

[0069] in is the pixel loss between the predicted result and the standard result, r and gt i represent the first denoised image and the standard noise-free image in the training set respectively;

[0070] L' SSIM =1-SSIM(pre r ,gt i ) is the SSIM loss of the predicted result and the standard result, and SSIM is the structural similarity between the predicted result and the standard result;

[0071] is the edge loss between the predicted result and the standard result;

[0072] 3.3) Set the loss function L of the predictor output noise n :

[0073] L n =L char-n +0.5L asymm

[0074] in is the noise and standard result pixel loss, pre n represents the predicted noise, gt n is the standard noise, gt n It is obtained by subtracting the standard noise-free image from the noisy image in the training set;

[0075] is the asymmetric loss of noise estimation, For all pre n -gt n <0 timepre n -gt n The factorial of , α is a constant set to 0.3;

[0076] 3.4) Using the loss function from step 3.1) to step 3,3), the loss function L of the feature interaction complementary learning denoising network constructed is:

[0077] L=L fin +0.5L n +0.5L rsi .

[0078] Step 4: Train the image denoising network based on feature fusion complementary learning.

[0079] 4.1) Setting the feature interaction complementary learning denoising network to training mode;

[0080] 4.2) Inputting the noisy image in the training set into the network, the noisy image is forward propagated through the feature interactive complementary learning denoising network, the first predictor is used as the denoised image predictor for training and calculates and outputs the first denoised image, the second predictor is used as the noise predictor for training and calculates and outputs the noise, the original noisy image is subtracted from the noise output by the second predictor to obtain the second denoised image, and the fusion module is used as the fusion of the first denoised image and the second denoised image for training and calculates and outputs the fused denoised image;

[0081] 4.3) Use the first denoised image, the fused denoised image and the standard noise-free image to calculate the loss function L rsi , L fin , use the noise output by the second predictor and the standard noise to calculate the loss function L n ;

[0082] 4.4) Calculate the loss function L = L according to the back propagation method fin +0.5L n +0.5L rsi The gradient of

[0083] 4.5) Use the Adam optimizer to update the model parameters according to the change of the loss function gradient;

[0084] 4.6) Repeat process 4.2) to 4.5) until the loss function reaches stability and a trained image denoising network is obtained.

[0085] Step 5: Use the trained denoising network to denoise the image.

[0086] 5.1) Set the feature interaction complementary learning denoising network to test mode;

[0087] 5.2) Input the noisy image in the test set into the trained image denoising network and transform it into a first denoised image through its denoised image predictor; and transform the noisy image in the test set into noise through the noise predictor, and then subtract the noise from the original noisy image to obtain a second denoised image;

[0088] 5.3) The first denoised image and the second denoised image are connected in parallel in the channel dimension, and then transformed into the final denoised image through a fusion module.

[0089] Reference Figure 5 In this example, the image denoising system based on feature fusion complementary learning includes: a denoising image predictor, a noise predictor, a feature interaction module and a fusion module, wherein:

[0090] The denoised image predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract features of image content in a noisy image and input the features to a first-level decoder, and outputs of the first to M-1-th levels of the M-level decoders are weightedly coupled through a feature interaction module and then input to a next-level decoder, and the M-th-level decoder outputs a preliminary denoised image;

[0091] The noise predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract the noise features in the noise image and input the features to the first-level decoder, and the outputs of the first to the M-1-level decoders are weighted coupled through a feature interaction module and then input to the next-level decoder, and the M-level decoder outputs the image noise, and then the image noise output by the predictor is subtracted from the original noise image to obtain a preliminary denoised image;

[0092] The feature interaction module includes a weight generator and a convolutional network layer, which is used to weight the output features of each level of decoders in the two predictors;

[0093] The fusion module includes a convolutional network layer, which is used to fuse the preliminary denoised images output by the last-stage decoders of the two predictors to obtain a fused final denoised image.

[0094] The effect of the present invention will be further described below in conjunction with experiments.

[0095] 1. Experimental conditions:

[0096] The experiments of the present invention were carried out in a hardware environment of CPU Intel 7800X, memory 32GB, graphics card RTX 3090Ti, and a software environment of Windows 10 operating system and Python 3.8.16.

[0097] 2. Experimental content:

[0098] In this experiment, Gaussian white noise with σ between 0 and 60 is added to the NWPU-RESISC45 remote sensing dataset to evaluate the performance stability of the method under different noise intensities, and the PolyU and SIDD real denoising datasets are used to evaluate the denoising performance of the method. On each dataset, the detection effect of the method of the present invention is compared with the existing five mainstream methods BM3D, DnCNN, DUMRN, CBDNet and DGCL from the two indicators of peak signal-to-noise ratio PSNR and structural similarity SSIM.

[0099] Experiment 1: Gaussian white noise with σ of 5, 10, 20, 30, 40 and 50 is added to the NWPU-RESISC45 remote sensing dataset, and five mainstream methods and the method of the present invention are used for image denoising. The PSNR and SSIM of the denoised images obtained by each method and the standard denoised images are shown in Table 1:

[0100] Table 1 Comparison of the indicators of the NWPU-RESISC45 dataset of the present invention and the five existing mainstream methods

[0101]

[0102] The larger the peak signal-to-noise ratio PSNR and structural similarity SSIM in the table, the closer the denoised image is to the standard image of the dataset, and the better the image denoising effect.

[0103] It can be seen from Table 1 that the method proposed in the present invention is superior to other mainstream methods both in low-noise scenarios with small σ values ​​and in high-noise scenarios with large σ values.

[0104] The denoising effect comparison between the present invention and the five existing denoising methods in a high noise scene with σ=50 is shown in the figure below: Figure 6 As shown, 6(a) is a noisy image, 6(b) is a denoised image, 6(c) is a denoising effect diagram of the BM3D method, 6(d) is a denoising effect diagram of the DnCNN method, 6(e) is a denoising effect diagram of the DUMNP method, 6(f) is a denoising effect diagram of the CBDNet method, 6(g) is a denoising effect diagram of the DGCL method, and 6(h) is a denoising effect diagram of the method of the present invention.

[0105] from Figure 6 It can be seen that the performance of other comparison methods is attenuated to varying degrees in high-noise scenes and cannot complete the denoising task well. The method of the present invention can not only stably remove noise, but also restore image details as much as possible.

[0106] Experiment 2: Five mainstream methods and the proposed method are used to perform image denoising on the test set of the PolyU dataset. The PSNR and SSIM of the denoised images obtained by each method and the standard denoised images are shown in Table 2:

[0107] Table 2 Comparison of PolyU dataset indicators between the present invention and five existing mainstream methods

[0108]

[0109]

[0110] It can be seen from Table 2 that the index of the denoised image generated by the present invention is significantly higher than that of other comparison methods, and the image denoising effect is the best.

[0111] Experiment 3: The test set in the SIDD dataset is subjected to image denoising using five mainstream methods and the method of the present invention. The PSNR and SSIM of the denoised images obtained by each method and the standard denoised images are shown in Table 3:

[0112] Table 3 Comparison of SIDD data set indicators between the present invention and the five existing mainstream methods

[0113]

[0114] It can be seen from Table 3 that the index of the denoised image generated by the present invention is significantly higher than that of other comparison methods, and the image denoising effect is the best.

[0115] The sources of the above five mainstream methods are as follows:

[0116] BM3D: [Reference: Dabov K, Foi A, Katkovnik V, et al.Image Denoising bySparse 3-DTransform-Domain Collaborative Filtering[J].IEEE Transactions onImage Processing, 2007,16(8):p.2080-2095.].

[0117] DnCNN: [Reference: Zhang K, Zuo W, Chen Y, et al. Beyond a GaussianDenoiser: Residual Learning of Deep CNN for Image Denoising[J]. IEEE Transactions on Image Processing, 2016, 26(7): 3142-3155.].

[0118] CBDNet: [Reference: Guo S, Yan Z, Zhang K, et al. "Toward ConvolutionalBlind Denoising of Real Photographs," In Proceedings of the 2019IEEE / CVFConference on Computer Vision and Pattern Recognition (CVPR)., pp.1712–1722, Jun.2019.].

[0119] DUMRN: [Reference: Xu J, Yuan M, Yan DM, et al. Deep unfolding multi-scaleregularizer network for image denoising[J]. Computational Visual Media, 2023, 9(2): 335-350.].

[0120] DGCL: [Reference: Zhao S.; Lin S.; Cheng

[0121] The above experimental results prove the correctness and effectiveness of the method proposed in the present invention.

[0122] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Any modification, equivalent substitution, improvement, etc. made without departing from the principle of the present invention should be included in the protection scope of the present invention.

[0123] It should be noted that the step numbers in the specification and claims of the present invention are only for a clear description of the implementation scheme of the present invention to facilitate understanding, and the order of the numbers is not limited.

Claims

1. An image denoising method based on feature fusion complementary learning, characterized in that: The steps include: 1) Obtain a noisy data image set and a denoised data image set in pairs from a public data set, perform normalization preprocessing on them, and change the pixel value range of the image from [0, 255] to [-1, 1]; and divide the preprocessed data image set into a training set and a test set in a ratio of 8:2; 2) Constructing a feature interactive complementary learning denoising network: 2a) establishing two predictors each including an N-level encoder and an M-level decoder, for predicting the denoised image and the noise respectively; 2b) establishing a feature interaction module including a conditional weight generator and a convolutional network for weighted coupling of the outputs of each level of the decoder of the two predictors; 2c) establishing a fusion module including a convolutional layer, for fusing the denoised images predicted by the last-stage decoder outputs of the two predictors to obtain a fused denoised image; 2d) The outputs of the 1st to M-1th level decoders of the two predictors are connected to the input of the next level decoder after weighted coupling by the feature interaction module, and the Mth level decoders of the two predictors respectively output a first denoised image and noise, and the noise is subtracted from the original noise image to obtain a second denoised image, and then the first denoised image and the second denoised image are connected in parallel according to the channel dimension and connected to the input end of the fusion module to form a feature interaction complementary learning denoising network; 3) Design the loss function L of the feature interactive complementary learning denoising network as: L=L fin +0.5L n +0.5L rsi Among them, L fin is the loss function of the denoised image output by the fusion module, L rsi is the first denoising image loss function, L n is the loss function for the predictor output noise; 4) Input the training set image into the feature interaction complementary learning denoising network, use the back propagation method to calculate the gradient of the loss function, and use the optimizer to optimize the parameters of the two predictors and the feature interaction module and the fusion module respectively with the goal of predicting the denoised image and the goal of predicting the noise, until the loss function tends to be stable to obtain the trained feature interaction complementary learning denoising network, which includes the trained denoised image predictor and the noise predictor; 5) Input the noisy image in the test set into the trained feature interaction complementary learning denoising network to obtain the denoised image.

2. The method according to claim 1, characterized in that: In step 2a), two predictors are established, each including a multi-stage encoder and a multi-stage decoder. The implementation steps include the following: 2a1) The encoder is composed of a convolutional layer with a kernel size of 4×4 and a stride of 2, a batch normalization layer, and a LeakRelu activation function; 2a2) The decoder is composed of a deconvolution layer with a kernel size of 4×4 and a stride of 2, a batch normalization layer, a LeakRelu and a Tanh activation function; 2a3) Connect the convolutional layer and the batch normalization layer to form the first-level encoder, and connect the convolutional layer, batch normalization layer, and LeakRelu The activation functions are connected in sequence to form encoders at other levels; The deconvolution layer, batch normalization layer and Tanh activation function are connected in sequence to form the last level of decoder, and the deconvolution layer, batch normalization layer and LeakRelu activation function are connected in sequence to form other levels of decoders; each level output in the encoder is connected to the corresponding level in the decoder to obtain a predictor; 2a4) Repeat step 2a3) to obtain another predictor.

3. The method according to claim 1, characterized in that: Step 2b) Establish a feature interaction module including a conditional weight generator and a convolutional network, and the implementation steps include the following: 2b1) Cascade the average pooling layer, the linear layer, and the Sigmoid activation function to form a conditional weight generator CWG; 2b2) Construct a convolution layer Conv with a convolution kernel size of 1×1 and a stride of 1 1×1 ; 2b3) The initial feature PF1 of the first predictor and the PF1 after CWG processing and the PF1 after Conv 1×1 The processed initial feature PF2 of the second predictor is multiplied to obtain the intermediate feature MPF1 of the first predictor. The initial feature PF2 of the second predictor is multiplied by the PF2 processed by CWG and the intermediate feature MPF1 of the first predictor. 1×1 The processed initial feature PF1 of the first predictor is multiplied to obtain the intermediate feature MPF2 of the second predictor: 2b4) The initial feature PF1 of the first predictor minus MPF1 plus MPF2 obtains the first predictor coupling feature CPF1, and the initial feature PF2 of the second predictor minus MPF2 plus MPF1 obtains the second predictor coupling feature CPF2, thereby achieving weighted coupling of the two predictor features.

4. The method according to claim 1, characterized in that: Step 2c) forms the convolutional layer of the fusion module, with a convolution kernel size of 1×1 and a stride of 1.

5. The method according to claim 1, characterized in that: The loss function L of the fusion module output denoised image in step 3) fin And the first denoising image loss function L rsi They are respectively expressed as follows: L fin =L char +0.15L SSIM +0.2L edge L rsi =The char +0.15L' SSIM +0.2L' edge in is the pixel loss between the predicted result and the standard result, Pre f and gt i Represent the fused denoised image and the standard noise-free image in the training set, respectively, and ε is a constant of 10 -3 ; L SSIM =1-SSIM(pre f ,gt i ) is the SSIM loss of the predicted result and the standard result, and SSIM is the structural similarity between the predicted result and the standard result; is the edge loss between the predicted result and the standard result, Δ is the Laplace operator; is the pixel loss between the predicted result and the standard result, r and gt i represent the first denoised image and the standard noise-free image in the training set respectively; L' SSIM =1-SSIM(pre r ,gt i ) is the SSIM loss of the predicted result and the standard result, and SSIM is the structural similarity between the predicted result and the standard result; is the edge loss between the predicted result and the standard result.

6. The method according to claim 1, characterized in that: The loss function L of the predictor output noise in step 3) n It is expressed as follows: L n =L char-n +0.5L asymm in is the noise and standard result pixel loss, pre n represents the predicted noise, gt n is the standard noise, gt n It is obtained by subtracting the standard noise-free image from the noisy image in the training set, and ε is a constant of 10 -3 ; is the asymmetric loss of noise estimation, For all pre n -gt n <0 timepre n -gt n The factorial of , α is a constant set to 0.

3.

7. The method according to claim 1, characterized in that: In step 4), the gradient of the loss function is calculated by back propagation method, and the parameters of the two predictors, the feature interaction module and the fusion module are optimized by using an optimizer with the goal of predicting the denoised image and the goal of predicting the noise respectively; the implementation steps include the following: 4a) setting the feature interaction complementary learning denoising network to training mode; 4b) The noisy image in the training set is forward propagated through the feature interactive complementary learning denoising network, the first predictor is used as the denoised image predictor for training and calculates and outputs the first denoised image, the second predictor is used as the noise predictor for training and calculates and outputs the noise, the original noisy image is subtracted from the noise output by the second predictor to obtain the second denoised image, and the fusion module is used as the fusion of the first denoised image and the second denoised image for training and calculates and outputs the fused denoised image; 4c) Use the first denoised image, the fused denoised image and the standard noise-free image to calculate the loss function L rsi , L fin , use the noise output by the second predictor and the standard noise to calculate the loss function L n ; 4d) Calculate the loss function L = L according to the back propagation method fin +0.5L n +0.5L rsi The gradient of 4e) Use the Adam optimizer to update the model parameters according to the change of the loss function gradient; 4f) Repeat process 4b) to 4e) until the loss function reaches stability.

8. The method according to claim 1, characterized in that: In step 5), the noisy image in the test set is input into the trained feature interaction complementary learning denoising network to obtain a denoised image. The implementation steps include the following: 5a) Setting the feature interaction complementary learning denoising network to test mode; 5b) the denoised image predictor converts the noisy image in the test set into a first denoised image; 5c) The noise predictor converts the noisy image in the test set into noise, and then subtracts the noise from the original noisy image to obtain a second denoised image; 5d) The first denoised image and the second denoised image are connected in parallel in the channel dimension, and then converted into a final denoised image through a fusion module.

9. An image denoising system based on feature fusion complementary learning, characterized in that: include: Denoised image predictor, noise predictor, feature interaction module and fusion module; The denoised image predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract features of image content in a noisy image and input the features to a first-level decoder, and outputs of the first to M-1-th levels of the M-level decoders are weightedly coupled through a feature interaction module and then input to a next-level decoder, and the M-th-level decoder outputs a preliminary denoised image; The noise predictor comprises an N-level encoder and an M-level decoder, wherein the N-level encoder is used to extract the noise features in the noise image and input the features to the first-level decoder, and the outputs of the first to the M-1-level decoders are weighted coupled through a feature interaction module and then input to the next-level decoder, and the M-level decoder outputs the image noise, and then the image noise output by the predictor is subtracted from the original noise image to obtain a preliminary denoised image; The feature interaction module includes a weight generator and a convolutional network layer, which is used to weight the output features of each level of decoders in the two predictors; The fusion module includes a convolutional network layer, which is used to fuse the preliminary denoised images output by the last-stage decoders of the two predictors to obtain a fused final denoised image.

Images