A method and apparatus for denoising real images

Abstract

This invention discloses a method and apparatus for denoising real images, belonging to the field of image processing technology. The method includes training a denoising module using training samples as input to obtain a trained denoising module; acquiring a real image to be denoised; inputting the features of the real image into the trained denoising module; and having the blind spot branch of the trained denoising module denoise the real image, outputting a denoised image. The denoising module includes a parallel non-blind spot branch and a blind spot branch. The non-blind spot branch is a gradient-free non-blind spot branch, outputting a first denoising result; the blind spot branch generates mask features, which are denoised through convolution operations to obtain denoised features, serving as a second denoising result for the blind spot branch; and the first and second denoising results are fused as the output of the denoising module. This invention improves image detail recovery capabilities and achieves higher denoising accuracy and image quality.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and apparatus for denoising real images. Background Technology

[0002] In the field of image processing, image denoising is a crucial step in improving image quality and enhancing image details. Traditional denoising methods rely on the statistical properties of images or prior knowledge. In recent years, deep learning-based denoising methods have been widely used, especially Blind-Spot Networks (BSNs), which demonstrate certain denoising capabilities due to their ability to perform self-supervised learning without clean labels. However, BSNs suffer from two main drawbacks: first, when dealing with truly noisy images, the noise often exhibits pixel-level correlation, and BSNs, based on the assumption of independent noise, perform poorly in denoising; second, during the denoising process, the masking operation of BSNs may lead to the loss of key image information, particularly in recovering high-frequency information such as edges and textures. Therefore, there is an urgent need for a new image denoising method that can address the problem of truly correlated noise and avoid information loss.

[0003] Blind spot networks are a variant of convolutional neural networks and are a self-supervised learning-based image denoising method. When applied to image denoising tasks with real-world noise, blind spot networks present the following two problems:

[0004] (1) In the process of image denoising, blind spot networks will block out each neighborhood region x in the input image. RF(i) center pixel x i And x i The pixels contain the corresponding pixels in the output image. The loss of valuable image information reduces the upper limit of denoising, thus affecting the performance of denoising algorithms.

[0005] (2) Blind spot networks predict information about the center pixel by using decentralized neighborhood regions. When noise signals in an image exhibit spatial correlation, meaning that noise signals from neighboring pixels are correlated, the information about the center pixel predicted by the blind spot network will inevitably introduce noise signals, thus failing to effectively remove noise. However, noisy images in the real world often exhibit spatial correlation, which does not conform to the blind spot network's assumption of independent and zero-mean noise distribution. Therefore, blind spot networks often perform poorly in real-world noise removal tasks. Summary of the Invention

[0006] The present invention provides a method and apparatus for denoising real images, which is used to solve the above-mentioned problems or at least partially solve the above-mentioned problems.

[0007] In a first aspect, a method for denoising real images is disclosed, the method comprising:

[0008] Step S1: Obtain training samples and use them as input to train the denoising module to obtain the trained denoising module.

[0009] Step S2: Obtain the real image to be denoised, input the real image into the trained denoising module, and the blind point branch of the trained denoising module performs denoising processing on the real image and outputs the denoised image.

[0010] The noise reduction module includes parallel non-blind spot branches and blind spot branches;

[0011] The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch.

[0012] The downsampling module includes a convolutional layer with a kernel size of 1×1 and a pooling layer connected in sequence. The convolutional layer with a kernel size of 1×1 performs dimensionality upsampling in the channel direction on the training samples, and the pooling layer performs max pooling to achieve downsampling in the spatial direction.

[0013] The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch.

[0014] The first denoising result and the second denoising result are combined and used as the output of the denoising module.

[0015] Preferably, the formula for fusing the first denoising result and the second denoising result is:

[0016]

[0017] Where a is a hyperparameter, x b As the first denoising result, x unb For the second denoising result, when the number of training epochs ≤ 8, a = 2.0; when the number of training epochs > 8,

[0018] Preferably, the loss function of the non-blind spot branch during training is:

[0019] L reg =‖x b -y‖1

[0020] Among them, L reg y represents the loss function value of the non-blind spot branch, y represents the input noisy image, and ||·||1 represents the L1 loss function.

[0021] Preferably, the loss function of the blind spot branch during training is:

[0022]

[0023] Among them, L rev Let ||·||2 be the loss function value of the blind point branch, and let ||·||2 be the mean squared error loss function.

[0024] Preferably, the total loss function L is:

[0025]

[0026] Preferably, in step S2, the real image to be denoised is obtained, the features of the real image are input into the trained denoising module, and the blind spot branch of the trained denoising module performs denoising processing on the real image, wherein:

[0027] The features of the real image are input into the dense sampling block blind spot convolution module. The blind spot convolution kernel is used to perform a masking operation on the input. During the masking operation, the masked pixels are recovered based on the pixel information of the neighborhood of the masked pixels. Mask features are generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask features through convolution operations to obtain denoised features, which serve as the second denoising result of the blind spot branch.

[0028] The second denoising result is used as the denoised image.

[0029] Secondly, an apparatus for denoising real images is disclosed, the apparatus comprising:

[0030] Training module: Configured to acquire training samples, use the training samples as input to train the denoising module, and obtain the trained denoising module;

[0031] Real Image Denoising Module: Configured to acquire a real image to be denoised, input the real image into the trained denoising module, and perform denoising processing on the real image by the blind point branch of the trained denoising module, and output the denoised image;

[0032] The noise reduction module includes parallel non-blind spot branches and blind spot branches;

[0033] The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch.

[0034] The downsampling module includes a convolutional layer with a kernel size of 1×1 and a pooling layer connected in sequence. The convolutional layer with a kernel size of 1×1 performs dimensionality upsampling in the channel direction on the training samples, and the pooling layer performs max pooling to achieve downsampling in the spatial direction.

[0035] The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch.

[0036] The first denoising result and the second denoising result are combined and used as the output of the denoising module.

[0037] Thirdly, an electronic device is disclosed, the electronic device comprising:

[0038] At least one processor; and

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0041] Fourthly, a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method described above is disclosed.

[0042] The present invention has the following technical effects:

[0043] This invention designs a blind spot network framework to remove real-world correlated noise, employing a combination of blind spot and non-blind spot branches. The blind spot branch utilizes a densely sampled block blind spot convolutional module (DSPMC, disclosed in the LG-BPN method) and a U-Net network to break the spatial correlation of noise through locally decentralized convolution operations, thereby solving the problem of poor performance of traditional blind spot networks when processing real-world noisy images.

[0044] This invention addresses the problem of key information loss caused by masking operations in blind spot networks by introducing a gradient-free non-blind spot training branch and a re-visibility loss function. The gradient-free non-blind spot branch reintroduces pixel or feature information masked by the blind spot network and weightedly fuses the denoising results from the blind spot branch with those from the non-blind spot branch, ensuring that high-frequency information, especially texture and edges, is not lost during denoising, thereby improving the image denoising effect.

[0045] This invention also maintains the stability of the training process by keeping the original loss function of the blind spot network as a regularization term while introducing a new loss function. Through this technique, the denoising network further improves its image detail recovery capability without increasing computational complexity, achieving higher denoising accuracy and image quality. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the method for denoising real images according to the present invention.

[0047] Figure 2 This is a schematic diagram of the training process architecture of the present invention;

[0048] Figure 3 This is a schematic diagram of the reasoning process architecture of the present invention;

[0049] Figure 4This is a schematic diagram of the blind spot convolution module of the present invention;

[0050] Figure 5 This is a schematic diagram of the downsampling module of the present invention;

[0051] Figure 6 This is a schematic diagram of the device for denoising real images according to the present invention. Detailed Implementation

[0052] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] like Figures 1-3 As shown, the present invention provides a method for denoising real images, the method comprising:

[0054] Step S1: Obtain training samples and use them as input to train the denoising module to obtain the trained denoising module.

[0055] Step S2: Obtain the real image to be denoised, input the real image into the trained denoising module, and the blind point branch of the trained denoising module performs denoising processing on the real image and outputs the denoised image.

[0056] The noise reduction module includes parallel non-blind spot branches and blind spot branches;

[0057] The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch.

[0058] The downsampling module includes a convolutional layer with a kernel size of 1×1 and a pooling layer connected in sequence. The convolutional layer with a kernel size of 1×1 performs dimensionality upsampling in the channel direction on the training samples, and the pooling layer performs max pooling to achieve downsampling in the spatial direction.

[0059] The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch.

[0060] The first denoising result and the second denoising result are combined and used as the output of the denoising module.

[0061] In this invention, convolutional neural network computations typically involve gradients and automatically update network parameters. However, in the code implementation of this invention, the values ​​of parameters related to gradients are changed from the default True to False, thus rendering them gradient-free.

[0062] Furthermore, the formula for fusing the first denoising result and the second denoising result is as follows:

[0063]

[0064] Where a is a hyperparameter, x b As the first denoising result, x unb For the second denoising result, when the number of training epochs ≤ 8, a = 2.0; when the number of training epochs > 8, Training rounds: The number of times the entire training dataset is traversed during the training process.

[0065] In this invention, 'a' is a hyperparameter controlling the fusion ratio of two images. During training, the value of 'a' gradually increases with the number of epochs. When the denoising network is first trained, the convolutional denoising network module does not yet have denoising capabilities, thus relying more on the blind spot branch to train the network to have initial denoising ability. However, as the training epochs increase, the convolutional denoising network module gradually develops denoising capabilities. Because the input information of the non-blind spot branch does not lack key information, the output denoising effect is better. Therefore, the network increasingly relies on the training process of the non-blind spot branch.

[0066] In this invention, the convolutional denoising network module employs the U-Net network. As the primary framework for image denoising, the U-Net network aims to effectively remove noise while preserving image details and global information through its symmetrical encoder-decoder structure and multi-scale feature extraction and restoration capabilities. U-Net's unique skip connections enable the network to retain high-resolution image details during denoising, resulting in a more natural denoising effect and making it suitable for various complex noise environments.

[0067] The U-Net network consists of two parts: an encoder that progressively downsamples the image through multiple convolutions and max pooling operations to extract multi-scale features; and a decoder that progressively upsamples the features through transposed convolutions to restore the image's original resolution. Simultaneously, skip connections fuse the features from the encoder and decoder, ensuring sufficient detail is preserved during the restoration process. Finally, a denoised image is output through a 1×1 convolution.

[0068] The advantage of the U-Net framework lies in its powerful multi-scale feature extraction capabilities and skip connection mechanism, which enables it to not only effectively remove noise but also preserve image details and global structure. Especially when processing images with complex noise distributions, U-Net can accurately capture and effectively remove noise features while preserving the image's texture and edge information, resulting in a clearer and more natural denoised image.

[0069] The upsampling module primarily restores the denoised image to its original resolution through transposed convolution. Transposed convolution (also known as deconvolution) expands the image in spatial dimensions, thereby inversely restoring the resolution reduction caused by the convolution operation. The module's function is to progressively restore the image's spatial resolution while preserving or enhancing feature information, ensuring that important details are not lost during the restoration process, and providing a high-quality denoising result for the final output.

[0070] In this invention, noisy images or feature maps are masked using a dense sampling block blind convolution module before being input into a convolutional denoising network module for denoising. The function of the dense sampling block blind convolution module is to mask part of the pixel information in the input image or feature map, and then recover the masked pixels from the pixel information in the decentralized neighborhood. This process is mainly to avoid convergence to the identity mapping during training. Simultaneously, the dense sampling block blind convolution module can also break the spatial correlation of noise; different blind convolution modules have different functions. This invention uses the DSPMC module (disclosed in the LG-BPN method), which uses a convolution kernel with a diamond-shaped region centered at the center point as the blind zone to perform a convolution operation on the input feature map, thereby achieving masking of the entire image's pixel information. The dense sampling block blind convolution module is as follows: Figure 4 As shown.

[0071] The dense sampling block blind convolution module of this invention, through a specific convolution structure, allows the network to ignore local noise points during processing, thereby reducing the impact of noise on the model. Its main function is to further break down noise correlation, enabling the network to extract cleaner image features better without relying on noise points. This blind convolution strategy effectively enhances denoising performance and is particularly suitable for scenarios with high noise correlation.

[0072] This invention adds a gradient-free non-blind spot branch, which, together with the blind spot branch, constitutes the non-blind spot training strategy. In the gradient-free non-blind spot branch, the image does not need to pass through the dense sampling block blind spot convolution module; instead, it is directly input into the convolutional denoising network module after downsampling to obtain the denoised image. Since the image or features in this branch do not need to pass through the dense sampling block blind spot convolution module, the image information in this branch is not lost due to occlusion. The denoised image x obtained from this branch... ub Relatively complete information is preserved. At the same time, this branch is set to a gradient-free process, meaning that when the image passes through the convolutional denoising network module, it does not affect the network parameters, thus avoiding the possibility of the network converging to the identity mapping.

[0073] like Figure 5 As shown, the convolutional layer of the downsampling module increases the number of channels without changing the spatial dimension of the image, thereby enhancing the feature representation capability. Max pooling reduces the spatial resolution of the image by extracting local maxima, while preserving key features.

[0074] Furthermore, the loss function of the non-blind spot branch during the training process is:

[0075] L reg =‖x b -y‖1

[0076] Among them, L reg y represents the loss function value of the non-blind spot branch, and y represents the input noisy image; ||·||1 represents the L1 loss function (absolute loss function).

[0077] The loss function of the blind spot branch during training is:

[0078]

[0079] Among them, L rev Let ||·||2 be the loss function value for the blind spot branch, and let ||·||2 be the mean squared error (MSE) loss function.

[0080] The total loss function L is:

[0081]

[0082] Further, in step S2, the real image to be denoised is obtained, and the features of the real image are input into the trained denoising module. The blind spot branch of the trained denoising module performs denoising processing on the real image, wherein:

[0083] The features of the real image are input into the dense sampling block blind spot convolution module. The blind spot convolution kernel is used to perform a masking operation on the input. During the masking operation, the masked pixels are recovered based on the pixel information of the neighborhood of the masked pixels. Mask features are generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask features through convolution operations to obtain denoised features, which serve as the second denoising result of the blind spot branch.

[0084] The second denoising result is used as the denoised image.

[0085] In this invention, during the training process, noisy images are passed through blind spot branch networks and non-blind spot branch networks respectively to obtain denoised images x. b , and image x unb Finally, the two images are weighted and averaged to obtain the denoised image x. During inference, the noisy image is directly denoised through the trained blind spot branch network, resulting in the denoised image x.

[0086] The non-blind spot branch in the training process proposes a training strategy to address the problem of key information loss in the denoising process of blind spot networks. The blind spot mechanism is a method proposed to address the shortcomings of blind spot networks in denoising tasks of real noisy images.

[0087] This invention can solve the problem of key information loss during the prediction process of blind spot networks, and overcome the problem of blind spot networks failing in the task of denoising real noisy images.

[0088] This invention designs a network framework to remove true correlation noise. Pixel information in an image generally exhibits spatial correlation, meaning that the information of the central pixel can be predicted using pixel information within its neighborhood. The blind spot network utilizes this characteristic, using a decentralized neighborhood interval x... RF(i)This process predicts information about the center pixel. When the noise signal in the image is spatially independent and has zero mean, this prediction process can remove noise information from the image and generate ground truth pixels. However, noise in real-world noisy images often exhibits pixel-level correlation, which does not conform to the assumptions of blind spot networks. Therefore, blind spot networks perform poorly in denoising tasks for real-world noisy images.

[0089] This invention designs a blind spot network framework for removing real-world correlation noise. The framework consists of blind spot branches and non-blind spot branches. The blind spot branches are primarily used to break down noise correlations and obtain a preliminary denoised image. The non-blind spot branches are mainly used to address the degradation in denoising performance caused by the loss of key information in the blind spot network.

[0090] The non-blind spot branch of this invention uses revisible loss and gradient-free non-blind spot training branch to solve the problem of degraded denoising performance caused by the loss of key information in blind spot networks.

[0091] Blind spot networks (BSNs) are a self-supervised learning method that can be trained on noisy images without corresponding clean images. Since the input and target of a BSN are the same noisy image, to avoid the network learning identity mappings, BSNs introduce a masking mechanism, similar to traditional convolutional neural networks. This masking operation masks some pixels in the input image or features, allowing the predicted pixels to learn from their neighborhood intervals. However, masking the input image or features in BSNs leads to the loss of crucial information, lowering the denoising ceiling and causing the loss of high-frequency information such as texture and edges during the denoising process.

[0092] Building upon blind spot denoising networks, this invention proposes a gradient-free, non-blind spot training branch. This method, avoiding non-identity mappings, reintroduces pixel or feature information obscured by the blind spot network into the network. The denoised image obtained from this branch is then weighted and fused with the denoised image obtained from the blind spot network branch to obtain a new denoised image. A re-visibility loss function is introduced, which calculates the loss between the newly obtained denoised image and the noisy image, thereby guiding the update of network parameters. Furthermore, to ensure training stability, the original blind spot network loss is retained as a regularization term.

[0093] The blind spot branch of this invention combines the blind spot mechanism and the U-Net network to solve the problem that blind spot networks cannot effectively denoise real noise.

[0094] Pixel information in an image generally exhibits spatial correlation, meaning that the information of the central pixel can be predicted using pixel information within its neighborhood. Blind spot networks leverage this characteristic, using a decentralized neighborhood interval x... RF(i)This process predicts information about the center pixel. When the noise signal in the image is spatially independent and has zero mean, this prediction process can remove noise information from the image and generate ground truth pixels. However, noise in real-world noisy images often exhibits pixel-level correlation, which does not conform to the assumptions of blind spot networks. Therefore, blind spot networks perform poorly in denoising tasks for real-world noisy images.

[0095] This invention designs a dense sampling block blind convolution module (DSPMC) and uses this module as a preprocessing module. It is then combined with the U-Net network to form a blind network branch that can remove correlated noise.

[0096] like Figure 6 As shown, the present invention provides an apparatus for denoising real images, the apparatus comprising:

[0097] Training module: Configured to acquire training samples, use the training samples as input to train the denoising module, and obtain the trained denoising module;

[0098] Real Image Denoising Module: Configured to acquire a real image to be denoised, input the real image into the trained denoising module, and perform denoising processing on the real image by the blind point branch of the trained denoising module, and output the denoised image;

[0099] The noise reduction module includes parallel non-blind spot branches and blind spot branches;

[0100] The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch.

[0101] The downsampling module includes a convolutional layer with a kernel size of 1×1 and a pooling layer connected in sequence. The convolutional layer with a kernel size of 1×1 performs dimensionality upsampling in the channel direction on the training samples, and the pooling layer performs max pooling to achieve downsampling in the spatial direction.

[0102] The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch.

[0103] The first denoising result and the second denoising result are combined and used as the output of the denoising module.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein, and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for denoising real images, characterized in that, The method includes the following steps: Step S1: Obtain training samples and use them as input to train the denoising module to obtain the trained denoising module. Step S2: Obtain the real image to be denoised, input the real image into the trained denoising module, and have the trained denoising module perform denoising processing on the real image and output the denoised image. The noise reduction module includes parallel non-blind spot branches and blind spot branches; The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch. The downsampling module includes sequentially connected convolutional kernels with a size of 1. A convolutional layer and a pooling layer of size 1, wherein the convolutional kernel size is 1. The convolutional layer of 1 performs dimensionality upsampling in the channel direction on the training samples, and then the pooling layer performs max pooling to achieve downsampling in the spatial direction; The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch. The first denoising result and the second denoising result are combined and used as the output of the denoising module.

2. The method as described in claim 1, characterized in that, The formula for fusing the first denoising result and the second denoising result is as follows: in, For hyperparameters, This is the first denoising result. For the second denoising result, in the training rounds hour, Number of training rounds hour, 。 3. The method as described in claim 2, characterized in that, The loss function of the non-blind spot branch during training is: in, The loss function value for the non-blind spot branch. The input is a noisy image; This is the L1 loss function.

4. The method as described in claim 3, characterized in that, The loss function of the blind spot branch during training is: in, The loss function value for the blind branch. Let be the mean squared error loss function.

5. The method as described in claim 4, characterized in that, The total loss function L is: .

6. An apparatus for denoising real images, characterized in that, The device includes: Training module: Configured to acquire training samples, use the training samples as input to train the denoising module, and obtain the trained denoising module; Real image denoising module: configured to acquire a real image to be denoised, input the real image into a trained denoising module, and have the trained denoising module perform denoising processing on the real image and output the denoised image; The noise reduction module includes parallel non-blind spot branches and blind spot branches; The non-blind spot branch is a gradient-free non-blind spot branch, comprising a downsampling module, a convolutional denoising network module, and an upsampling module connected in sequence. The non-blind spot branch receives input, and the downsampling module performs spatial downsampling on the input. The convolutional denoising network module is a U-Net network, used to denoise the downsampled result to obtain denoised features. The upsampling module restores the denoised features to the original resolution through a transposed convolution operation, outputting the first denoising result of the non-blind spot branch. The downsampling module includes sequentially connected convolutional kernels with a size of 1. A convolutional layer and a pooling layer of size 1, wherein the convolutional kernel size is 1. The convolutional layer of 1 performs dimensionality upsampling in the channel direction on the training samples, and then the pooling layer performs max pooling to achieve downsampling in the spatial direction; The blind spot branch includes a dense sampling block blind spot convolution module and a convolutional denoising network module connected in sequence. The dense sampling block blind spot convolution module receives input and performs a masking operation on the input using a blind spot convolution kernel. During the masking operation, the masked pixels are recovered based on the pixel information in the neighborhood of the masked pixels, and a mask feature is generated based on the recovered pixels and the unmasked pixels. The blind spot convolution kernel is obtained by multiplying the convolution kernel and the noise mask matrix element by element. The noise mask matrix is ​​a randomly generated matrix composed of 0s and 1s with the same dimension as the convolution kernel. Elements with a value of 0 in the noise mask matrix represent blind spots. The convolutional denoising network module denoises the mask feature through a convolution operation to obtain the denoised feature, which serves as the second denoising result of the blind spot branch. The first denoising result and the second denoising result are combined and used as the output of the denoising module.

7. A computer-readable storage medium storing a plurality of instructions; the plurality of instructions being loaded by a processor and executing the method as claimed in any one of claims 1-5.

8. An electronic device, characterized in that, The electronic device includes: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded by the processor and executed as described in any one of claims 1-5.

Images