CBDNet denoising network optimization method and system for enhancing edge features

Abstract

The application discloses a kind of enhanced edge feature's CBDNet denoising network optimization method and system, the method includes: obtaining real image dataset and synthetic image dataset;Original image is alternately input into noise estimation network, and the noise estimation graph is output;Original image and noise estimation graph are spliced;After image is input into non-blind denoising network, the image after denoising is output, and the non-blind denoising network adopts U-Net architecture, including a plurality of convolution layers, two bilinear interpolation up-sampling layers and two IndexNet modules.The enhanced edge feature's CBDNet denoising network optimization method and system of the application improve the effect of subsequent image processing or detection by reducing the loss of image edge information during denoising, retain more image texture features using the non-blind denoising network of CBDNet, while reducing the impact of excessive denoising caused by noise estimation network overestimating noise level.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for optimizing a CBDNet denoising network to enhance edge features. Background Technology

[0002] Image denoising is an important technique in digital image processing, aiming to reduce various noises introduced during the imaging process or added artificially. Denoising algorithms are often used as an important part of the image preprocessing stage, or to enhance image sharpness. Today, denoising techniques strive to eliminate as much noise as possible while ensuring that the information in the image is not damaged by excessive denoising.

[0003] Current image denoising algorithms generally achieve good denoising results. For example, CBDNet uses a serial branching structure, first estimating the noise level through a noise estimation network, and then performing denoising operations based on the noise level through a non-blind denoising network. However, although CBDNet's denoising effect is good, the pooling process may cause loss of detail, and U-Net suffers from the semantic gap problem, resulting in blurred image edges.

[0004] Therefore, there is an urgent need for an optimization method and system for CBDNet denoising network that enhances edge features. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for optimizing CBDNet denoising networks to enhance edge features, thereby solving the problems in the prior art and optimizing the non-blind denoising network model in CBDNet to preserve edge details as much as possible.

[0006] This invention provides a method for optimizing a CBDNet denoising network to enhance edge features, comprising:

[0007] Obtain real image datasets and synthetic image datasets respectively;

[0008] The original images selected from the real image dataset and the synthetic image dataset are alternately input into the noise estimation network, and the output is a noise estimation map;

[0009] The original image and the noise estimation map are stitched together;

[0010] The stitched original image and the noise estimation map are input into a non-blind denoising network, which outputs a denoised image. The non-blind denoising network adopts the U-Net architecture, which includes several convolutional layers, two bilinear interpolation upsampling layers, and two IndexNet modules.

[0011] The CBDNet denoising network optimization method for enhancing edge features as described above, preferably, involves obtaining the real image dataset and the synthetic image dataset separately, specifically including:

[0012] The real image dataset was obtained by capturing images using real imaging equipment;

[0013] A noiseless image dataset is obtained, and Poisson-Gaussian noise is added to each noiseless image in the noiseless image dataset. Then, the noiseless images with added Poisson-Gaussian noise are subjected to de-mosaic and gamma correction processes to obtain the synthetic image dataset.

[0014] The CBDNet denoising network optimization method for enhancing edge features as described above, preferably, involves alternately inputting original images selected from the real image dataset and the synthetic image dataset into the noise estimation network to output a noise estimation map, specifically including:

[0015] The original image input to the noise estimation network is the same size as the noise estimation map output by the noise estimation network. The noise estimation network is a fully convolutional network and has no pooling layers or BN layers.

[0016] In the CBDNet denoising network optimization method for enhancing edge features as described above, preferably, the resolution of both the original image input to the noise estimation network and the noise estimation map output by the noise estimation network is H×W×3; the noise estimation network includes 5 standard convolutional layers with ReLU activation, the first 4 convolutional layers contain 32 convolutional kernels, the last convolutional layer contains 3 convolutional kernels, and the kernel size of all convolutional layers is 3×3.

[0017] The CBDNet denoising network optimization method for enhancing edge features as described above, preferably, involves concatenating the original image and the noise estimation map, specifically including:

[0018] The concatenate function is used to concatenate the original image and the noise estimation image into a vector with a resolution of H×W×6.

[0019] The CBDNet denoising network optimization method for enhancing edge features as described above preferably includes a non-blind denoising network comprising 5 convolutional blocks, 1 convolutional layer, 2 average pooling layers, 2 bilinear interpolation upsampling layers, and 2 IndexNet modules. Specifically, the first convolutional block consists of 2 convolutional layers, the second convolutional block consists of 3 convolutional layers, the third convolutional block consists of 6 convolutional layers, the fourth convolutional block consists of 3 convolutional layers, and the fifth convolutional block consists of 2 convolutional layers. The first IndexNet module is added after the first convolutional block to guide the first average pooling layer and the second bilinear interpolation upsampling layer, and the second IndexNet module is added after the second convolutional block to guide the second average pooling layer and the first bilinear interpolation upsampling layer.

[0020] The CBDNet denoising network optimization method for enhancing edge features as described above preferably includes each IndexNet module comprising an IndexBlock, a Sigmoid activation function, and a Softmax activation function. The IndexBlock, as a learning module, consists of one convolutional layer and one pixel reconstruction layer.

[0021] The step of inputting the stitched original image and the noise estimation map into a non-blind denoising network and outputting the denoised image specifically includes:

[0022] The stitched original image and the noise estimation map are fed into IndexBlock. First, the image is halved in length and width by passing through a convolutional layer with a kernel stride of 2 and an output channel of 4. Then, the four feature maps are merged into one feature map by passing through a pixel reconstruction layer, and its size is restored to the original size.

[0023] The image output by IndexBlock is used to guide the upsampling process after passing through the Sigmoid activation function. The image output by IndexBlock is multiplied element-wise with the image before upsampling, and then the upsampling operation is performed.

[0024] The image output by IndexBlock, after passing through the Sigmoid function and then the Softmax activation function, is used to guide the downsampling process. The image output by IndexBlock is first multiplied element-wise with the image before downsampling, then downsampled, and finally each element in the image is multiplied by 4 after downsampling.

[0025] The CBDNet denoising network optimization method for enhancing edge features as described above, preferably, involves inputting the stitched original image and the noise estimation map into a non-blind denoising network to output the denoised image, specifically including:

[0026] The original image and the noise estimation map, which are stitched together with a resolution of H×W×6, are input into a non-blind denoising network. After two IndexNet-guided average pooling layers, the pixel size gradually decreases. After two IndexNet-guided bilinear interpolation upsampling layers, the pixel size is restored to its original size, and an image with a resolution of H×W×3 is output.

[0027] The CBDNet denoising network optimization method for enhancing edge features as described above preferably involves training the noise estimation network and the non-blind denoising network based on the total loss function, and updating the parameters using stochastic gradient descent. Both networks are trained together. The total loss function is a weighted sum of the asymmetric loss function and total variation loss function at the output position of the noise estimation network, and the pixel-level mean square error loss function at the output position of the non-blind denoising network.

[0028] The asymmetric loss function L at the output position of the noise estimation network is calculated using the following formula. asymm ,

[0029]

[0030] Among them, y i This represents the image input to the noise estimation network. σ(y) represents the estimated noise intensity. i ) represents the noise intensity of the Ground-Truth noise-free image; for I e When e is true, I e =1, otherwise I e =0; By assigning a value of α < 0.5, a greater penalty is imposed for underestimating the noise intensity;

[0031] The total variation loss function L at the output location of the noise estimation network is calculated using the following formula. TV ,

[0032]

[0033] Among them, graf h Graf represents the gradient in the horizontal direction. v || represents the gradient in the vertical direction; ||2 represents the L2 norm;

[0034] The pixel-level mean square error loss function L at the output position of the non-blind denoising network is calculated using the following formula. rec ,

[0035]

[0036] in, This represents the denoising result, where x represents the Ground-Truth noise-free image.

[0037] The total loss function is calculated using the following formula.

[0038] L = L rec +λ asymm L asymm +λ TV L TV (4)

[0039] Where, λ asymm Represents the asymmetric loss function L asymm The weight, λ TV L represents the total variation loss function TV The weights, and L is not used when training with real data. asymm Considering only L TV and L rec At this time L asymm =0.

[0040] This invention also provides a CBDNet denoising network optimization system for enhancing edge features, comprising:

[0041] The dataset acquisition module is used to acquire real image datasets and synthetic image datasets respectively;

[0042] The noise estimation module is used to alternately input original images selected from the real image dataset and the synthetic image dataset into the noise estimation network and output a noise estimation map;

[0043] An image stitching module is used to stitch the original image and the noise estimation map together;

[0044] The non-blind denoising module is used to input the stitched original image and the noise estimation map into the non-blind denoising network and output the denoised image. The non-blind denoising network adopts the U-Net architecture, which includes several convolutional layers, two bilinear interpolation upsampling layers and two IndexNet modules.

[0045] This invention provides a CBDNet denoising network optimization method and system to enhance edge features. It incorporates IndexNet into a non-blind denoising network, replaces the two transposed convolutional layers with bilinear interpolation upsampling layers, and guides the sampling process by element-wise multiplication of the two weight maps output by IndexNet with the image before sampling. This solves the problem of edge blurring caused by information loss during downsampling in non-blind denoising networks. Combining U-Net and IndexNet retains U-Net's ability to acquire multi-level features while enhancing edge detail features through the IndexNet module. It improves the effectiveness of subsequent image processing or detection by reducing the loss of image edge information during denoising. The non-blind denoising network approach of CBDNet preserves more image texture features while reducing the impact of over-denoising caused by noise estimation networks overestimating noise levels. Attached Figure Description

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings, wherein:

[0047] Figure 1 A flowchart illustrating an embodiment of the CBDNet denoising network optimization method for enhancing edge features provided by the present invention;

[0048] Figure 2 The architecture diagram of the noise estimation network in the embodiment of the CBDNet denoising network optimization method for enhancing edge features provided by the present invention;

[0049] Figure 3 This is an architecture diagram of a non-blind denoising network in an embodiment of the CBDNet denoising network optimization method for enhancing edge features provided by the present invention.

[0050] Figure 4 This is a structural block diagram of an embodiment of the CBDNet denoising network optimization system for enhancing edge features provided by the present invention. Detailed Implementation

[0051] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the present disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that the present disclosure will be thorough and complete, and will fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless specifically stated otherwise, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0052] The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Terms such as “including” or “contains” mean that the element preceding the term encompasses the element listed after it, and do not exclude the possibility of encompassing other elements as well. Terms such as “above” and “below” are used only to indicate relative positional relationships; when the absolute position of the described object changes, this relative positional relationship may also change accordingly.

[0053] In this disclosure, when a specific component is described as being located between a first component and a second component, an intermediary component may or may not be present between the specific component and the first or second component. When a specific component is described as connecting to other components, the specific component may be directly connected to the other components without having an intermediary component, or it may not be directly connected to the other components but may have an intermediary component.

[0054] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary, such as a dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0055] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0056] Due to factors such as imaging principles, equipment defects, and environmental conditions, noise is inevitably introduced into images during acquisition. This noise can damage some image information, reduce image quality, and potentially affect the effectiveness of subsequent tasks. On one hand, noise can blur images and impair human judgment, especially in fields like medical image analysis, where even a small loss of detail can have serious consequences. On the other hand, noise in images can also affect the accuracy and stability of subsequent analyses such as image processing, target recognition, and image detection. Therefore, reducing noise, improving the signal-to-noise ratio, and enhancing detailed features are of great significance in the field of digital imaging.

[0057] Image denoising remains a challenging topic. Since noise, edges, and textures are all high-frequency regions, computers struggle to distinguish them. This can lead to the loss of some texture information in the denoised image, resulting in blurred edges. Therefore, efficiently removing noise from images while maintaining the integrity of image information during the denoising process remains a current research trend.

[0058] To address this issue, filter-based denoising methods and deep learning-based denoising methods have been proposed.

[0059] Among filter-based denoising methods, traditional methods use a single filter to process the image. While simple and easy to use, this often results in some information loss in the original image. For example, mean filters can blur the image and cause loss of edge details; high-pass filters can blur sharp areas. The BM3D algorithm, proposed in recent years, is a block-matching-based filtering algorithm. By matching with adjacent image blocks, several similar blocks are integrated into a three-dimensional matrix, filtered in three-dimensional space, and then the result is inversely transformed and fused into a two-dimensional image to form the denoised image. This algorithm achieves significant denoising results and the highest peak signal-to-noise ratio, but its time complexity is relatively high.

[0060] Among deep learning-based denoising methods, DnCNN is one of the most classic denoising network models. This model uses only a linear combination of convolutional layers, batch normalization layers, and ReLU activation functions, and employs residual learning. Although the structure is simple, it has good denoising performance, effectively removing uniform Gaussian noise, and even surpassing BM3D in PSNR.

[0061] FFDNet is an improvement on DnCNN, enhancing the model's ability to handle complex noise by adding a noise estimation map as input. It adds a downsampling layer before the model and an upsampling layer after it. The image is first downsampled to obtain several sub-images, which are then fed into the model along with the noise estimation map. This outputs several noise estimation sub-images, which are then upsampled to generate the final denoised image. The idea of ​​using a noise estimation map as input in FFDNet also provides a roadmap for future CBDNet implementations.

[0062] CBDNet follows the approach of FFDNet in estimating noise levels from input noise maps, designing a sub-network to estimate noise levels. Therefore, CBDNet comprises two networks: a noise estimation network and a non-blind denoising network. The former uses a fully convolutional network structure to estimate noise levels; the latter uses a U-Net structure to adaptively perform denoising based on the noise level. Since non-blind denoising algorithms are more sensitive to underestimating the noise level, CBDNet employs an asymmetric learning method, imposing a greater penalty when the noise estimation network underestimates the noise level. Furthermore, CBDNet is a model that more closely resembles real-world scenarios, using Poisson noise, which more closely approximates true noise, for training while still achieving good denoising performance. While CBDNet demonstrates good performance in real-world environments, it still has the following drawbacks: Firstly, since the denoising intensity of the non-blind denoising network in CBDNet is determined by the estimation of the noise estimation network, if the noise estimation network overestimates the noise level, the non-blind denoising network will over-denoise, resulting in loss of image detail and blurred edges after denoising. Secondly, although the U-Net structure of the non-blind denoising network can combine features from different structural parts, the transposed convolution process may result in the loss of detailed information due to the different levels of the two.

[0063] like Figure 1 As shown, the CBDNet denoising network optimization method for enhancing edge features provided in this embodiment includes the following steps in actual execution:

[0064] Step S1: Obtain the real image dataset and the synthetic image dataset respectively.

[0065] This invention primarily targets Poisson-Gaussian noise removal in real-world photographs. Therefore, it uses both real images and synthetically produced noisy images as a dataset, and alternately inputs them into the network. In one embodiment of the CBDNet denoising network optimization method for enhancing edge features of this invention, step S1 may specifically include:

[0066] Step S11: Obtain the real image dataset by taking pictures using a real imaging device.

[0067] The real image dataset consists of 120 images from the RENOIR dataset, all of which were captured by real imaging devices.

[0068] Step S12: Obtain a noiseless image dataset, add Poisson-Gaussian noise to each noiseless image in the noiseless image dataset, and then perform de-mosaic and gamma correction processes on the noiseless images with added Poisson-Gaussian noise to obtain the synthetic image dataset.

[0069] Specifically, the synthetic image dataset uses 400 images from the BSD500 dataset, 1600 images from the Waterloo dataset, and 1600 images from the MIT-Adobe FiveK dataset, all of which are noise-free images. Then, Poisson-Gaussian noise, a common feature in real-world applications, is added to these images. Next, mimicking the in-camera processing of imaging devices, de-mosaicing and gamma correction are performed on these images. In some embodiments of this invention, JPEG compression is also applied to some images to mimic common image post-processing, ultimately resulting in a synthetic noisy image.

[0070] Step S2: Input the original images selected from the real image dataset and the synthetic image dataset into the noise estimation network alternately, and output a noise estimation map.

[0071] The noise-free image, noise level, and noise image of the artificially synthesized image can be obtained directly; for real images, only the noise image can be obtained, but the noise level cannot; the noise-free image is obtained by averaging different noisy images of the same scene. However, this method has the drawback of overly smooth images. Therefore, in order to obtain a moderate denoising effect, this invention uses artificially synthesized images and real images alternately during the training process, and these images are first input into the noise estimation network.

[0072] Specifically, the original image input to the noise estimation network is the same size as the noise estimation map output by the network. The noise estimation network employs a fully convolutional network without pooling or batch normalization (BN) layers. This invention estimates the noise level of the input image using a noise estimation network and outputs a noise estimation map of constant size, representing the noise intensity detected by the network, which serves as one of the inputs to the subsequent non-blind denoising network. Since the input and output sizes of the noise estimation network are the same, it employs a fully convolutional network without pooling or BN layers. The original image and the noise estimation map are then fed together into the optimized non-blind denoising network for the next step of denoising (described in detail below).

[0073] In one embodiment of the present invention, the resolution of both the original image input to the noise estimation network and the noise estimation map output by the noise estimation network is H×W×3; for example... Figure 2 As shown, the noise estimation network includes 5 standard convolutional layers with ReLU activation. The first 4 convolutional layers contain 32 convolutional kernels, and the last convolutional layer contains 3 convolutional kernels. The kernel size of all convolutional layers is 3×3.

[0074] Step S3: Stitch together the original image and the noise estimation image.

[0075] Specifically, the concatenate function is used to concatenate the original image and the noise estimation image into a vector with a resolution of H×W×6.

[0076] Step S4: Input the stitched original image and the noise estimation map into a non-blind denoising network and output the denoised image. The non-blind denoising network adopts the U-Net architecture, which includes several convolutional layers, two bilinear interpolation upsampling layers and two IndexNet modules.

[0077] The non-blind denoising network of this invention takes the concatenated original image and noise estimation map as input, passes through several convolutional layers, and undergoes two IndexNet-guided upsampling and downsampling operations to finally output the denoised image. The noise estimation map obtained through the noise estimation network guides the network in image denoising, i.e., non-blind denoising. The accuracy of noise level estimation has a certain impact on the denoising effect.

[0078] In step S4, the non-blind denoising network takes the concatenated original image and noise estimation map as input. After processing by the network, it finally outputs the denoised image. Current non-blind denoising networks use a U-Net structure, consisting of 17 convolutional layers, two average pooling layers, and two transposed convolutional layers, with two skip connections. Finally, a residual learning method is used. The model of this invention uses residual learning. The network first calculates the noise residual, then subtracts the noise residual from the original image, and outputs the final denoising result. The optimized solution of this invention is to introduce an IndexNet module into the network to guide the downsampling and upsampling processes, thereby reducing loss and preserving more image edge information. The two transposed convolutional layers are replaced with bilinear interpolation upsampling layers, and an IndexNet is added at each of the two skip connections. By extracting the image features before downsampling, guidance maps for downsampling and upsampling are generated to guide the sampling process and preserve more image edge features.

[0079] Specifically, such as Figure 3As shown, the non-blind denoising network of the present invention includes 5 convolutional blocks, 1 convolutional layer, 2 average pooling layers (AvgPool1 and AvgPool2), 2 bilinear interpolation upsampling layers, and 2 IndexNet modules (IndexNet1 and IndexNet2), and adopts a residual learning method. The first convolutional block (Pre_Conv) consists of 2 convolutional layers, the second convolutional block (Conv1) consists of 3 convolutional layers, the third convolutional block (Conv2) consists of 6 convolutional layers, the fourth convolutional block (Conv3) consists of 3 convolutional layers, and the fifth convolutional block (Conv4) consists of 2 convolutional layers. Therefore, the non-blind denoising network of this invention includes a total of 17 convolutional layers. The first IndexNet module is added after the first convolutional block to guide the first average pooling layer and the second bilinear interpolation upsampling layer. The second IndexNet module is added after the second convolutional block to guide the second average pooling layer and the first bilinear interpolation upsampling layer. In one embodiment of this invention, the specific network parameters of the non-blind denoising network are shown in Table 1. It should be noted that this invention does not specifically limit the network parameters.

[0080] Table 1. Specific network parameters of the non-blind denoising network.

[0081]

[0082]

[0083] This invention, by introducing the IndexNet module, can preserve more image details and edge features. Since downsampling loses some image information and may cause edge blurring, this invention uses the IndexNet module to extract features from the image before downsampling and uses them to guide the corresponding downsampling and upsampling processes. IndexNet is a module that optimizes the network at the sampling angle. Because it can guide the upsampling and downsampling processes to preserve more edge information, IndexNet is often used in image segmentation algorithms. Like other parts of the model, IndexNet learns continuously during training. Specifically, each IndexNet module includes an IndexBlock, a Sigmoid activation function, and a Softmax activation function. The IndexBlock, as a learning module, consists of one convolutional layer and one pixel reconstruction layer.

[0084] In one embodiment of the CBDNet denoising network optimization method for enhancing edge features of the present invention, step S4 may specifically include:

[0085] Step S41: The stitched original image and the noise estimation map are fed into IndexBlock. First, the image is passed through a convolutional layer with a kernel stride of 2 and an output channel of 4 to halve its length and width. Then, the image is passed through a pixel reconstruction layer to merge the four feature maps into one feature map, and its size is restored to its original size.

[0086] Step S42: The image output by IndexBlock is used to guide the upsampling process after passing through the Sigmoid activation function. The image output by IndexBlock is multiplied element-wise with the image before upsampling, and then the upsampling operation is performed.

[0087] Step S43: The image output by IndexBlock, after passing through the Sigmoid function and then the Softmax activation function, is used to guide the downsampling process. The image output by IndexBlock is first multiplied element-wise with the image before downsampling, then downsampling is performed, and after downsampling, each element in the image is multiplied by 4.

[0088] In a specific implementation of this invention, the original image after stitching together with a resolution of H×W×6 and the noise estimation map are input into a non-blind denoising network. After passing through two IndexNet-guided average pooling layers, the pixel size gradually decreases. After passing through two IndexNet-guided bilinear interpolation upsampling layers, the pixel size is restored to its original size, and an image with a resolution of H×W×3 is output.

[0089] Furthermore, in this invention, the noise estimation network and the non-blind denoising network are trained according to the total loss function, and the parameters are updated using the stochastic gradient descent method. Both networks are trained together. During training, the three loss functions are linearly added to obtain the total loss function, which is then used to train the two networks. The total loss function is a weighted sum of the asymmetric loss function and the total variation loss function at the output position of the noise estimation network, and the pixel-level mean square error loss function at the output position of the non-blind denoising network. This invention defines two loss functions at the output position of the noise estimation network and one loss function at the output position of the non-blind denoising network.

[0090] Specifically, the asymmetric loss function L at the output position of the noise estimation network is calculated using the following formula. asymm ,

[0091]

[0092] Among them, y i This represents the image input to the noise estimation network. σ(y) represents the estimated noise intensity. i) represents the noise intensity of the Ground-Truth noise-free image; for I e When e is true, I e =1, otherwise I e =0; By assigning a value of α < 0.5, a greater penalty is applied to the underestimation of noise intensity. In the specific implementation of this invention, α = 0.3 is used by default.

[0093] The noise estimation network outputs a noise estimation map of the image, which represents the detected noise intensity. The best denoising effect is achieved when this map almost matches the noise intensity of the ground-truth noise level input to the non-blind denoising network. However, insufficient denoising leaves some noise in the image, significantly impacting visual quality; while excessive denoising, although potentially causing edge loss, results in relatively better visual quality. Therefore, this invention defines an asymmetric loss function at the output of the noise estimation network, as shown in formula (1), and trains the network by comparing the ground-truth noise level map with the noise estimation map output by the noise estimation network. In the training set, the ground-truth noise level map of the artificially synthesized image is directly available, but images of the real environment are not. Therefore, this loss function is only used when the input is an artificially synthesized image. Underestimating noise intensity results in a more severe penalty than overestimating it.

[0094] The total variation loss function L at the output location of the noise estimation network is calculated using the following formula. TV ,

[0095]

[0096] Among them, graf h Graf represents the gradient in the horizontal direction. v || represents the gradient in the vertical direction; ||2 represents the L2 norm;

[0097] The total variation loss function is also defined at the output of the noise estimation network. Since Bosson-Gaussian noise is usually non-smooth, this invention defines this loss function to limit the smoothness of the noise estimation map, as shown in Equation (2).

[0098] The pixel-level mean square error loss function L at the output position of the non-blind denoising network is calculated using the following formula. rec ,

[0099]

[0100] in, This represents the denoising result, where x represents the Ground-Truth noise-free image.

[0101] The pixel-level mean squared error loss function trains the network by comparing the Ground-Truth noise-free image with the denoised image output by the non-blind denoising network. The Ground-Truth noise-free image of the artificially synthesized image can also be obtained directly. For images of real-world environments, an almost noise-free image is obtained by averaging different noisy images of the same scene.

[0102] The total loss function is calculated using the following formula.

[0103] L = L rec +λ asymm L asymm +λ TV L TV (4)

[0104] Where, λ asymm Represents the asymmetric loss function L asymm The weight, λ TV L represents the total variation loss function TV The weights, and L is not used when training with real data. asymm Considering only L TV and L rec At this time L asymm =0. In the specific implementation, λ asymm The default setting is 0.9, λ TV The default value is 0.05.

[0105] This invention uses formula (4) as the loss function for training, adopts stochastic gradient descent for parameter updating, and trains two networks, a noise estimation network and a non-blind denoising network, to finally obtain a denoising network that effectively preserves edge details. This solves the problem of edge blurring caused by information loss during the downsampling process of the non-blind denoising network and improves the image recognition effect.

[0106] This invention, as an image preprocessing procedure, primarily targets subsequent image edge detection and semantic segmentation. In its specific implementation, the noisy image to be processed is fed into the model of this invention to obtain a denoised image that preserves edge details. This denoised image is then used for subsequent operations to improve accuracy and precision.

[0107] The CBDNet denoising network optimization method for enhancing edge features provided in this invention incorporates IndexNet into a non-blind denoising network, replaces the two transposed convolutional layers with bilinear interpolation upsampling layers, and guides the sampling process by performing element-wise multiplication between the two weight maps output by IndexNet and the image before sampling. This solves the problem of edge blurring caused by information loss during downsampling in non-blind denoising networks. By combining U-Net and IndexNet, it retains the ability of U-Net to acquire multi-level features while enhancing edge detail features through the IndexNet module. It improves the effect of subsequent image processing or detection by reducing the loss of image edge information during denoising. The non-blind denoising network approach of CBDNet preserves more image texture features while reducing the impact of over-denoising caused by the noise estimation network overestimating the noise level.

[0108] Accordingly, such as Figure 4 As shown, the present invention also provides a CBDNet denoising network optimization system for enhancing edge features, comprising:

[0109] Dataset acquisition module 1 is used to acquire real image datasets and synthetic image datasets respectively;

[0110] Noise estimation module 2 is used to alternately input original images selected from the real image dataset and the synthetic image dataset into the noise estimation network and output a noise estimation map;

[0111] Image stitching module 3 is used to stitch the original image and the noise estimation map together;

[0112] The non-blind denoising module 4 is used to input the stitched original image and the noise estimation map into the non-blind denoising network and output the denoised image. The non-blind denoising network adopts the U-Net architecture, which includes several convolutional layers, two bilinear interpolation upsampling layers and two IndexNet modules.

[0113] The CBDNet denoising network optimization system for enhanced edge features provided in this invention incorporates IndexNet into a non-blind denoising network, replaces the two transposed convolutional layers with bilinear interpolation upsampling layers, and guides the sampling process by performing element-wise multiplication between the two weight maps output by IndexNet and the image before sampling. This solves the problem of edge blurring caused by information loss during downsampling in non-blind denoising networks. By combining U-Net and IndexNet, it retains U-Net's ability to acquire multi-level features while enhancing edge detail features through the IndexNet module. It improves the effect of subsequent image processing or detection by reducing the loss of image edge information during denoising. It uses the non-blind denoising network approach of CBDNet to retain more image texture features while reducing the impact of over-denoising caused by the noise estimation network overestimating the noise level.

[0114] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0115] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for optimizing a CBDNet denoising network to enhance edge features, characterized in that, include: Obtain real image datasets and synthetic image datasets respectively; The original images selected from the real image dataset and the synthetic image dataset are alternately input into the noise estimation network, and the output is a noise estimation map; The original image and the noise estimation map are stitched together; The stitched original image and the noise estimation map are input into a non-blind denoising network, which outputs a denoised image. The non-blind denoising network adopts a U-Net architecture, including several convolutional layers, two bilinear interpolation upsampling layers, and two IndexNet modules. The non-blind denoising network includes 5 convolutional blocks, 1 convolutional layer, 2 average pooling layers, two bilinear interpolation upsampling layers, and two IndexNet modules. The first convolutional block consists of 2 convolutional layers, the second convolutional block consists of 3 convolutional layers, the third convolutional block consists of 6 convolutional layers, the fourth convolutional block consists of 3 convolutional layers, and the fifth convolutional block consists of 2 convolutional layers. The first IndexNet module is added after the first convolutional block to guide the first average pooling layer and the second bilinear interpolation upsampling layer, and the second IndexNet module is added after the second convolutional block to guide the second average pooling layer and the first bilinear interpolation upsampling layer.

2. The CBDNet denoising network optimization method for enhancing edge features according to claim 1, characterized in that, The process of acquiring real image datasets and synthetic image datasets respectively includes: The real image dataset was obtained by capturing images using real imaging equipment; A noiseless image dataset is obtained, and Poisson-Gaussian noise is added to each noiseless image in the noiseless image dataset. Then, the noiseless images with added Poisson-Gaussian noise are subjected to de-mosaic and gamma correction processes to obtain the synthetic image dataset.

3. The CBDNet denoising network optimization method for enhancing edge features according to claim 1, characterized in that, The step of alternately inputting original images selected from the real image dataset and the synthetic image dataset into the noise estimation network and outputting a noise estimation map specifically includes: The original image input to the noise estimation network is the same size as the noise estimation map output by the noise estimation network. The noise estimation network is a fully convolutional network and has no pooling layers or BN layers.

4. The CBDNet denoising network optimization method for enhancing edge features according to claim 3, characterized in that, The original image input to the noise estimation network and the noise estimation map output by the noise estimation network both have a resolution of H×W×3. The noise estimation network includes 5 standard convolutional layers with ReLU activation. The first 4 convolutional layers contain 32 convolutional kernels, and the last convolutional layer contains 3 convolutional kernels. The kernel size of all convolutional layers is 3×3.

5. The CBDNet denoising network optimization method for enhancing edge features according to claim 4, characterized in that, The process of stitching together the original image and the noise estimation image specifically includes: The concatenate function is used to concatenate the original image and the noise estimation image into a vector with a resolution of H×W×6.

6. The CBDNet denoising network optimization method for enhancing edge features according to claim 1, characterized in that, Each IndexNet module includes one IndexBlock, a Sigmoid activation function, and a Softmax activation function. The IndexBlock, as the learning module, consists of one convolutional layer and one pixel reconstruction layer. The step of inputting the stitched original image and the noise estimation map into a non-blind denoising network and outputting the denoised image specifically includes: The stitched original image and the noise estimation map are fed into IndexBlock. First, the image is halved in length and width by passing through a convolutional layer with a kernel stride of 2 and an output channel of 4. Then, the four feature maps are merged into one feature map by passing through a pixel reconstruction layer, and its size is restored to the original size. The image output by IndexBlock is used to guide the upsampling process after passing through the Sigmoid activation function. The image output by IndexBlock is multiplied element-wise with the image before upsampling, and then the upsampling operation is performed. The image output by IndexBlock, after passing through the Sigmoid function and then the Softmax activation function, is used to guide the downsampling process. The image output by IndexBlock is first multiplied element-wise with the image before downsampling, then downsampled, and finally each element in the image is multiplied by 4 after downsampling.

7. The CBDNet denoising network optimization method for enhancing edge features according to claim 6, characterized in that, The step of inputting the stitched original image and the noise estimation map into a non-blind denoising network and outputting the denoised image specifically includes: The original image and the noise estimation map, which are stitched together with a resolution of H×W×6, are input into a non-blind denoising network. After two IndexNet-guided average pooling layers, the pixel size gradually decreases. After two IndexNet-guided bilinear interpolation upsampling layers, the pixel size is restored to its original size, and an image with a resolution of H×W×3 is output.

8. The CBDNet denoising network optimization method for enhancing edge features according to claim 6, characterized in that, The noise estimation network and the non-blind denoising network are trained based on the total loss function, and the parameters are updated using stochastic gradient descent. Both networks are trained together. The total loss function is a weighted sum of the asymmetric loss function and total variation loss function at the output position of the noise estimation network, and the pixel-level mean square error loss function at the output position of the non-blind denoising network. The asymmetric loss function at the output location of the noise estimation network is calculated using the following formula. , （1） in, This represents the image input to the noise estimation network. This represents the estimated noise intensity. This represents the noise intensity of a noise-free ground-truth image; for , When true, ,otherwise ; By assigning values To impose greater penalties for underestimating noise intensity; The total variation loss function at the output location of the noise estimation network is calculated using the following formula. , （2） in, Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction; express Norm; The pixel-level mean square error loss function at the output position of the non-blind denoising network is calculated using the following formula. , （3） in, This indicates the denoising result. This represents a noise-free ground-truth image. The total loss function is calculated using the following formula. （4） in, Represents the asymmetric loss function The weight, Represents the total variation loss function The weights are set, and no weights are used when training with real data. Only considering and ,at this time .

9. A CBDNet denoising network optimization system for enhancing edge features, characterized in that, include: The dataset acquisition module is used to acquire real image datasets and synthetic image datasets respectively; The noise estimation module is used to alternately input original images selected from the real image dataset and the synthetic image dataset into the noise estimation network and output a noise estimation map. An image stitching module is used to stitch the original image and the noise estimation map together; A non-blind denoising module is used to input the stitched original image and the noise estimation map into the non-blind denoising network and output the denoised image. The non-blind denoising network adopts a U-Net architecture, including several convolutional layers, two bilinear interpolation upsampling layers, and two IndexNet modules. The non-blind denoising network includes 5 convolutional blocks, 1 convolutional layer, 2 average pooling layers, two bilinear interpolation upsampling layers, and two IndexNet modules. Specifically, the first convolutional block consists of 2 convolutional layers, the second convolutional block consists of 3 convolutional layers, the third convolutional block consists of 6 convolutional layers, the fourth convolutional block consists of 3 convolutional layers, and the fifth convolutional block consists of 2 convolutional layers. The first IndexNet module is added after the first convolutional block to guide the first average pooling layer and the second bilinear interpolation upsampling layer, and the second IndexNet module is added after the second convolutional block to guide the second average pooling layer and the first bilinear interpolation upsampling layer.

Images