Region-aware driven self-supervised image denoising method, system, device and medium

Abstract

This application provides a region-aware driven self-supervised image denoising method, system, apparatus, and medium, belonging to the field of image processing technology. The method includes: constructing an adaptive region-aware network framework, including an initial adaptive region-aware blind spot network, an adaptive hybrid scrambling branch, a non-scrambling branch, and a random geometric transformation pixel downsampling module; training the adaptive hybrid scrambling branch and the non-scrambling branch to generate a trained adaptive region-aware blind spot network; inputting a noisy image into the trained adaptive region-aware network to obtain a tested and verified adaptive region-aware blind spot network; and inputting the noisy image into the tested and verified adaptive region-aware blind spot network to obtain a denoised image. This achieves superior denoising performance.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a region-aware driven self-supervised image denoising method, system, device and medium. Background Technology

[0002] Image denoising, a fundamental task in low-level vision, aims to recover clean information from noisy images, and its performance directly affects the effectiveness of subsequent advanced tasks such as image analysis and object recognition. With the development of computer vision technology, denoising methods have gradually evolved from traditional optimization strategies to deep learning-based solutions, forming two major research directions: supervised learning and self-supervised learning.

[0003] Traditional image denoising methods can be broadly categorized into spatial domain and transform domain methods based on the statistical analysis and processing of noise signals. Spatial domain methods focus on the two-dimensional pixel space of the image, directly manipulating pixel values. For example, Gaussian filtering, based on the assumption that "the farther away from the center pixel, the smaller its contribution to the new value of the center pixel," constructs a Gaussian kernel function to perform a weighted average on the image, which can smooth noise to some extent. However, manually setting the kernel size and parameters is difficult to adapt to complex scenes, and it can easily over-blur the image when there is a lot of noise. Bilateral filtering introduces two weighting factors: spatial distance and pixel value difference, attempting to preserve edges while denoising. However, it has high computational complexity and is time-consuming when processing large images. Median filtering replaces the center pixel with the median of the pixel's neighborhood, which is effective against salt-and-pepper noise, but it can easily destroy the continuity of image texture. Transform domain methods, on the other hand, first transform the image to the frequency domain. Common examples include Fourier transform and wavelet transform. They utilize the different distribution characteristics of noise and signal in the frequency domain. Noise is mostly concentrated in the high frequency domain, and image details and edges are also often associated with high frequencies, making them difficult to distinguish. This approach is used to process noise. For example, Fourier transform denoising, which suppresses high-frequency noise components through filters, can easily inadvertently delete high-frequency details of the image itself. Wavelet transform can decompose images at multiple scales and denoise them specifically at different scales, but the selection of wavelet basis functions and threshold settings rely on experience, making it difficult to accurately balance denoising and detail preservation. These traditional methods are essentially based on prior assumptions, such as the statistical distribution of noise and local image smoothing, and design fixed rules. However, they have poor adaptability to the complex and varied noise in real-world scenarios, such as sensor noise under different lighting conditions and random noise in complex environments, and it is difficult to achieve both denoising effect and image detail preservation.

[0004] With the rise of deep learning, denoising algorithms based on convolutional neural networks have emerged. Supervised learning methods rely on large-scale, precisely labeled "clean-noise image" paired datasets to train models. In synthetic additive white Gaussian noise scenes, the model can learn noise patterns and effectively denoise. However, real-world noise is affected by the shooting equipment; different sensors have significantly different noise characteristics, such as CMOS and CCD sensors exhibiting different noise performances. It is also affected by ambient lighting, with noise distribution differing under strong light, weak light, and complex lighting conditions. Furthermore, transmission channels can influence noise, with network packet loss and interference introducing new noise. Noise exhibits spatial correlation, with adjacent pixel noise being correlated, and also signal dependence. Noise intensity is related to image content, for example, noise is more pronounced in dark areas. These complex characteristics are vastly different from simple synthetic additive white Gaussian noise. Meanwhile, collecting high-quality paired datasets of real-world scenes requires strict control of shooting conditions, including fixed equipment and environment, and multiple collections of clean and noisy images of the same scene. However, this is almost impossible in dynamic scenes such as traffic videos, field monitoring, and scenes that cannot be reproduced, such as single-shot shooting for the restoration of art and cultural relics. This results in the supervised denoising model having extremely poor generalization ability in real-world scenes, and its denoising effect drops significantly when faced with images with unknown noise distribution.

[0005] To overcome the data bottleneck of supervised learning, self-supervised denoising methods have emerged. These methods attempt to break free from the strong dependence on "clean-noise" paired data and learn denoising patterns from single noisy images or data without precise annotations. However, existing self-supervised methods still have significant limitations: First, some methods implicitly assume "noise pixels are independent," but the spatial correlation of real noise renders the denoising patterns learned by the model ineffective, failing to effectively suppress correlated noise. Second, strategies such as pixel resampling and random perturbation introduced to weaken noise correlation do not consider regional differences, easily leading to detail degradation in textured regions and potentially leaving dense noise in flat regions. These problems make it difficult for existing methods to achieve a fine balance between noise removal and texture preservation in real-world image denoising. Summary of the Invention

[0006] This invention provides a region-aware driven self-supervised image denoising method, system, apparatus, and medium to at least solve the problem in the prior art where blind spot networks lack the ability to perceive the local structure of images, resulting in uneven denoising effects between flat and textured regions.

[0007] In a first aspect, embodiments of this application provide a region-aware driven self-supervised image denoising method, the method comprising:

[0008] Step S1: Construct a noisy image dataset;

[0009] Step S2: Construct an adaptive region-aware network framework, which includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch.

[0010] Step S3: Perform parallel training on the adaptive hybrid scrambling branch and the non-scrambling branch to generate a trained adaptive region-aware blind spot network;

[0011] Step S4: Input the noisy image into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, return to step S3 to continue optimization.

[0012] Step S5: Input the noisy image into the tested and verified adaptive region-aware blind spot network to obtain the denoised image.

[0013] Further, in step S2, the adaptive hybrid scrambling branch performs coarse scrambling on the noisy image to generate scrambled sub-images;

[0014] The scrambled sub-image is processed by an initial adaptive region-aware blind spot network to obtain a coarsely estimated image;

[0015] Based on the local standard deviation of the coarsely estimated image, flat regions and textured regions are distinguished.

[0016] Pixel perturbation is enhanced in the flat region and pixel perturbation is reduced in the textured region to generate finely scrambled training samples;

[0017] The non-scrambled branch performs random transformation pixel downsampling on the noisy image through the random geometric transformation pixel downsampling module, generating training samples enhanced by random transformation pixel downsampling.

[0018] Furthermore, the non-scrambled branch performs random transformation pixel downsampling on the noisy image through a random geometric transformation pixel downsampling module to generate training samples enhanced by random transformation pixel downsampling, specifically including:

[0019] The noisy image is subjected to block-level random geometric transformation by the random geometric transformation pixel downsampling module to obtain an unrandomized sub-image;

[0020] The unscrambled sub-images are input into the initial adaptive region-aware blind spot network for processing to obtain training samples with randomized pixel downsampling enhancement;

[0021] The block-level random geometric transformations include: random rotation and random flipping.

[0022] Furthermore, in step S3, the adaptive hybrid scrambling branch and the non-scrambling branch are trained in parallel to generate a trained adaptive region-aware blind spot network, specifically including:

[0023] The finely scrambled training samples from the adaptive hybrid scrambling branch output and the randomly transformed pixel downsampling enhanced training samples from the non-scrambling branch output are combined, and the initial adaptive region-aware blind spot network is trained using a composite loss function and then trained using the Adam optimizer to obtain the trained adaptive region-aware blind spot network.

[0024] Further, in step S4, the noisy image is input into the trained adaptive region perception network for testing, resulting in a tested and validated adaptive region perception blind spot network, specifically including:

[0025] The test dataset is input into the trained adaptive region-aware network for testing. Specifically, a multi-step long region-aware fusion strategy based on spatial distribution is used to fuse the denoising results of different long-step pixel reconstruction downsampling, and the final denoised image is obtained and output to complete the network test and obtain the tested and verified adaptive region-aware blind spot network.

[0026] Furthermore, a coarse scrambling process is performed on the noisy image to generate a scrambled sub-image, the expression of which is:

[0027]

[0028] in, This represents the input noisy image. This indicates that a sliding window of size 2×2 with a step size of 2 is used to randomly scramble pixels in the noisy image. This indicates pixel re-sampling with a step size of 2;

[0029] The scrambled sub-image is processed by an initial adaptive region-aware blind spot network to obtain a coarsely estimated image, the expression of which is:

[0030]

[0031] in, Represents a scrambled sub-image. This indicates a rough estimate of the image;

[0032] Based on the local standard deviation of the roughly estimated image, flat regions and textured regions are distinguished, and the expression is as follows:

[0033]

[0034] in, This represents an n×n window centered at pixel (i,j). This represents the standard deviation of pixel values ​​within the calculation window; Indicates local standard deviation;

[0035] The pixel perturbation is enhanced in the flat region and weakened in the textured region to generate finely scrambled training samples, the expression of which is:

[0036] when hour,

[0037] The non-scrambled branch performs random geometric transformation pixel downsampling on the noisy image through the random geometric transformation pixel downsampling module, generating training samples enhanced by random transformation pixel downsampling, the expression of which is:

[0038] when hour,

[0039] in, This represents random scrambling within an n×n window, where r is a preset threshold for distinguishing between flat and textured regions. This indicates that the training samples were scrambled. This represents training samples enhanced by random pixel downsampling.

[0040] The expression for the composite loss function is:

[0041]

[0042]

[0043]

[0044]

[0045]

[0046] in, Represents the composite loss function. This represents the adaptive hybrid scrambling branch loss. Indicates the loss of the non-disordered branch. This represents the denoised output of the adaptive hybrid scrambling branch. This represents the denoised output of the non-scrambled branch. and They represent step sizes of 1 and 2 respectively. Random transformation pixel downsampling and its inverse operation.

[0047] Furthermore, a multi-step region-aware fusion strategy based on spatial distribution is used to fuse the denoising results of pixel recombination downsampling with different step lengths, and the final denoised image is obtained and output. The specific steps include the following:

[0048] Using a trained adaptive region-aware network, noisy input images are processed. Each of them uses a different set of downsampling step sizes To process, Three sets of denoising results were obtained. Its expression is:

[0049]

[0050] in, and These are standard pixel recombining downsampling and its inverse operation, respectively.

[0051] Select the denoising result with a step size of 2 As a benchmark for structural analysis, its local standard deviation plot is calculated; and based on a preset low-frequency threshold... and high frequency threshold Accurately segment the image into low frequencies. , intermediate frequency and high frequency Binary masks for three regions;

[0052] Calculate intermediate blending results for flat and textured regions. and :

[0053] ;

[0054] in and The hyperparameter dataset for controlling the fusion weights, , ;

[0055] Generate the final denoised image Its expression is:

[0056] .

[0057] Secondly, embodiments of this application also provide a system for applying the region-aware driven self-supervised image denoising method as described in the above aspects, the system comprising:

[0058] The dataset construction module is used to construct noisy image datasets;

[0059] A network architecture construction module is used to build an adaptive region-aware network framework, which includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch.

[0060] The network training module is used to train the adaptive hybrid scrambling branch and the non-scrambling branch in parallel to generate a trained adaptive region-aware blind spot network.

[0061] The network testing module is used to input noisy images into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, it returns to the network optimization module for further optimization.

[0062] The denoising module is used to input the noisy image into the tested and verified adaptive region-aware blind spot network to obtain the denoised image.

[0063] Thirdly, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the region-aware driven self-supervised image denoising method as described in the preceding aspects.

[0064] Fourthly, a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the region-aware driven self-supervised image denoising method as described in the preceding aspects.

[0065] As can be seen from the above technical solutions, the present invention has the following advantages:

[0066] The region-aware driven self-supervised image denoising method provided in this application can adaptively balance dense noise removal and texture detail preservation, eliminate spatially correlated and signal-dependent noise, and significantly improve the self-supervised denoising performance and inference efficiency of real-world images.

[0067] This invention employs an adaptive hybrid scrambling strategy, enabling the network to apply perturbations of varying intensities to flat and textured regions based on the image content. This effectively protects the fine structure of the image while enhancing data and suppressing dense noise.

[0068] This invention utilizes random geometric transformation in the random geometric transformation pixel downsampling module to effectively break the inherent spatial correlation in real-world noise, forcing the network to learn more generalizable denoising features and improving the thoroughness of denoising.

[0069] This invention employs a multi-step region-aware fusion strategy during the inference stage, which combines the denoising advantages of different downsampling steps for different regions of the image. This significantly improves the noise residue in flat areas and the detail blurring in textured areas, ultimately achieving a better denoising effect. Attached Figure Description

[0070] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart of the method of the present invention.

[0072] Figure 2 This is a schematic diagram of the adaptive region perception blind spot network framework of the present invention.

[0073] Figure 3 This is a schematic diagram illustrating the principle of the random pixel downsampling operation of the present invention.

[0074] Figure 4 This is a schematic diagram illustrating the execution process of the multi-step long-area perception fusion strategy of the present invention. Detailed Implementation

[0075] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this patent, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this patent.

[0076] This application provides a region-aware driven self-supervised image denoising method, system, apparatus, and medium, which solves the current urgent technical problem of achieving a balance in denoising effect between flat and textured regions.

[0077] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0078] Figure 1 This is a flowchart illustrating a region-aware driven self-supervised image denoising method provided in an embodiment of this application. Figure 1 As shown in the figure, the region-aware driven self-supervised image denoising method provided in this application embodiment specifically includes the following steps:

[0079] The method includes:

[0080] Step S1: Construct a noisy image dataset;

[0081] The noisy image dataset includes a training dataset, a validation dataset, and a test dataset. Constructing the noisy image dataset specifically includes:

[0082] Obtain a real-world sRGB image dataset;

[0083] Noisy sRGB images were selected from the SIDDMedium dataset, containing 320 pairs of images. All images were cropped to a predetermined size (120×120) and used as the training dataset.

[0084] sRGB image patches were selected from SIDDValidation as the validation dataset; specifically, 1280 image patches of size 256×256 from the SIDDValidation subset were used as the validation data.

[0085] Finally, unlabeled sRGB images were selected from the SIDDBenchmark dataset as the test dataset for evaluation using the SIDD online evaluation website. During the testing phase, the SIDDBenchmark dataset was chosen from the denoised datasets. Since clean, real-world images are not publicly available for the benchmark datasets, the denoised results needed to be submitted to their official evaluation server for performance evaluation.

[0086] Step S2: Construct an adaptive region-aware network framework using an asymmetric downsampling blind spot network framework. The adaptive region-aware network framework includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch.

[0087] In this embodiment, the adaptive hybrid scrambling branch applies coarse-to-fine perturbations step-by-step according to the texture structure of the noisy image, achieving data augmentation and guiding region-aware denoising to simultaneously suppress dense noise and preserve texture details, thereby generating training samples that can better distinguish between noise and structure. The non-scrambling branch provides a stable training signal through a random geometric transformation pixel downsampling module.

[0088] In both the adaptive hybrid scrambling and non-scrambling branches, a random geometric transformation pixel downsampling module is used to destroy the spatial correlation of noise and enhance the generalization ability of the adaptive region perception network framework to different noise patterns.

[0089] In a specific implementation, see Figure 2 As shown, existing self-supervised methods based on blind spot networks struggle to balance dense noise removal and texture preservation when dealing with real-world noise due to their fixed downsampling step size and assumptions about pixel-independent noise. This invention constructs a dual-branch parallel structure adaptive region-aware network framework. This framework includes an adaptive hybrid scrambling branch and a non-scrambling branch, which share the same initial weight parameters of the adaptive region-aware blind spot network (BSN).

[0090] In step S2, the adaptive hybrid scrambling branch performs an adaptive hybrid scrambling operation.

[0091] The adaptive hybrid scrambling branch coarsely scrambles the noisy image to generate scrambled sub-images.

[0092] The scrambled sub-image is processed by an initial adaptive region-aware blind spot network to obtain a coarsely estimated image;

[0093] Based on the local standard deviation of the coarsely estimated image, flat regions and textured regions are distinguished.

[0094] Pixel perturbation is enhanced in the flat region and pixel perturbation is reduced in the textured region to generate finely scrambled training samples;

[0095] Perform a coarse scrambling step: Use a 2x2 window with a step size of 2 to perform global random pixel scrambling on the noisy image. The purpose of this step is to preserve the macroscopic texture structure of the image to the maximum extent while introducing slight perturbations.

[0096] The step of generating a coarse estimation image is as follows: After the image that has undergone coarse scrambling is downsampled by pixel recombination with a step size of 2, it is input into the initial adaptive region-aware blind spot network to obtain a preliminary denoised image. This image will be used as a coarse estimation image for the structured guidance of subsequent steps.

[0097] The fine scrambling step involves calculating the local window standard deviation of the coarsely estimated image and identifying pixel locations in the noisy image as flat or textured regions based on a preset numerical threshold. In this embodiment, the preset numerical threshold is 1.5.

[0098] For the flat regions, a 3×3 window is used to apply stronger pixel randomization to enhance the data and improve the model's ability to suppress dense noise, while the texture regions remain unchanged to preserve details. Then, a pixel resampling module with a stride of 2 is used to generate the final enhanced image for training.

[0099] An adaptive hybrid scrambling branch is set in the adaptive region-aware blind spot network framework. The noisy image is first coarsely scrambled to generate a scrambled sub-image through the adaptive hybrid scrambling branch, and then processed by the initial adaptive region-aware blind spot network to obtain a coarsely estimated image.

[0100] Based on the local standard deviation distribution of the coarsely estimated image, flat regions and textured regions are distinguished. The perturbation of flat regions is enhanced and the perturbation of textured regions is weakened to perform fine scrambling, generating perturbation-enhanced training samples, and obtaining the output of the adaptive hybrid scrambling branch.

[0101] It should be further explained that the distinction between flat and textured regions based on the local standard deviation of the roughly estimated image includes:

[0102] The local standard deviation of the image is roughly estimated by calculating a sliding window. Regions with local standard deviation below the standard deviation threshold are defined as flat regions, and regions with local standard deviation above the standard deviation threshold are defined as textured regions. The window size and standard deviation threshold are set according to the image characteristics.

[0103] The non-scrambled branch performs random pixel downsampling on the noisy image using a random geometric transformation pixel downsampling module, generating training samples enhanced by random pixel downsampling. In this embodiment, the non-scrambled branch performs random pixel downsampling on the noisy image using the random geometric transformation pixel downsampling module to destroy the spatial correlation of noise, obtaining training samples enhanced by random pixel downsampling. The non-scrambled branch only provides a stable training signal through the random geometric transformation pixel downsampling module.

[0104] The non-scrambled branch performs random transformation pixel downsampling on the noisy image through a random geometric transformation pixel downsampling module to generate training samples enhanced by random transformation pixel downsampling, specifically including:

[0105] The noisy image is subjected to block-level random geometric transformation by the random geometric transformation pixel downsampling module to obtain an unrandomized sub-image;

[0106] The unscrambled sub-images are input into the initial adaptive region-aware blind spot network for processing to obtain training samples with randomized pixel downsampling enhancement;

[0107] The block-level random geometric transformations include: random rotation and random flipping.

[0108] In the adaptive region-aware blind spot network framework, a non-scrambled branch is set up. The non-scrambled branch only performs block-level random geometric transformations (including random rotation and random flipping) on ​​the noisy image to generate non-scrambled sub-images. These sub-images are then divided into non-scrambled sub-images to destroy the spatial correlation of noise. The output of the non-scrambled branch is obtained after the initial adaptive region-aware blind spot network is processed.

[0109] It should be further explained that a training step size factor of 5 is set, the noisy image is decomposed into sub-image blocks of the step size factor squared, and then the sub-image blocks are aggregated into several sub-images of the same size as the original image.

[0110] For each sub-image, a transformation is independently and randomly selected from a preset set of geometric transformations and applied. The set of transformations includes rotations of 0, 90, 180, or 270 degrees, as well as horizontal flips, vertical flips, diagonal flips, or anti-diagonal flips. The purpose is to force the initial adaptive region-aware blind spot network to learn features that are not sensitive to spatial location, thereby improving its generalization ability.

[0111] In the adaptive hybrid scrambling branch, that is, the input noisy image... An adaptive hybrid scrambling strategy is applied, which aims to dynamically generate enhanced training samples based on the local structure of the image to address the texture blurring problem caused by traditional uniform perturbation methods. The specific steps are as follows:

[0112] The first step is to perform a coarse scrambling operation and generate preliminary sub-images, which can be described as follows:

[0113]

[0114] in, This represents the input noisy image. This indicates that a sliding window of size 2×2 with a step size of 2 is used to randomly shuffle the pixels of the input image. This represents pixel resampling with a step size of 2. This operation decomposes the scrambled image into four sub-images. ( ).

[0115] The second step is to generate a coarsely estimated image. This involves generating the preliminary sub-image. The input is fed into the currently shared BSN network for one forward propagation, resulting in a set of initially denoised sub-images. Then The images are reconstructed into a complete, roughly denoised image, which will serve as a structured guide for subsequent steps.

[0116] The third step is to calculate the local standard deviation to identify image regions, and the process can be described as follows:

[0117]

[0118] in, This represents an n×n window centered at pixel (i,j) (n=3 in this embodiment). This represents the standard deviation of the pixel values ​​within the calculation window. This standard deviation value... It can effectively divide an image into flat regions and textured regions.

[0119] The fourth step is to perform selective fine-grained scrambling. Based on the local standard deviation, the original sub-image is then processed... Perform selective scrambling to obtain enhanced sub-images The process can be described as follows:

[0120] when hour,

[0121] when hour,

[0122] in, This represents random scrambling within an n×n window, where r is a preset threshold for distinguishing between flat and textured regions (r=1.5 in this embodiment). This operation applies stronger pixel random scrambling only to flat regions, while preserving the details of textured regions.

[0123] The fifth step is to generate the final enhanced image. This involves combining the four sub-images that have undergone fine scrambling. The inverse pixel recombining downsampling module merges the pixels to generate the final enhanced image used for training this branch. .

[0124] In the data processing flow of the two branches, because traditional fixed-mode sampling preserves the fixed relative positions of pixels, spatial correlation cannot be completely destroyed. Therefore, this invention introduces a random geometric transformation pixel downsampling module. See also... Figure 3 As shown, this module breaks the fixed structure by applying random geometric transformations to the sub-image blocks after downsampling. Specifically, the operation is as follows: First, a training step size factor (5 in this embodiment) is set, and the input image is decomposed into sub-image blocks equal to the square of the step size factor. Then, for each sub-image block, a transformation is independently and randomly selected from a preset set of geometric transformations and applied. The set of transformations includes rotations of 0 degrees, 90 degrees, 180 degrees, or 270 degrees, as well as horizontal flips, vertical flips, diagonal flips, or anti-diagonal flips.

[0125] Since the two-branch design aims to promote network convergence and enhance sample diversity while avoiding overfitting, a loss function is proposed based on the two-branch structure. This loss function is calculated by subtracting the outputs of the two branches from the original noisy image. The calculated L1 norm loss is weighted.

[0126] Denoising output of adaptive hybrid scrambling branch (Adaptive Hybrid Scrambling Branch) The generation process is as follows:

[0127]

[0128] Denoising output of the non-scrambled branch (non-scrambled branch) The generation process is as follows:

[0129]

[0130] in and They represent step sizes of 1 and 2 respectively. Random transformation pixel downsampling and its inverse operation.

[0131] Adaptive Hybrid Scrambled Branch Loss Non-disordered branch loss The calculation is as follows:

[0132]

[0133]

[0134] Composite loss function The weighted sum of the losses from the two branches takes the following form:

[0135]

[0136] Among them, weighting factors It can be set to 0.8 based on experience to focus on learning the adaptive hybrid scrambling branch, thereby guiding the network to better learn to distinguish between dense noise and real structure.

[0137] Step S3: Perform parallel training on the adaptive hybrid scrambling branch and the non-scrambling branch to generate a trained adaptive region-aware blind spot network;

[0138] In the adaptive region-aware blind spot network framework, the adaptive hybrid scrambling branch and the non-scrambling branch are combined, and region-aware parameters are added to the blind spot network to generate the adaptive region-aware blind spot network.

[0139] In a specific implementation, a region perception discrimination threshold is set for the adaptive hybrid scrambling module. Based on the asymmetric downsampling blind spot network framework, the adaptive hybrid scrambling branch, the non-scrambling branch with randomly transformed pixel downsampling, and the blind spot network are combined as the adaptive region perception blind spot network.

[0140] In step S3, the adaptive hybrid scrambling branch and the non-scrambling branch are trained in parallel to generate a trained adaptive region-aware blind spot network, specifically including:

[0141] The finely scrambled training samples from the adaptive hybrid scrambling branch output and the randomly transformed pixel downsampling enhanced training samples from the non-scrambling branch output are combined, and the initial adaptive region-aware blind spot network is trained using a composite loss function and then trained using the Adam optimizer to obtain the trained adaptive region-aware blind spot network.

[0142] The finely scrambled training samples from the adaptive hybrid scrambling branch output and the randomly transformed pixel downsampling enhanced training samples from the non-scrambling branch output are combined, and the initial adaptive region-aware blind spot network is trained using a composite loss function, including:

[0143] The L1 norm loss, which is the sum of the absolute values ​​of the pixel-by-pixel differences, is calculated between the finely scrambled training samples and their noisy images to obtain the loss of the adaptive hybrid scrambling branch.

[0144] The L1 norm loss, which is the sum of the absolute values ​​of the pixel-by-pixel differences, is calculated between the training samples enhanced by random pixel downsampling and their noisy images to obtain the loss of the non-scrambled branch.

[0145] The composite loss function is obtained by weighting the loss of the adaptive hybrid scrambling branch with the loss of the non-scrambling branch.

[0146] The weight factor of the composite loss function is set to 0.8 to balance the contributions of the adaptive hybrid scrambling branch and the non-scrambling branch during the training process.

[0147] The initial adaptive region-aware blind spot network was trained using the Adam optimizer to obtain a trained adaptive region-aware blind spot network. The outputs of the adaptive hybrid scrambled and non-scrambled branches were mutually constrained as the global optimization objective, and the Adam optimizer was used for training. The initial learning rate was set to 0.0001, and the learning rate decreased with the number of training epochs. The batch size was set to 4, and the total training period was set to 30 epochs. When the loss function value stabilized, the network weights were saved, and the training was completed.

[0148] Step S4: Input the noisy image into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, return to step S3 to continue optimization.

[0149] In step S4, the noisy image is input into the trained adaptive region perception network for testing, resulting in a tested and validated adaptive region perception blind spot network, specifically including:

[0150] The test dataset is input into the trained adaptive region-aware network for testing. Specifically, a multi-step long region-aware fusion strategy based on spatial distribution is used to fuse the denoising results of different long-step pixel reconstruction downsampling, and the final denoised image is obtained and output to complete the network test and obtain the tested and verified adaptive region-aware blind spot network.

[0151] This invention employs a multi-step region-aware fusion strategy to dynamically fuse denoising results with different downsampling steps. The fused result is used as the final denoised image to complete network testing, resulting in a tested and verified adaptive region-aware blind spot network.

[0152] The noisy image to be processed is input into the tested and verified adaptive region perception blind spot network to perform denoising processing on the acquired noisy image, thus obtaining a denoised image.

[0153] During the testing phase, the noisy images were subjected to random pixel downsampling with different time lengths, and the trained adaptive region perception network obtained multiple sets of denoising results.

[0154] Based on the local standard deviation distribution of the denoising results with intermediate step size, low-frequency, mid-frequency, and high-frequency regions are divided. Denoising results with corresponding step sizes are fused for different regions to generate the final denoised image. Specifically, large-step-size denoising results are fused for low-frequency regions, small-step-size denoising results are fused for high-frequency regions, and intermediate-step-size results are used for mid-frequency regions. The final denoised image is obtained through weighted fusion.

[0155] The multi-step long-region-aware fusion strategy used in noise reduction processing includes:

[0156] Parallel inference steps: Using a trained adaptive region-aware network, the input image to be denoised is processed with a set of different downsampling strides to obtain three sets of denoising results with different denoising intensities and detail preservation levels. Combined with... Figure 2 In this embodiment, step sizes 1, 2, and 4 are used.

[0157] Region segmentation steps: Calculate the local standard deviation map of the denoising result with a step size of 2, since the structure of the result is most similar to the clean image, and segment the image into three regions: low frequency, medium frequency and high frequency according to the low frequency threshold (1.5) and the high frequency threshold (5.0).

[0158] The result fusion step is as follows: Based on the results of region segmentation, the denoising results of different regions are weighted and fused. In the low-frequency region, the denoising results with a larger step size are given preference to achieve better noise suppression, while in the high-frequency region, the denoising results with a smaller step size are given preference to retain more texture details, thereby generating the final denoised image.

[0159] Specifically, using the test dataset, a multi-step region-aware fusion strategy is adopted to dynamically fuse denoising results with different downsampling steps, and the fused result is used as the final denoising result to complete the network test.

[0160] In a specific implementation, see Figure 4 As shown, different downsampling step sizes have different denoising effects on different regions of the image (larger step sizes are beneficial to flat areas, while smaller step sizes are beneficial to textured areas). Therefore, this invention employs the MPDF strategy during the testing phase to achieve optimal region-specific denoising. The specific steps are as follows:

[0161] The first step is parallel inference. Using a trained adaptive region-aware network, the noisy input image is processed... Each of them uses a different set of downsampling step sizes (In this embodiment) The data was processed to obtain three sets of denoising results. Its expression is:

[0162]

[0163] in and This is a standard pixel re-sampling and inverse operation.

[0164] The second step is region segmentation. The denoising result with a step size of 2 is selected. As a benchmark for structural analysis, its local standard deviation plot is calculated. And based on a preset low-frequency threshold... (1.5 in this embodiment) and high-frequency threshold (In this embodiment, version 5.0) The image is precisely segmented into low-frequency components ( ), intermediate frequency ( ) and high frequency ( Binary masks for the three regions.

[0165] The third step is adaptive fusion. First, intermediate fusion results are calculated for flat and textured regions. and :

[0166] ;

[0167] in and To control the hyperparameters of the fusion weights (e.g., for the SIDD dataset, , ).

[0168] Final denoised image Combine them using the following formula:

[0169]

[0170] This operation uses different fusion strategies for different regions based on the results of region segmentation to generate a high-quality final output image.

[0171] Step S5: Input the noisy image into the tested and verified adaptive region-aware blind spot network to obtain the denoised image.

[0172] This invention adaptively balances dense noise removal with texture detail preservation, eliminating spatially correlated and signal-dependent noise, and significantly improving the self-supervised denoising performance and inference efficiency of real-world images. Image denoising using this method demonstrates significantly better results than existing techniques. The denoising results of this invention not only effectively reduce noise but also eliminate problems such as checkerboard artifacts, color artifacts, or excessive texture blurring that may occur in other methods, exhibiting excellent performance in preserving the global structure and fine texture details of the image.

[0173] To better demonstrate the noise reduction effect of this invention, the following comparative experiments can be conducted for verification:

[0174] Comparative experimental scheme: On the SIDD benchmark set, the present invention is quantitatively compared with advanced denoising networks, and the indicators include peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0175] This includes comparisons with the self-supervised denoising method CVF-SID (based on self-supervised learning) and the self-supervised denoising method AP-BSN (based on asymmetric downsampling blind spot network). A higher PSNR value indicates a closer pixel-value ratio between the denoised image and the real image; a higher SSIM value indicates a greater structural similarity. Higher values ​​for both are better.

[0176] Experimental results:

[0177] Table 1 below shows the denoising performance of each method on the SIDD dataset. It is clear that the performance of this invention is significantly better than CVF-SID and AP-BSN.

[0178] Table 1 Comparison of Noise Reduction Effects

[0179]

[0180] In summary, this invention introduces a structure-aware adaptive hybrid scrambling strategy and a more thorough random geometric transformation pixel downsampling module, enabling the network to better distinguish between noise and signal during the training phase; and during the testing phase, a multi-step pixel recombination strategy is used to perform refined processing on different regions of the image, thereby achieving a better balance between thorough denoising and detail preservation, significantly improving the overall quality of image denoising.

[0181] This invention also provides a self-supervised image denoising system, the system comprising:

[0182] The dataset construction module is used to construct noisy image datasets;

[0183] A network architecture construction module is used to build an adaptive region-aware network framework, which includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch.

[0184] The network training module is used to train the adaptive hybrid scrambling branch and the non-scrambling branch in parallel to generate a trained adaptive region-aware blind spot network.

[0185] The network testing module is used to input noisy images into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, it returns to the network optimization module for further optimization.

[0186] The denoising module is used to input the noisy image into the tested and verified adaptive region-aware blind spot network to obtain the denoised image.

[0187] The present invention further discloses a self-supervised image denoising device, which integrates a processor and a memory; wherein, when the processor executes the self-supervised image denoising program stored in the memory, it can implement the steps of the region-aware driven self-supervised image denoising method as described in any of the above claims.

[0188] Furthermore, the self-supervised image denoising device in this embodiment may also include, but is not limited to:

[0189] The input interface is used to receive external data and user instructions, such as acquiring noisy images or updated program code to be processed through physical interfaces (such as USB, Ethernet interfaces) or wireless interfaces (such as Wi-Fi, Bluetooth), and transmitting the acquired information to the processor.

[0190] Output interface, used to output various data such as denoised images or processing status generated by the processor to the connected terminal device, such as a display or other storage device.

[0191] The communication unit is used to establish a remote communication connection between the device and an external server, so that the device can submit denoising results to an online evaluation server or obtain model updates from the cloud.

[0192] Human-computer interaction devices, such as keyboards, mice, and touch screens, are used to obtain user operation commands, such as selecting images to be processed and adjusting denoising parameters, and can display the denoising process and results in real time on a monitor.

[0193] The present invention also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, can implement the steps of the region-aware driven self-supervised image denoising method as described in any of the preceding claims. Specifically, it may be a random access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), optical disk, or any other non-transitory storage medium known to those skilled in the art.

[0194] In summary, this invention effectively solves the balance problem between noise removal and texture preservation in existing self-supervised methods by introducing a structure-aware adaptive hybrid scrambling strategy and a randomized pixel downsampling strategy aimed at completely destroying spatial correlations during the training phase, and combining it with a multi-step region-aware fusion strategy during the inference phase, thus significantly improving the denoising performance on noisy real-world images.

[0195] The embodiments in this specification are described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0196] Those skilled in the art will understand that the algorithm steps and functional units described herein can be implemented using electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, a general functional description of the composition and steps of each example has been provided above. Whether these functions are implemented in hardware or software depends on the specific application scenario and design constraints of the technical solution.

[0197] It should be understood that the systems, modules, and units disclosed herein are only illustrative of logical functions. In practical applications, multiple units or components may be merged, integrated into another system, or certain features may be omitted. The units may be physically separate entities or logical entities distributed across multiple network nodes; that is, they may be located in one place or distributed across multiple network units. Depending on actual needs, some or all units can be selected to achieve the purpose of this embodiment.

[0198] In addition, the functional modules in the various embodiments of the present invention can be integrated into a processing unit, or each module can exist physically separately, or two or more modules can be integrated into a unit.

[0199] The methods or algorithm steps described herein can be implemented directly by hardware, by a software module executed by a processor, or by a combination of both. The software module can be stored in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0200] Finally, it should be emphasized that the relational terms such as "first" and "second" used in this document are only used to distinguish different entities or operations and do not necessarily indicate or imply their actual order or relationship. The terms "comprising," "including," and any variations thereof are intended to cover a non-exclusive inclusion relationship, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0201] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A region-aware driven self-supervised image denoising method, characterized in that, The method includes: Step S1: Construct a noisy image dataset; Step S2: Construct an adaptive region-aware network framework, which includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch. Step S3: Train the adaptive hybrid scrambling branch and the non-scrambling branch in parallel to generate the trained adaptive region-aware blind spot network; Step S4: Input the noisy image into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, return to step S3 to continue optimization. Step S5: Input the noisy image into the tested and verified adaptive region perception blind spot network to obtain the denoised image; In step S2, the adaptive hybrid scrambling branch performs coarse scrambling on the noisy image to generate scrambled sub-images; The scrambled sub-image is processed by an initial adaptive region-aware blind spot network to obtain a coarsely estimated image; Based on the local standard deviation of the coarsely estimated image, flat regions and textured regions are distinguished. Pixel perturbation is enhanced in the flat region and pixel perturbation is reduced in the textured region to generate finely scrambled training samples; The non-scrambled branch performs random transformation pixel downsampling on the noisy image through the random geometric transformation pixel downsampling module to generate training samples enhanced by random transformation pixel downsampling. The expression for coarsely scrambling a noisy image to generate a scrambled sub-image is as follows: in, This represents the input noisy image. This represents a sliding window of size 2×2 with a step size of 2 that randomly scrambles pixels in a noisy image. This indicates pixel re-sampling with a step size of 2; The scrambled sub-image is processed by an initial adaptive region-aware blind spot network to obtain a coarsely estimated image, the expression of which is: in, Represents a scrambled sub-image. This indicates a rough estimate of the image; Based on the local standard deviation of the roughly estimated image, flat regions and textured regions are distinguished, and the expression is as follows: in, This represents an n×n window centered at pixel (i,j). This represents the standard deviation of pixel values ​​within the calculation window; Indicates local standard deviation; The pixel perturbation is enhanced in the flat region and weakened in the textured region to generate finely scrambled training samples, the expression of which is: when hour, The non-scrambled branch performs random geometric transformation pixel downsampling on the noisy image through the random geometric transformation pixel downsampling module, generating training samples enhanced by random transformation pixel downsampling, the expression of which is: when hour, in, This represents random scrambling within an n×n window, where r is a preset threshold for distinguishing between flat and textured regions. This indicates that the training samples were scrambled. This represents training samples enhanced by random pixel downsampling. The expression for the composite loss function is: in, Represents the composite loss function. This represents the adaptive hybrid scrambling branch loss. Indicates the loss of the non-disordered branch. This represents the denoised output of the adaptive hybrid scrambling branch. This represents the denoised output of the non-scrambled branch. and They represent step sizes of 1 and 2 respectively. Random transformation pixel downsampling and its inverse operation.

2. The region-aware driven self-supervised image denoising method as described in claim 1, characterized in that, The non-scrambled branch performs random transformation pixel downsampling on the noisy image through a random geometric transformation pixel downsampling module to generate training samples enhanced by random transformation pixel downsampling, specifically including: The noisy image is subjected to block-level random geometric transformation by the random geometric transformation pixel downsampling module to obtain an unrandomized sub-image; The unscrambled sub-images are input into the initial adaptive region-aware blind spot network for processing to obtain training samples with randomized pixel downsampling enhancement; The block-level random geometric transformations include: random rotation and random flipping.

3. The region-aware driven self-supervised image denoising method as described in claim 2, characterized in that, In step S3, the adaptive hybrid scrambling branch and the non-scrambling branch are trained in parallel to generate a trained adaptive region-aware blind spot network, specifically including: The finely scrambled training samples from the adaptive hybrid scrambling branch output and the randomly transformed pixel downsampling enhanced training samples from the non-scrambling branch output are combined, and the initial adaptive region-aware blind spot network is trained using a composite loss function and then trained using the Adam optimizer to obtain the trained adaptive region-aware blind spot network.

4. The region-aware driven self-supervised image denoising method as described in claim 3, characterized in that, In step S4, the noisy image is input into the trained adaptive region perception network for testing, resulting in a tested and validated adaptive region perception blind spot network, specifically including: The test dataset is input into the trained adaptive region-aware blind spot network for testing. Specifically, a multi-step long region-aware fusion strategy based on spatial distribution is used to fuse the denoising results of different long pixel reconstruction downsampling, and the final denoised image is obtained and output to complete the network test and obtain the tested and verified adaptive region-aware blind spot network.

5. The region-aware driven self-supervised image denoising method as described in claim 4, characterized in that, A spatially distributed multi-step region-aware fusion strategy is used to fuse the denoising results of pixel reconstruction downsampling with different step lengths to obtain and output the final denoised image. The specific steps include the following: Using an optimized adaptive region-aware network, the input noisy image is processed. Each of them uses a different set of downsampling step sizes To process, Three sets of denoising results were obtained. Its expression is: in, and These are standard pixel recombining downsampling and its inverse operation, respectively. Select the denoising result with a step size of 2 As a benchmark for structural analysis, its local standard deviation plot is calculated; and based on a preset low-frequency threshold... and high frequency threshold Accurately segment the image into low frequencies. , intermediate frequency and high frequency Binary masks for three regions; Calculate intermediate blending results for flat and textured regions. and : ； in and The hyperparameter dataset for controlling the fusion weights, , ; Generate the final denoised image Its expression is: 。 6. A system applied to the region-aware driven self-supervised image denoising method as described in any one of claims 1-5, characterized in that, The system includes: The dataset construction module is used to construct noisy image datasets; A network architecture construction module is used to build an adaptive region-aware network framework, which includes an initial adaptive region-aware blind spot network, a parallel adaptive hybrid scrambling branch and a non-scrambling branch, and a random geometric transformation pixel downsampling module deployed on the adaptive hybrid scrambling branch and the non-scrambling branch. The network training module is used to train the adaptive hybrid scrambling branch and the non-scrambling branch in parallel to generate a trained adaptive region-aware blind spot network. The network testing module is used to input noisy images into the trained adaptive region perception network for testing. If the test passes, the tested and verified adaptive region perception blind spot network is obtained. If the test fails, it returns to the network optimization module for further optimization. The denoising module is used to input the noisy image into the tested and verified adaptive region-aware blind spot network to obtain the denoised image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the region-aware driven self-supervised image denoising method as described in any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the region-aware driven self-supervised image denoising method as described in any one of claims 1-5.

Images