Self-supervision real image denoising method based on next scale prediction
By using a self-supervised real image denoising method based on next-scale prediction, high-resolution denoising results are predicted using low-resolution sub-images. This solves the problem that existing methods cannot simultaneously destroy noise correlation and preserve details, achieving better image denoising effects and model adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-03-27
AI Technical Summary
Existing self-supervised real image denoising methods based on blind spot networks cannot simultaneously and effectively destroy the spatial correlation of noise and preserve image details, resulting in poor denoising performance.
A self-supervised real image denoising method based on next-scale prediction is adopted. Multiple low-resolution sub-images are generated by pixel shuffling downsampling. High-resolution denoising prediction is performed using a pre-trained blind spot network. High-resolution denoised images are generated by averaging and pixel rearrangement upsampling techniques. A training sample set is constructed to update the network parameters by minimizing the loss value.
By removing spatial correlations of noise at low-resolution levels while preserving image details at high-resolution levels, better denoising results are achieved, enhancing the model's adaptability to complex real-world noise, and eliminating the need for clean image supervision.
Smart Images
Figure CN121746233A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of self-supervised real image denoising technology, specifically to a self-supervised real image denoising method based on next-scale prediction. Background Technology
[0002] Self-supervised real-image denoising is a crucial underlying vision technique, aiming to recover sharp images using only real-world noisy images without relying on any clean images for supervision. In recent years, some self-supervised image denoising methods based on blind-spot networks (BSNs) have achieved excellent denoising results on synthetically noisy data. Generally, BSNs use neighboring noisy pixels to predict the denoised version of the center pixel, achieving self-supervised denoising. This approach assumes that the noise has no spatial correlation; otherwise, BSNs will inevitably learn the spatial correlation of the noise, thus predicting the noise at the center pixel, leading to denoising failure. However, in real-world scenes, image noise is generated by the complex imaging mechanisms within the camera and often exhibits strong spatial correlation, failing to satisfy the assumptions of BSNs.
[0003] To apply the BSN method to real-world image denoising, existing techniques generally involve transforming real-world noise into approximately pixel-level independent noise before training the BSN to denoise the noisy image. For example, existing techniques such as "Self-supervised denoising for real-world images via asymmetric pd and blind-spot network," "Random sub-samples generation for self-supervised real image denoising," and "Rethinking transformer-based blind-spot network for self-supervised image denoising" all use pixel-shuffle downsampling (PD) to downsample the noisy image into multiple low-resolution sub-images, while transforming spatially correlated real-world noise into approximately pixel-level independent noise in the sub-images, and then training the BSN on the sub-images.
[0004] Despite the promising results achieved by these methods, they all face a dilemma. Specifically, to fully break down the spatial correlation of real noise and ensure effective training of the BSN, these methods must use a sufficiently large PD factor. However, a large PD factor severely damages the detail information of the original image, causing the BSN to learn only on low-resolution sub-images, severely limiting its ability to recover image details. To alleviate the detail destruction problem, existing methods can only reduce the PD factor. But this introduces a more serious problem. Reducing the PD factor prevents the spatial correlation of real noise from being fully broken down; that is, the noise in the sub-image still has some degree of spatial correlation, which does not conform to the assumption of the BSN, ultimately leading to the ineffective training of the BSN.
[0005] In summary, the shortcomings of existing BSN-based self-supervised real image denoising methods can be summarized as follows: while the applied dynamic range (PD) sufficiently destroys the spatial correlation of real noise, it also severely damages image details, failing to simultaneously achieve the dual objectives of destroying noise correlation and preserving details. Therefore, decoupling these two contradictory objectives, allowing BSN to learn more high-resolution details from low-resolution sub-images where noise has been sufficiently decorrelated—that is, based on next-scale prediction—is a pressing problem to be solved in the current field of BSN-based self-supervised real image denoising. Summary of the Invention
[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a self-supervised real image denoising method based on next-scale prediction, which solves the problem that existing BSN-based self-supervised real image denoising methods cannot simultaneously achieve the dual objectives of destroying noise correlation and preserving details.
[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A self-supervised real image denoising method based on next-scale prediction is provided, including the following steps: S1. Obtain a noisy image contaminated by real noise, and obtain multiple sub-images by pixel shuffling downsampling; S2. Input all sub-images into the pre-trained blind spot network. The blind spot network performs the next scale prediction for each sub-image and outputs the high-resolution denoising result for each sub-image. The resolution of each high-resolution denoising result is the same as the resolution of the noisy image. S3. Average all the high-resolution denoising results to obtain the denoised image of the noisy image.
[0008] Further methods for pre-training blind spot networks include: S21. Obtain at least one training noise image contaminated by real noise, and construct a training sample set for each training noise image. Each training sample pair in the training sample set includes a low-resolution training sub-image and a high-resolution real label corresponding to the low-resolution training sub-image. S22. Input a low-resolution training sub-image from all training sample sets into the blind spot network. The blind spot network performs the next-scale prediction on the low-resolution training sub-image and outputs the predicted label corresponding to the low-resolution training sub-image. The resolution of the predicted label is the same as the resolution of the true label. S23. Calculate the loss value between the predicted label and the real label corresponding to the low-resolution training sub-image, and update the network parameters in the blind spot network with the goal of minimizing the loss value. S24. Repeat steps S22 to S23 until the optimization objective converges or all training sample sets are input, thus completing the training of the blind spot network.
[0009] Furthermore, the method for constructing the training sample set is as follows: S211. Divide a training noisy image into multiple non-overlapping local blocks, and select some pixels in each local block as at least one true label block. S212. Assign the remaining pixels in each local block to a sub-image container with the same number of remaining pixels, and then concatenate the remaining pixels at the same position in all sub-image containers to obtain a low-resolution training sub-image with the same number of remaining pixels as the local block. S213. Concatenate the real label blocks located in the same pixel region in all local blocks to obtain at least one real label, and pair all low-resolution training sub-images with all real labels to obtain a training sample set.
[0010] Furthermore, the selection strategy for choosing a subset of pixels as the real label block within each local block includes: randomly selecting pixels, randomly selecting pixels and then sorting them according to their row index, and randomly selecting... OK The pixels at the intersection of columns, or randomly selecting a continuous block. pixel area, where This is the upsampling factor.
[0011] Furthermore, the expression for the training sample set is: ,in: ; ; ; ; in, For the first A low-resolution training sub-image; The range is ; and These represent the number of pixels in the local block and the pixel side length of the local block, respectively. The number of actual tag blocks; This represents the upsampling factor for the blind spot network. For the first And and The corresponding real tags; The range is ; for The first in One remaining pixel; For the first row and number Local blocks of a column; and These are the pixel height and pixel width of the noisy image, respectively; For splicing operations; for The Middle One real label block; For allocation operations; for Remove from The remaining pixels after the first pixel; A collection of real tag blocks; To train noisy images; For partitioning operations.
[0012] Furthermore, the upsampling factor of the blind spot network Greater than 1 and less than .
[0013] Furthermore, the self-supervised real image denoising method based on next-scale prediction also includes acquiring a super-resolution image of the noisy image. : in, For pixel rearrangement upsampling operation; The first pixel in the shuffling downsampling A low-resolution sub-image; This is the sampling factor for pixel rearrangement upsampling.
[0014] Compared with the prior art, the present invention has the following significant advantages: 1. This invention uses a low-resolution sub-image to predict its high-resolution denoising result, thereby effectively removing spatial correlation of noise at the low-resolution level while better preserving image details at the high-resolution level. By decoupling "removing spatial correlation of noise" and "preserving image details," it effectively alleviates the dilemma faced by existing paradigms of joint pixel shuffling downsampling (PD) and blind spot networks (BSN), and solves the contradiction that existing methods cannot simultaneously destroy noise correlation and preserve details.
[0015] 2. This invention uses only real noisy images to construct training sample pairs, without requiring any clean images as supervision signals, thus achieving fully self-supervised training. At the same time, by iteratively updating the network parameters by minimizing the loss value between the predicted label and the real label, the blind spot network can learn the mapping relationship between noise statistics and image details, enhancing the model's adaptability to complex real noise.
[0016] 3. The local block of this invention uses a portion of pixels that retain spatial relationships to form a high-resolution true label as the learning target of BSN, so that BSN can learn more high-resolution image information and retain more image details in the denoising process. At the same time, the remaining pixels of the local block are allocated to the sub-image container, which implements indexing for the remaining pixels of the local block. This makes it easy to stitch together low-resolution sub-images from non-adjacent pixels of different local blocks to form a low-resolution grid as input to BSN, which greatly destroys the spatial correlation of the original noise and makes it satisfy the training assumption of BSN.
[0017] 4. This invention provides multiple label selection strategies, with continuous label selection being preferred. Pixel regions serve as the real label blocks; that is, the real labels are selected from each local block. The image is composed of consecutive pixel regions stitched together, preserving the normal spatial relationships and details between pixels within these regions. The BSN is configured to predict this high-resolution prediction label from the low-resolution subimage, thus enabling the BSN to learn how to recover and generate details.
[0018] 5. This invention can not only achieve the following through the application of... The high-resolution denoising results of each low-resolution sub-image are averaged to obtain a denoised image (clean image) with the same resolution as the noisy image. At the same time, the pixel-shuffle upsampling technique is used to fuse multiple high-resolution denoising results into a final image with a higher resolution, thus achieving super-resolution. Attached Figure Description
[0019] Figure 1 This is a flowchart of a self-supervised real image denoising method based on next-scale prediction.
[0020] Figure 2 This diagram illustrates four selection strategies for real label blocks. Detailed Implementation
[0021] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0022] Example 1 refer to Figure 1 This embodiment provides a self-supervised real image denoising method based on next-scale prediction, wherein, Figure 1 Figure (a) shows the training process. Figure (b) shows the process of constructing training sample pairs. Figure (c) shows the modifications to the BSN architecture, only changing the tail part after the backbone network. Figure (d) shows the testing process. In the last step of the testing process, the denoising task uses an averaging operation. The noisy image super-resolution task uses pixel reordering upsampling operation .
[0023] Specifically, the self-supervised real image denoising method based on next-scale prediction includes the following steps: S1. Obtain a noisy image contaminated with real noise, and obtain it through pixel shuffling downsampling (PD). Each sub-image has a low resolution, and the resolution of each sub-image is equal to the resolution of the noisy image. The downsampling factor of PD and the upsampling factor of Blind Spot Network (BSN) are both 1. .
[0024] The expression for the low-resolution subgraph is: in, Noisy image; , and These represent the color channels, pixel height, and pixel width of the noisy image, respectively. For PD operation, It is the first one generated by PD A low-resolution sub-image.
[0025] S2. Input all sub-images into the pre-trained blind spot network. The blind spot network performs the next scale prediction for each sub-image and outputs a high-resolution denoising result for each sub-image. The resolution of each high-resolution denoising result is the same as the resolution of the noisy image.
[0026] The expression for the high-resolution denoising result is: in, yes High-resolution denoising results; This is the BSN network.
[0027] S3. Average all high-resolution denoising results to obtain the denoised image of the noisy image. The expression for the denoised image is: in, For denoised images, This is for averaging operations.
[0028] As a further embodiment, the self-supervised real image denoising method based on next-scale prediction also includes acquiring a super-resolution image of the noisy image. : in, For pixel rearrangement upsampling operation; The first pixel in the shuffling downsampling A low-resolution sub-image; This is the sampling factor for pixel rearrangement upsampling.
[0029] In this implementation, the BSN network structure is an improvement upon the SDAP network structure in the existing technology "Random sub-samples generation for self-supervised real image denoising," with only the tail portion of the network modified. Specifically, the dimension of the input data in the tail portion is... ,in For batch size, Let be the dimension of the latent space. The size of the feature map.
[0030] Specifically, the tail portion of the SDAP network is: Layer 1: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0. Layer 2: GELU layer; Layer 3: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0. Layer 4: GELU layer; Layer 5: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0. Layer 6: GELU layer; Layer 7: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0. This embodiment adds a pixel shuffle operation to the tail part of SDAP to upsample the feature map, thereby enabling BSN to output a higher resolution denoised image for loss calculation with the higher resolution label. Specifically, this embodiment modifies the tail part as follows: Layer 1: Rollup layer, input channel dimension is The output channel dimension is 2. The kernel size is 1, the stride is 1, and the padding is 0. Layer 2: GELU layer; Layer 3: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0. Layer 4: GELU layer; Layer 5: Pixel Shuffle layer, which will... Feature maps transformed Higher resolution feature maps; Layer 6: Conveyor Layer, input channel dimension is The output channel dimension is 2. The kernel size is 1, the stride is 1, and the padding is 0. Layer 7: GELU layer; Layer 8: Rollup layer, input channel dimension is The output channel dimension is 2. The kernel size is 1, the stride is 1, and the padding is 0. Layer 9: GELU layer; Layer 10: Rollup layer, input channel dimension is The output channel dimension is The kernel size is 1, the stride is 1, and the padding is 0.
[0031] As a further embodiment, the method for pre-training a blind spot network includes: S21. Obtain at least one training noise image contaminated by real noise, and construct a training sample set for each training noise image. Each training sample pair in the training sample set includes a low-resolution training sub-image and the real label corresponding to the low-resolution training sub-image.
[0032] S22. Input a low-resolution training sub-image from all training sample sets into the blind spot network. The blind spot network performs the next-scale prediction on the low-resolution training sub-image and outputs the predicted label corresponding to the low-resolution training sub-image. The resolution of the predicted label is the same as the resolution of the true label.
[0033] S23. Calculate the loss value between the predicted label and the real label corresponding to the low-resolution training sub-image, and update the network parameters in the blind spot network with the goal of minimizing the loss value.
[0034] S24. Repeat steps S22 to S23 until the optimization objective converges or all training sample sets are input, thus completing the training of the blind spot network.
[0035] The self-supervised loop expression for the pre-trained blind spot network is: in, To train noisy images, , and These represent the color channels, pixel height, and pixel width of the noisy image, respectively. The training sample set construction strategy designed for this method will be introduced later; It is a multiple of PD; It is the upsampling factor. It is a low-resolution sub-image output by PD; yes The corresponding real tags; It is BSN; It is an optimization parameter set; yes Predicted labels; It is the L1 norm of the matrix. In this embodiment, the upsampling factor of the blind spot network. Greater than 1 Less than This setup can more effectively break down the spatial correlation of noise because each local block is larger, resulting in greater pixel spatial distances in the rearranged subimages. A smaller block is used during the testing phase. Downsampling can produce fewer sub-images while ensuring that the image size of the BSN output is the same as the original image.
[0036] Specifically, the method for constructing the training sample set is as follows: S211. Divide a training noisy image into multiple non-overlapping local blocks, and select some pixels in each local block as at least one true label block.
[0037] S212. Assign the remaining pixels in each local block to a sub-image container with the same number of remaining pixels, and then concatenate the remaining pixels at the same position in all sub-image containers to obtain a low-resolution training sub-image with the same number of remaining pixels as the local block.
[0038] S213. Concatenate the real label blocks located in the same pixel region in all local blocks to obtain at least one real label, and pair all low-resolution training sub-images with all real labels to obtain a training sample set.
[0039] refer to Figure 2 The selection strategies for choosing a subset of pixels as the real label block within each local block include: randomly selecting pixels, randomly selecting pixels and then sorting them according to their row index (row-first sorting after random selection), and randomly selecting... OK The pixels at the intersection of columns, or randomly selecting a continuous block. pixel area, where This is the upsampling factor. In this embodiment, it is preferable to randomly select a continuous block. The pixel region, i.e., the actual label block, has the following number of pixels. The number of real labels is set to The number of pixels in the local block is set to Therefore, the number of remaining pixels in the local block is Therefore, the expression for the training sample set is: ,in: ; ; ; ; in, For the first A low-resolution training sub-image; The range is ; and These represent the number of pixels in the local block and the pixel side length of the local block, respectively. The number of actual tag blocks; This represents the upsampling factor for the blind spot network. For the first And and The corresponding real tags; The range is ; for The first in One remaining pixel; For the first row and number Local blocks of a column; and These are the pixel height and pixel width of the noisy image, respectively; For splicing operations; for The Middle One real label block; For allocation operations; for Remove from The remaining pixels after the first pixel; A collection of real tag blocks; To train noisy images; For partitioning operations.
[0040] In summary, this scheme uses a low-resolution sub-image to predict its high-resolution denoising result, thereby effectively removing spatial correlations of noise at the low-resolution level while better preserving image details at the high-resolution level. By decoupling "removing spatial correlations of noise" and "preserving image details," it effectively alleviates the dilemma faced by existing paradigms of joint pixel shuffling downsampling (PD) and blind spot networks (BSN), resolving the contradiction that traditional methods cannot simultaneously destroy noise correlations and preserve details.
[0041] Example 2 This embodiment is based on Embodiment 1 with further limitations. The specific improvement is that the parameters of the self-supervised real image denoising method based on next-scale prediction (hereinafter referred to as NSP) are described in detail, and real noise denoising experiments are carried out to verify the effectiveness of the NSP paradigm. Other parts not mentioned refer to Embodiment 1 or the prior art.
[0042] This method selects two representative BSNs for the NSP paradigm: DBSN based on a convolutional neural network architecture and TBSN based on a Transformer architecture. This method only modifies the tail portion of DBSN and TBSN to obtain BSNs adapted to the NSP paradigm, denoted as NSP(DBSN) and NSP(TBSN).
[0043] DBSN and TBSN respectively refer to the existing technology (Wu 2025: 4788-4796.).
[0044] Network parameters: Both input and output spatial dimensions are 1. That is, both the input and output are RGB color images; the latent spatial dimension is Upsampling factor .
[0045] Training details: The batch size during training is 16, and the patch size is... PD multiple Training uses the Adam optimizer, with parameters... , The learning rate is constant. The training consisted of 750 rounds, with 400 iterations per round. Data augmentation was also used during training. Specifically, for each input subgraph... All The probability is flipped horizontally. The probability is vertically flipped. The probability is rotated 90° counterclockwise.
[0046] To verify the effectiveness of the invented NSP paradigm, this method was tested on real-world image denoising datasets using SIDD Validation, SIDD Benchmark, and DND datasets, and compared with 20 state-of-the-art denoising methods to highlight the superiority of the NSP paradigm. This method uses two authoritative quantitative evaluation metrics in the field of image denoising, PSNR and SSIM, to evaluate the denoising performance of various methods. Higher values for PSNR and SSIM indicate better denoising performance.
[0047] Experiment 1 This method validates the effectiveness of the proposed NSP paradigm on real-world image denoising tasks. The method uses 320 noisy images from SIDDMedium as the training set, and SIDD Validation, SIDD Benchmark, and DND as the test set. SIDD Validation and SIDD Benchmark each contain 1280 images. RGB true noise images of pixels and their clean versions; DND contains 1000 images. The RGB images of pixels with actual noise and their clean versions are shown. The results on SIDD Benchmark and DND require uploading the denoised images to a designated website for evaluation. The experimental results are shown in Table 1. Table 1 - Results of Experiment 1 In the "Parameter Quantity" column of Table 1, the suffix "M" indicates "million". In the "SIDD Benchmark" column of this table, data marked with an asterisk (*) indicates that the result was obtained from the old SIDD Benchmark website. Currently, the old website is no longer accessible.
[0048] As shown in Table 1, the invented NSP(DBSN) and NSP(TBSN) achieved the best denoising performance among self-supervised methods, with only a slightly lower PSNR than TBSN on the DND dataset. Furthermore, NSP(DBSN) and NSP(TBSN) outperformed some supervised methods, such as DnCNN, TNRD, CBDNet, and PDD. It is noteworthy that NSP(DBSN) and SDAP use the same backbone network DBSN and have similar parameter counts, while NSP(DBSN) has a slightly higher parameter count due to modifications to the tail portion. Nevertheless, NSP(DBSN) significantly outperforms SDAP in denoising. This verifies that the invented NSP paradigm enables BSN to better predict high-resolution details. Similarly, both NSP(TBSN) and TBSN use TBSN as their backbone network and have similar parameter counts. The comparison between the two presents the same conclusion.
[0049] [Experiment 2] This method also validates the effectiveness of the NSP paradigm on the noisy image super-resolution task. This task requires removing real noise from low-resolution images and simultaneously super-resolution them to obtain high-resolution denoised images. Our method first downsamples the SIDD Medium and SIDD Validation datasets by a factor of 2. Then, we train NSP(DBSN) on the downsampled SIDD Medium dataset and test the super-resolution performance on the downsampled SIDD Validation dataset. (The only difference is changing the patch size...) Apart from that, all training settings were identical to those in Experiment 1. Given that the invented NSP is a self-supervised learning paradigm, the comparison method should also be a self-supervised image super-resolution method. Therefore, this method selects four representative zero-shot super-resolution methods: ZSSR, RZSR, MZSR, and dualSR.
[0050] ZSSR, RZSR, MZSR and dualSR respectively refer to existing technologies (Shocher A, Cohen N, Irani M. "zero-shot" super-resolution using deep internal learning[C] / / In Proceedings of the IEEE conference on computer vision and pattern recognition. 2018:3118-3126.), (Yoo J, Kim D, Lu Y, et al. RZSR: Reference-based zero-shot super-resolution with depth guided self-exemplars[J] IEEE Transactions onMultimedia, 25, 5972-5983.), (Soh J, Cho S, Cho N. Meta-transfer learning for zero-shot super-resolution[C] / / In Proceedings of the IEEE / CVF conference oncomputer vision and pattern recognition. 2020: 3516-3525.) and (Emad M, PeemenM, Corporaal H. Dualsr: Zero-shot dual learning for real-world super-resolution. In Proceedings of the IEEE / CVF winter conference on applications of computer vision. 2021: 1630-1639.).
[0051] Since these methods only have super-resolution capabilities and not denoising capabilities, for fair comparison, this method introduces the denoising method SDAP and adopts a two-stage strategy of "denoising first, then super-resolution" to complete the super-resolution task for noisy images. Specifically, the denoising method SDAP is trained on the downsampled SIDD Medium dataset, and the trained SDAP is used to denoise the downsampled SIDD Validation dataset. Finally, zero-shot super-resolution methods are run on each image in the downsampled and denoised SIDD Validation dataset to obtain the super-resolution results. In addition, this method also attempts to rewrite four zero-shot super-resolution methods into versions trained on the entire dataset. The rewritten methods are indicated by the prefix "U-". For example, the U-ZSSR method is trained on the downsampled and denoised SIDD Medium dataset, and then tested on the downsampled and denoised SIDD Validation dataset. The results of Experiment 2 are shown in Table 2.
[0052] Table 2 - Results of Experiment 2 Among them, RZSR and MZSR cannot be rewritten to be trained on the entire dataset due to their own mechanisms; the suffix "M" in the parameter count column means "million".
[0053] In the first four rows of Table 2, the four zero-shot super-resolution methods perform poorly because they only have super-resolution capabilities and lack denoising capabilities. However, the noisy image super-resolution task requires both denoising and super-resolution. This functional incompleteness leads to the poor performance of zero-shot super-resolution methods. The next four rows introduce the "denoise first, then super-resolution" two-stage method using SDAP. The introduction of SDAP significantly improves the performance. The following four rows represent versions of the zero-shot super-resolution methods rewritten for training on the entire dataset. Due to the unique mechanisms of some methods, they can only be trained on a single low-resolution image and cannot be rewritten. The invented NSP paradigm achieves optimal performance in the noisy image super-resolution task, achieving better results than two-stage methods such as SDAP+U-ZSSR with a smaller number of parameters.
[0054] Experiment 3 During the training phase, the NSP paradigm requires constructing training sample pairs. This method designs four label selection strategies, including 1) random selection, 2) random selection followed by row-priority sorting, 3) random selection... OK 4) Randomly select a continuous block of pixels at the intersection of columns. Region. To verify the impact of different strategies on the denoising effect, this method tests four strategies one by one. This experiment uses NSP (DBSN) as the testing method, with training settings identical to Experiment 1, and only SIDDValidation is selected as the test dataset. Simultaneously, this experiment tests different numbers of labels. The impact on noise reduction effect. The results of this experiment are shown in Table 3.
[0055] Table 3 - Results of Experiment 3 As shown in Table 3, regardless of Regarding the selection of values, strategies 3) and 4) are significantly more effective than strategies 1) and 2). This is because strategies 3) and 4) better preserve the selected values. The relative positions of pixels are considered, but strategies 1) and 2) will cause varying degrees of damage to these relative positions. Since strategy 4) achieves the best noise reduction effect, this method selects strategy 4) as the default label selection strategy. Furthermore, regardless of the strategy chosen for label construction, the optimal effect always appears in... This means that when there are a large number of training sample pairs, it indicates that, to some extent, more training sample pairs contribute to the denoising effect of the NSP paradigm.
Claims
1. A self-supervised real image denoising method based on next-scale prediction, characterized in that, Including the following steps: S1. Obtain a noisy image contaminated by real noise, and obtain multiple sub-images by pixel shuffling downsampling; S2. Input all sub-images into the pre-trained blind spot network. The blind spot network performs the next scale prediction for each sub-image and outputs a high-resolution denoising result for each sub-image. The resolution of each high-resolution denoising result is the same as the resolution of the noisy image. S3. Average all the high-resolution denoising results to obtain the denoised image of the noisy image.
2. The self-supervised real image denoising method based on next-scale prediction according to claim 1, characterized in that, Methods for pre-training blind spot networks include: S21. Obtain at least one training noise image contaminated by real noise, and construct a training sample set for each training noise image. Each training sample pair in the training sample set includes a low-resolution training sub-image and the real label corresponding to the low-resolution training sub-image. S22. Input a low-resolution training sub-image from all training sample sets into the blind spot network. The blind spot network performs the next scale prediction on the low-resolution training sub-image and outputs the predicted label corresponding to the low-resolution training sub-image. S23. Calculate the loss value between the predicted label and the real label corresponding to the low-resolution training sub-image, and update the network parameters in the blind spot network with the goal of minimizing the loss value. S24. Repeat steps S22 to S23 until the optimization objective converges or all training sample sets are input, thus completing the training of the blind spot network.
3. The self-supervised real image denoising method based on next-scale prediction according to claim 2, characterized in that, The method for constructing the training sample set is as follows: S211. Divide a training noisy image into multiple non-overlapping local blocks, and select some pixels in each local block as at least one true label block. S212. Assign the remaining pixels in each local block to a sub-image container with the same number of remaining pixels, and then concatenate the remaining pixels at the same position in all sub-image containers to obtain a low-resolution training sub-image with the same number of remaining pixels as the local block. S213. Concatenate the real label blocks located in the same pixel region in all local blocks to obtain at least one real label, and pair all low-resolution training sub-images with all real labels to obtain a training sample set.
4. The self-supervised real image denoising method based on next-scale prediction according to claim 3, characterized in that, The selection strategies for choosing a subset of pixels as the real label block within each local block include: randomly selecting pixels, randomly selecting pixels and then sorting them according to their row index, and randomly selecting... OK The pixels at the intersection of columns, or randomly selecting a continuous block. pixel region, where This is the upsampling factor.
5. The self-supervised real image denoising method based on next-scale prediction according to claim 4, characterized in that, The expression for the training sample set is: ,in: ； ； ； ； in, For the first A low-resolution training sub-image; The range is ; and These represent the number of pixels in the local block and the pixel side length of the local block, respectively. The number of actual tag blocks; This represents the upsampling factor for the blind spot network. For the first And and The corresponding real tags; The range is ; for The first in One remaining pixel; For the first row and number Local blocks of a column; and These are the pixel height and pixel width of the noisy image, respectively; For splicing operations; for The Middle One real label block; For allocation operations; for Remove from The remaining pixels after the first pixel; A collection of real tag blocks; To train noisy images; For partitioning operations.
6. The self-supervised real image denoising method based on next-scale prediction according to claim 5, characterized in that, Upsampling factor of blind spot network Greater than 1 and less than .
7. The self-supervised real image denoising method based on next-scale prediction according to claim 1, characterized in that, It also includes acquiring super-resolution images of noisy images. : in, For pixel rearrangement upsampling operation; The first pixel in the shuffling downsampling A low-resolution sub-image; This is the sampling factor for pixel rearrangement upsampling.