Self-supervised image denoising method based on adaptive region division according to noise density

Abstract

This invention discloses a self-supervised image denoising method based on adaptive noise density region segmentation, comprising the following steps: performing local statistical analysis on the input noisy image to generate a noise density response map; adaptively segmenting the noisy image based on the noise density response map to generate a mask that distinguishes between high and low noise regions; obtaining corresponding region masks by using different mask rate sampling strategies for high and low noise regions, obtaining a training mask through element-wise combination, and performing blind spot processing on the occlusion positions of the training mask; constructing a denoising network architecture, using a hierarchical encoder-decoder structure as the backbone, with each input channel performing initial feature extraction through convolution operations, mapping noise density information into high-dimensional features and fusing them with image features; performing self-supervised network training on the denoising network architecture based on the training mask, and outputting a denoised image based on the trained denoising network. This invention achieves a balance between efficient noise reduction and detail preservation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a self-supervised image denoising method based on noise density adaptive region segmentation. Background Technology

[0002] Most existing image denoising methods rely on a uniform noise detection and processing strategy, which has significant limitations in real-world noisy scenarios. Noise in real images often exhibits a spatially non-uniform distribution, making it difficult for uniform processing methods to effectively adjust the noise intensity in different regions, thus affecting the overall denoising effect.

[0003] Traditional self-supervised methods typically employ fixed sampling strategies, which have limited generalization ability when faced with different types or intensities of noise. This can easily lead to training instability and degraded noise reduction performance, and they struggle to handle diverse noise scenarios. During noise suppression, local structural information and image details are easily blurred or lost, causing structural degradation and reducing the visual quality and information integrity of the image. Particularly in biomedical imaging such as single-photon calcium imaging, due to low signal intensity and limited photon counts, images are often severely contaminated by Poisson and Gaussian noise, with extremely uneven noise distribution. Traditional methods struggle to balance the denoising needs of different regions, easily leading to loss of detail or residual noise.

[0004] Most current methods lack adaptive perception of noise spatial distribution and fail to adequately address non-uniformity. The fixed sampling strategies in self-supervised methods have limited generalization ability, easily leading to underfitting or overfitting in scenarios with drastic noise density changes. Balancing detail preservation and noise suppression during denoising is difficult, often resulting in over-smoothing or texture destruction. Furthermore, efficient and optimized solutions are still lacking for special scenarios with extremely low signal-to-noise ratios and highly non-uniform noise distributions, such as single-photon calcium imaging. Therefore, a denoising technique that can adaptively perceive noise distribution and differentiate region processing is needed to maximize the preservation of image details and structural information while suppressing noise. Summary of the Invention

[0005] To overcome the defects and shortcomings of existing technologies, this invention provides a self-supervised image denoising method based on adaptive region segmentation of noise density. This method addresses the problems of uneven noise distribution, limited generalization ability of fixed sampling strategies, and easy degradation of image details in existing technologies. By introducing noise density as prior information, this invention performs adaptive region segmentation of the image, thereby enabling the adoption of differentiated processing strategies based on the noise characteristics of different regions, achieving a balance between efficient noise reduction and detail preservation.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides a self-supervised image denoising method based on noise density adaptive region segmentation, comprising the following steps:

[0008] Perform local statistical analysis on the input noisy image to generate a noise density response map;

[0009] Adaptive region segmentation of the noisy image is performed based on the noise density response map to generate a mask that distinguishes between high and low noise regions.

[0010] Different sampling strategies with different masking rates are used to obtain the corresponding region masks for high and low noise regions. The training mask is obtained by combining elements one by one, and blind spot processing is performed on the occlusion positions of the training mask.

[0011] A denoising network architecture is constructed, using a hierarchical encoder-decoder structure as the backbone. Each input channel completes the initial feature extraction through convolution operations, mapping the noise density information into high-dimensional features and fusing them with image features.

[0012] The denoising network architecture is trained using a training mask, and the denoised image is output based on the trained denoising network.

[0013] As a preferred technical solution, local statistical analysis is performed on the input noisy image to generate a noise density response map, specifically including:

[0014] Obtain the noisy image to be processed and perform normalization on the image;

[0015] A local neighborhood window of a preset size is constructed with each pixel of the normalized image as the center.

[0016] The local mean, local variance, and local standard deviation are calculated based on the local neighborhood window, and the local standard deviation is used as the noise response value corresponding to each pixel position.

[0017] The noise response values ​​corresponding to each pixel position are combined to form a noise density response map.

[0018] As a preferred technical solution, adaptive region segmentation of the noisy image is performed based on the noise density response map, specifically including:

[0019] A data-driven approach is used to determine the threshold, and the noise density response is based on the threshold. Figure 2 Values ​​are used to obtain a region mask, which is used to divide high-noise regions into low-noise regions.

[0020] As a preferred technical solution, sampling strategies with different mask rates are used for high and low noise regions to obtain corresponding region masks, and training masks are obtained through element-wise combination, specifically including:

[0021] For high-noise regions, a dense sampling strategy with a low mask rate is adopted. By setting the mask rate of high-noise regions, each pixel in the region is randomly masked with the high-noise region mask rate to generate a high-noise region mask.

[0022] For low-noise regions, a sparse sampling strategy with a high masking rate is adopted. By setting the masking rate of the low-noise region, each pixel in the region is randomly masked with the masking rate of the low-noise region to generate a low-noise region mask.

[0023] The training mask is obtained by combining elements one by one, specifically represented as follows:

[0024] ;

[0025] in, Indicates the training mask. Represents a mask for high-noise regions. Represents a low-noise region mask. Represents a region mask, with high-noise regions corresponding to... Low noise region corresponds , ⊙ represents element-wise multiplication.

[0026] As a preferred technical solution, blind spot processing is performed on the occlusion positions of the training mask, specifically including:

[0027] In the training mask, a morphological dilation operation is performed on all occluded pixels, so that pixels in their neighborhood are also occluded.

[0028] As a preferred technical solution, the denoising network architecture includes an encoder and a decoder;

[0029] The encoder consists of multiple cascaded coding blocks. Each coding block contains a noise density-aware window self-attention module and a feedforward network structure. The spatial resolution of the feature map is gradually reduced between adjacent coding blocks through downsampling operations. The bottleneck layer located between the encoder and the decoder is used for global feature modeling.

[0030] The decoder is symmetrical to the encoder. It gradually restores the spatial resolution through upsampling operations and has skip connections corresponding to the encoder layers. The skip connections are implemented by feature concatenation and channel compression and fusion are performed through convolution operations. The decoder output is convolved and mapped back to the image space to obtain a denoised image with the same number of input channels.

[0031] As a preferred technical solution, in the window self-attention module, the input feature map is divided into multiple non-overlapping local windows, and self-attention is calculated independently in each window. After generating query, key, and value vectors through linear transformation, noise density features are added to the modulation process of the key vector to achieve attention weight allocation for noise perception. Each window self-attention module is followed by a feedforward network structure.

[0032] As a preferred technical solution, the feedforward network structure expands the channel dimension through linear mapping, combines depthwise separable convolution to achieve local spatial modeling, and restores the number of channels to the original dimension after activation function processing. The feedforward network structure uses a gating mechanism to achieve feature modulation.

[0033] The present invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements the self-supervised image denoising method based on adaptive region partitioning of noise density as described above.

[0034] The present invention also provides a computer device, including a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the self-supervised image denoising method based on noise density adaptive region partitioning as described above.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] (1) The present invention extracts noise information from the input noisy image, generates a pixel-level noise response map through local statistical characteristics and residual analysis, and divides the image into regions based on the response map. This region division scheme does not rely on a fixed threshold or manual prior, but is dynamically generated based on the noise statistical characteristics of the image itself, thereby significantly enhancing the generalization ability of the model in various real noise scenarios.

[0037] This invention achieves adaptive noise density region segmentation, enhancing the ability to perceive real noise distribution. Combined with self-supervised learning and dynamic feature interaction mechanisms, it improves noise reduction performance and detail preservation. It does not require clean images as supervision and can be directly applied to real noisy image scenarios, exhibiting good generalization and practicality.

[0038] (2) This invention constructs a self-supervised denoising framework, which combines an adaptive masking strategy with a lightweight Transformer. By using a windowed attention mechanism, it models local spatial dependencies while ensuring computational efficiency. At the same time, it introduces noise information into the attention calculation process and dynamically modulates the intensity of feature interactions, enabling the network to exhibit differentiated response capabilities under different noise conditions, thus effectively improving denoising performance.

[0039] (3) This invention uses a dynamic feature interaction and sampling mechanism to adaptively adjust the feature interaction and sampling strategy according to the regional noise density. For noise-dominated regions, it enhances the utilization of contextual information, thereby effectively alleviating structural degradation and detail blurring while suppressing noise. The entire training process does not require clean images as supervision signals and can be directly applied to self-supervised or unsupervised learning scenarios of real noisy images to achieve end-to-end image denoising. It can achieve efficient noise reduction in noisy image scenarios while preserving the details and structural information of the image to the greatest extent. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the self-supervised image denoising method based on noise density adaptive region segmentation of the present invention.

[0041] Figure 2 This is a schematic diagram illustrating the generation of the noise density response map of the present invention;

[0042] Figure 3 This is a schematic diagram of the adaptive region division result of the present invention;

[0043] Figure 4(a) is a schematic diagram of the differential mask generation process of the present invention.

[0044] Figure 4(b) is a schematic diagram of the denoising network architecture combining noise density perception of the present invention;

[0045] Figure 5 This is a schematic diagram of the self-supervised noise reduction training process of the present invention;

[0046] Figure 6 This is a comparison chart showing the effects of different noise reduction methods. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Example 1

[0049] like Figure 1As shown, this embodiment provides a self-supervised image denoising method based on adaptive noise density region partitioning, which solves the problems of uneven noise distribution, insufficient generalization ability of fixed sampling strategies, and easy degradation of structural details in single-photon calcium imaging images. This method uses noise density as a priori, and differentiates regions with different noise intensities and structural complexities through an adaptive region-aware mechanism to suppress noise and improve image detail preservation. The input noisy image undergoes local statistical analysis and smoothing residual calculation to generate a pixel-level noise response map characterizing the noise distribution. The image is adaptively divided into low and high signal-to-noise ratio (SNR) regions, with different sampling strategies employed for each region. Low SNR regions undergo local fine sampling, while high SNR regions undergo global sparse sampling. A noise density-aware feature interaction mechanism is introduced to dynamically adjust information fusion between regions. Contextual redundancy is enhanced in noise-dominant regions. The network architecture is based on a self-supervised denoising framework, combining a lightweight Transformer and adaptive masking strategy for efficient feature modeling and region differentiation processing. Finally, the outputs of each region are fused to generate a high-quality denoised image. This embodiment can be trained with a single set of noise observations, improving the generalization ability in complex real-world noise scenes and balancing noise suppression with the preservation of key structural details. Specifically, it includes the following steps:

[0050] S1: To achieve adaptive noise sampling and partitioning of non-uniform noise images, local statistical analysis is first performed on the input noise image to generate a noise density response map, which characterizes the noise intensity distribution in different spatial regions of the image. Specifically, this includes:

[0051] S11: Image preprocessing: Obtain the noisy image to be processed, with a size of H×W (height×width). For video or sequence images, it can be processed frame by frame or aggregated along the time dimension. Normalize the image and map the pixel value range to the [0, 1] interval for subsequent statistical calculations.

[0052] S12: Local neighborhood definition: Based on each pixel in the image Construct a local neighborhood window of a preset size centered on the target area. The window size can be adjusted according to the image resolution and noise characteristics, such as 7×7, 11×11 or 15×15. For pixels at the edge of the image, the boundary is extended by mirror filling or zero filling to ensure that all pixels have a complete neighborhood window.

[0053] S13: Calculation of local statistics and noise response values;

[0054] For each pixel position Local neighborhood window Calculate the following statistics:

[0055] (1) Local mean:

[0056] ;

[0057] in, Indicates the number of pixels within the neighborhood window;

[0058] (2) Local variance :

[0059] ;

[0060] Local variance reflects the degree of fluctuation of pixel gray values ​​within a spatial region and is an important indicator of noise intensity;

[0061] (3) The noise response value Defined as the local standard deviation, which is the square root of the local variance:

[0062] ;

[0063] To improve robustness, the response values ​​are normalized:

[0064] ;

[0065] in and These are the minimum and maximum values ​​in the response graph, respectively.

[0066] S14: Generate a noise density response map;

[0067] The noise response value corresponding to each pixel position Combined to form a noise density response map Its size is the same as the input image (H×W), and the response map is continuously distributed in the spatial dimension. The numerical value reflects the relative level of noise intensity in the corresponding region.

[0068] like Figure 2 As shown, the noise density response map clearly presents the spatial pattern of noise distribution in the image. In background regions with weak signals, the response value is higher; in structural regions with strong signals, the response value is lower. This region-level noise intensity description provides reliable prior information for subsequent adaptive processing.

[0069] S2: As Figure 3 As shown, the input image is adaptively divided based on the noise density response map to generate a mask that distinguishes between high and low noise regions. Specifically, this includes:

[0070] S21: Determine the dynamic threshold;

[0071] Based on the generated noise density response map A threshold needs to be determined. To divide high-noise and low-noise regions, this embodiment uses a data-driven approach to automatically determine the threshold, avoiding manual setting, and employs the percentile threshold method.

[0072] ;

[0073] in, For percentile parameters, such as This represents the 60th percentile, and the division ratio can be adjusted according to the skewness of the noise distribution.

[0074] S22: Regional division;

[0075] Based on the determined threshold , response graph Binarization yields the region mask. :

[0076] ;

[0077] in, This indicates that the pixel belongs to a high-noise, dense area. This indicates that it belongs to a low-noise sparse region;

[0078] like Figure 3 As shown, the region division results can clearly distinguish between high-noise regions and low-noise regions. This division is highly consistent with the actual signal strength distribution, providing accurate guidance for differentiated processing.

[0079] S3: Constructing differentiated mask sampling and network architecture;

[0080] This embodiment proposes a differential masking sampling strategy for image data where noise density exhibits a significantly non-uniform spatial distribution, such as... Figure 5 As shown, a noise density-aware denoising network architecture is combined to improve the model's adaptability to different noise regions and its ability to preserve structural details under self-supervised training conditions.

[0081] As shown in Figure 4(a), during the differential mask generation process, the region mask is obtained based on the aforementioned noise density estimation results. The high-noise region corresponds to Low noise region corresponds For high-noise regions, a dense sampling strategy with a low mask rate is adopted, by setting the mask rate for high-noise regions. For each pixel in the region, with probability Random masking is performed to generate a high-noise region mask. This approach can reduce the risk of excessive information loss in high-noise regions while ensuring sufficient effective pixels participate in modeling. For low-noise regions, a sparse sampling strategy with a high mask rate is adopted, by setting the mask rate for low-noise regions. For each pixel in the region, with probability Perform random occlusion to generate low-noise region masks. To enhance the model's contextual reasoning ability in structurally complete regions, the overall training mask is finally obtained through element-wise combination, and its expression is:

[0082] ;

[0083] Where ⊙ represents element-wise multiplication;

[0084] To meet the constraints of blind spot network training, after generating the training mask, blind spot processing is needed for the occluded locations. That is, when predicting a pixel, the pixel itself and its immediate neighbors should not be used as network input. The algorithm performs morphological dilation on all occluded pixels, causing all pixels in their neighborhood to be occluded simultaneously, thereby preventing information leakage and ensuring the effectiveness and rationality of self-supervised training.

[0085] As shown in Figure 4(b), in terms of the overall network architecture, this embodiment adopts a hierarchical encoder-decoder structure as the backbone framework of the denoising network, and introduces a noise density sensing mechanism and a lightweight Transformer module to achieve collaborative modeling of local and global features. Each input channel first completes the initial feature extraction through convolution operation, and then maps the noise density information into high-dimensional features and fuses them with image features to enhance the noise sensing capability of feature expression.

[0086] The encoder consists of multiple cascaded coding blocks. Each coding block contains a noise density-aware window self-attention module and a feedforward network structure. The training process is stabilized through residual connections and normalization operations. Between adjacent coding blocks, the spatial resolution of the feature maps is reduced step by step through downsampling operations to expand the receptive field and extract high-level semantic information. The bottleneck layer located between the encoder and decoder is used for global feature modeling. It contains multiple noise density-aware self-attention modules to enhance the ability to model long-distance dependencies.

[0087] The decoder structure is symmetrical to the encoder. It gradually restores the spatial resolution through upsampling operations and introduces skip connections corresponding to the encoder layers to preserve low-level structural details. The skip connections are implemented using feature concatenation and channel compression and fusion are performed through convolution operations. Finally, the decoder output is convolved and mapped back to the image space to obtain a denoised image with the same number of input channels.

[0088] In the noise density-aware window self-attention module, the input feature map is first divided into multiple non-overlapping local windows, and self-attention is independently calculated within each window. After generating query, key, and value vectors through linear transformation, the noise density features are introduced into the modulation process of the key vector to achieve noise-aware attention weight allocation. Each self-attention module is followed by a feedforward network structure to further enhance the nonlinear representation capability of features. This feedforward network first expands the channel dimension through linear mapping and introduces local spatial modeling capability by combining depthwise separable convolution. After activation function processing, the number of channels is restored to the original dimension. The feedforward network uses a gating mechanism to achieve feature modulation, thereby improving feature selectivity and expression efficiency while ensuring model lightweightness.

[0089] In the training process, a self-supervised strategy is used to construct training samples by generating differential masks. In the inference phase, the noisy image or video to be denoised is first loaded and its format is converted and normalized. Then, the same noise density analysis and region segmentation as in the training phase are performed on each frame of the input data. This step can be pre-computed offline. Finally, the denoising results are saved to a specified path, and a quality assessment report can be generated selectively. The entire method starts from the original noise input, goes through data-driven noise analysis, adaptive region segmentation, mask-guided self-supervised network training, to model loading and adaptive denoising inference, and finally outputs high-quality results.

[0090] like Figure 6 As shown, compared with the effects of different denoising methods, the method in this embodiment exhibits excellent results.

[0091] Example 2

[0092] This embodiment provides a storage medium, which may be a ROM, RAM, disk, optical disk, or other storage medium. The storage medium stores one or more programs. When the programs are executed by a processor, they implement the self-supervised image denoising method based on noise density adaptive region segmentation as described in Embodiment 1.

[0093] Example 3

[0094] This embodiment provides a computing device, which may be a desktop computer, laptop computer, smartphone, PDA handheld terminal, tablet computer or other terminal device with display function. The computing device includes a processor and a memory. The memory stores one or more programs. When the processor executes the program stored in the memory, it implements the self-supervised image denoising method based on noise density adaptive region segmentation of Embodiment 1.

[0095] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A self-supervised image denoising method based on noise density adaptive region division, characterized in that, Includes the following steps: Perform local statistical analysis on the input noisy image to generate a noise density response map; Adaptive region segmentation of the noisy image is performed based on the noise density response map to generate a mask that distinguishes between high and low noise regions. Different sampling strategies with different masking rates are used to obtain the corresponding region masks for high and low noise regions. The training mask is obtained by combining elements one by one, and blind spot processing is performed on the occlusion positions of the training mask. A denoising network architecture is constructed, using a hierarchical encoder-decoder structure as the backbone. Each input channel completes the initial feature extraction through convolution operations, mapping the noise density information into high-dimensional features and fusing them with image features. The denoising network architecture is trained using a training mask, and the denoised image is output based on the trained denoising network.

2. The self-supervised image denoising method based on adaptive region division according to noise density according to claim 1, characterized in that, Perform local statistical analysis on the input noisy image to generate a noise density response map, specifically including: Obtain the noisy image to be processed and perform normalization on the image; A local neighborhood window of a preset size is constructed with each pixel of the normalized image as the center. The local mean, local variance, and local standard deviation are calculated based on the local neighborhood window, and the local standard deviation is used as the noise response value corresponding to each pixel position. The noise response values ​​corresponding to each pixel position are combined to form a noise density response map.

3. The self-supervised image denoising method based on adaptive region division according to noise density according to claim 1, characterized in that, Adaptive region segmentation of noisy images based on noise density response maps specifically includes: A data-driven approach is used to determine the threshold, and the noise density response map is binarized based on the threshold to obtain a region mask, which is then used to divide high-noise and low-noise regions.

4. The self-supervised image denoising method based on adaptive region division according to noise density according to claim 1, characterized in that, Different sampling strategies with different mask rates are used to obtain corresponding region masks for high and low noise regions. The training mask is then obtained through element-wise combination, specifically including: For high-noise regions, a dense sampling strategy with a low mask rate is adopted. By setting the mask rate of high-noise regions, each pixel in the region is randomly masked with the high-noise region mask rate to generate a high-noise region mask. For low-noise regions, a sparse sampling strategy with a high masking rate is adopted. By setting the masking rate of the low-noise region, each pixel in the region is randomly masked with the masking rate of the low-noise region to generate a low-noise region mask. The training mask is obtained by combining elements one by one, specifically represented as follows: ； in, Indicates the training mask. Represents a mask for high-noise regions. Represents a low-noise region mask. Represents a region mask, with high-noise regions corresponding to... Low noise region corresponds , ⊙ represents element-wise multiplication.

5. The self-supervised image denoising method based on noise density adaptive region segmentation according to claim 1, characterized in that, Blind spot handling is performed on the occlusion areas of the training mask, specifically including: In the training mask, a morphological dilation operation is performed on all occluded pixels, so that pixels in their neighborhood are also occluded.

6. The self-supervised image denoising method based on noise density adaptive region segmentation according to claim 1, characterized in that, The denoising network architecture includes an encoder and a decoder; The encoder consists of multiple cascaded coding blocks. Each coding block contains a noise density-aware window self-attention module and a feedforward network structure. The spatial resolution of the feature map is gradually reduced between adjacent coding blocks through downsampling operations. The bottleneck layer located between the encoder and the decoder is used for global feature modeling. The decoder is symmetrical to the encoder. It gradually restores the spatial resolution through upsampling operations and has skip connections corresponding to the encoder layers. The skip connections are implemented by feature concatenation and channel compression and fusion are performed through convolution operations. The decoder output is convolved and mapped back to the image space to obtain a denoised image with the same number of input channels.

7. The self-supervised image denoising method based on noise density adaptive region segmentation according to claim 6, characterized in that, In the window self-attention module, the input feature map is divided into multiple non-overlapping local windows, and self-attention is calculated independently in each window. After generating query, key, and value vectors through linear transformation, noise density features are added to the modulation process of the key vector to achieve noise-aware attention weight allocation. Each window self-attention module is followed by a feedforward network structure.

8. The self-supervised image denoising method based on noise density adaptive region segmentation according to claim 6, characterized in that, The feedforward network structure expands the channel dimension through linear mapping, combines depthwise separable convolution to achieve local spatial modeling, and restores the number of channels to the original dimension after activation function processing. The feedforward network structure uses a gating mechanism to achieve feature modulation.

9. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the self-supervised image denoising method based on noise density adaptive region segmentation as described in any one of claims 1-8.

10. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the self-supervised image denoising method based on noise density adaptive region partitioning as described in any one of claims 1-8.

Images