A U-Net image denoising method based on local perception and global feature modeling

Abstract

The application discloses a U-Net image denoising method based on local perception and global feature modeling, and relates to the technical field of image processing, which comprises the following steps: inputting an initial feature map into a parallel processing module, dividing the initial feature map into a local perception branch and a global feature modeling branch through channel-level learnable shunt division, respectively extracting image local texture and edge detail features and image global structure and long-range dependence features, and then fusing the features of the two branches and forming a global residual connection; inputting the fused features into an optimized U-Net architecture for multi-scale coding, bottleneck processing and decoding reconstruction, realizing cross-scale skip connection feature fusion in an element-by-element addition mode in the coding and decoding process, dynamically switching window attention modes according to the resolutions of feature maps at all levels, and outputting a clear image after denoising. The application can efficiently suppress noise while completely preserving image edge texture and overall structure, and can improve the effect, robustness and computational efficiency in the blind denoising scene.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a U-Net image denoising method based on local perception and global feature modeling, which can be widely used in photography and videography-related systems and terminal devices. Background Technology

[0002] Image denoising is a fundamental task in the field of digital image processing, aiming to suppress or eliminate random noise interference introduced during the imaging process while preserving as much of the image's true structure and detail as possible. Noise typically originates from multiple stages, including sensor readout, circuit thermal effects, photon statistical fluctuations, and subsequent image signal processing. Its presence reduces the signal-to-noise ratio of the image, obscures key visual information such as edges and textures, and thus adversely affects subsequent tasks such as visual perception, object detection, and image understanding.

[0003] To address the above problems, existing image denoising methods can be broadly categorized into the following technical approaches: The first category is denoising methods based on signal processing and prior modeling. Early image denoising methods mainly relied on classical signal processing theory, achieving noise suppression by modeling the statistical characteristics of the image or prior assumptions. These methods typically assume that a clean image possesses certain characterizable structural properties in the spatial or transform domain, such as local smoothness, sparsity, or frequency domain energy concentration, and utilize these properties through filtering, transforming, or optimization techniques. At the spatial domain level, linear filtering methods (such as mean filtering and Gaussian filtering) achieve noise smoothing through weighted averaging of neighboring pixels; these methods are computationally simple but prone to causing edge blurring. Nonlinear filtering methods (such as median filtering and bilateral filtering) suppress noise while preserving edge information through adaptive weight design. At the transform domain level, thresholding denoising techniques based on wavelet transform, Fourier transform, etc., utilize the difference in energy distribution between noise and signal in the transform domain, achieving noise separation by applying soft or hard thresholding to the transform coefficients.

[0004] The second category is image denoising methods based on nonlocal similarity. To overcome the limitations of local priors, subsequent research introduced the assumption of nonlocal self-similarity, which states that natural images generally contain a large number of repetitive or similar local structural blocks, which may be distributed in different spatial locations within the image. Methods based on this assumption search for a set of blocks with similar structures to the target block across the entire image and apply collaborative filtering to model them, thereby fully utilizing the redundant information of the image to effectively suppress noise. A typical example is nonlocal mean filtering, which denoises by weighted averaging of similar blocks.

[0005] The third category is image denoising methods based on deep learning. With the rapid development of deep learning technology, data-driven denoising methods based on neural networks have gradually become the mainstream of research. These methods automatically learn the complex nonlinear mapping relationship between clean and noisy images from a large number of training samples by constructing an end-to-end mapping model, thus avoiding the tedious process of manually designing denoising rules and prior models, and showing better denoising effects than traditional methods under various noise types and scenarios. According to the differences in network structure and modeling strategy, this category of methods can be further subdivided into: local perception modeling methods with convolutional neural networks as the core, such as DnCNN and FFDNet, which focus on capturing local spatial correlations by using convolutional operations and gradually extracting and fusing multi-level features by stacking convolutional layers; global feature modeling methods that introduce attention mechanisms or Transformer architectures, such as SwinIR and Restormer, which are used to characterize long-range dependencies in images and enhance the ability to perceive global contextual information; and hybrid modeling methods that combine encoder-decoder multi-scale structures for feature reconstruction, such as U-Net and its variants, which enhance the adaptability to noise and structural features at different spatial scales through cross-scale feature extraction and fusion.

[0006] Although the aforementioned noise modeling and degradation synthesis methods have promoted the development of image denoising technology to some extent, they still have limitations in general blind denoising application scenarios, as analyzed below: (1) Denoising methods based on signal processing and prior modeling: Although this method has the advantages of low computational cost, algorithm transparency and strong interpretability, its performance is highly dependent on the prior assumptions and filter parameters set by humans. At the spatial domain level, although linear filtering methods can effectively suppress high-frequency noise, they will inevitably smooth out the edges and texture details of the image, resulting in an overly blurred visual effect in the denoising result; although nonlinear filtering methods have improved in terms of edge preservation, their adaptive ability is limited and it is difficult to cope with complex scenarios where the spatial distribution of noise intensity is uneven or the noise type changes dynamically. At the transform domain level, thresholding-based denoising technology is more sensitive to the selection of threshold parameters. Too high a threshold will lead to the loss of signal details, while too low a threshold will result in obvious noise residue. Moreover, such methods are usually based on idealized assumptions such as additive white Gaussian noise. When facing signal-dependent noise, structured noise or mixed noise in real scenes, the denoising effect often drops significantly.

[0007] (2) Image denoising methods based on non-local similarity: This method exhibits excellent denoising performance in regions with regular textures and repetitive structures, but still has several inherent drawbacks. First, the computational complexity of this type of method is high. The similarity block search process needs to perform block-by-block matching within the entire image or a large window, and the time cost increases quadratically with the image size, making it difficult to meet the requirements of applications with high real-time requirements. Second, the accuracy of the similarity block search directly affects the denoising effect. When the image lacks sufficient repetitive structures or the noise intensity is large, causing the similarity measurement to be inaccurate, the robustness of the method will decrease significantly. In addition, for image regions with unique structures or non-repetitive details, it is difficult to find effective similar blocks for collaborative filtering, so the denoising effect is often unsatisfactory, and may even introduce artifacts or structural distortion.

[0008] (3) Deep learning-based image denoising methods: Although this method has achieved significant breakthroughs in denoising performance, existing models still face many challenges in practical applications. First, there is an inherent trade-off between local detail restoration and global structural consistency: methods based on convolutional neural networks are limited by the local receptive field of the convolutional kernel, making it difficult to effectively capture long-range dependencies in images. When processing large-scale structures, semantic incoherence or insufficient global consistency may occur. While global modeling methods based on the Transformer architecture can effectively characterize long-range dependencies, the computational complexity of its self-attention mechanism is quadratic with the image resolution, resulting in significant computational overhead and memory pressure when processing high-resolution images. Second, existing multi-scale fusion strategies are still not refined enough in terms of cross-scale feature alignment and information integration. The semantic gap between features at different scales may lead to incomplete detail restoration or structural inconsistency between scales. Third, the generalization ability of deep learning methods is limited by the distribution characteristics of training data. When the noise type, intensity, or scene features of the test image differ significantly from those of the training set, the model performance may degrade significantly and it is not robust enough in the complex and ever-changing blind denoising scenarios of the real world.

[0009] Therefore, how to solve the shortcomings of existing image denoising methods, such as difficulty in balancing local detail restoration and global structural consistency, redundancy in cross-scale feature fusion, high computational overhead of global modeling and insufficient adaptability, resulting in poor denoising effect, low computational efficiency and weak generalization in blind denoising scenarios, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0010] In view of this, the present invention provides a U-Net image denoising method based on local perception and global feature modeling, which solves the problems existing in the background technology.

[0011] To achieve the above objectives, the present invention provides the following technical solution: A U-Net image denoising method based on local perception and global feature modeling includes the following steps: S1. Extract features from the noisy input image to obtain an initial feature map; S2. Input the initial feature map into the parallel processing module. After channel-level learnable splitting, it is divided into a local perception branch and a global feature modeling branch. The local texture and edge detail features, global structure and long-range dependency features of the image are extracted respectively. Then, the features of the two branches are fused and a global residual connection is formed to obtain a fused feature that has both local fine information and global semantic information. S3. Input the fused features into the optimized U-Net architecture for multi-scale encoding, bottleneck processing and decoding reconstruction. In the encoding and decoding process, the cross-scale skip connection feature fusion is achieved by adding elements one by one. The window attention mode is dynamically switched according to the resolution of the feature map at each level, and the denoised clear image is output.

[0012] Optionally, in S2, the specific operation of channel-level learnable splitting is as follows: Let the initial feature map be ,in H , W , C These represent the height, width, and number of channels of the feature map, respectively; the initial feature map is then subjected to channel-level linear transformation and reconstruction using a 1×1 pointwise convolution:

[0013] Transformed features Divide the data evenly into two sub-features along the channel dimension:

[0014] In the formula: For learnable convolution kernel parameters, Indicates the convolution operation; The feature map representing the input local perceptual branch. This represents the feature map of the input global feature modeling branch.

[0015] Optionally, in S2, the local perception branch uses a residual convolution structure to extract local texture and edge detail features. The specific operation is as follows: set up This represents a nonlinear mapping consisting of two 3×3 convolutional layers and a ReLU activation function:

[0016] Output features of the local perceptual branch for:

[0017] In the formula: For convolution kernel parameters, Represents the ReLU activation function. F This represents the input features.

[0018] Optionally, in S2, the global feature modeling branch uses a multi-head self-attention mechanism based on a shifted window to model long-range dependencies. The specific operation is as follows: The feature map of the input global feature modeling branch Divided into [ H / M ]×[ W / M ] of size M × M The non-overlapping local windows, where M For the window size; for the first i Feature sequences within a window After layer normalization, a query matrix is ​​generated through linear projection. Key matrix Value matrix :

[0019] Attention within the window is calculated as follows:

[0020] To enable information exchange between adjacent windows, conventional window partitioning and shifted window partitioning are used alternately. Shifted windows are achieved by cyclically shifting the feature map along the horizontal and vertical directions. M Achieved using 2 pixels; The complete transformation of the global feature modeling branch includes a multi-head self-attention layer and a feedforward network layer, both using residual connections:

[0021] In the formula: The projection matrix; For each dimension of attention head, For the number of attention heads; It is a learnable relative position bias matrix used to encode the relative spatial position information between pixel pairs within the window; Presentation layer normalization operation, This represents a two-layer feedforward network. X Represents global input features. This represents the intermediate features after multi-head self-attention through the window. This indicates multi-head self-attention operation of the window. This represents the output feature of the global feature modeling branch.

[0022] Optionally, in S2, the specific method for obtaining the fused features is as follows: Output features of the local perceptual branch Output features of the global feature modeling branch After concatenation along the channel dimension, cross-branch feature interaction and fusion are performed through 1×1 convolution, and then combined with the initial feature map. Form a global residual connection:

[0023] In the formula: As a feature of fusion, To fuse the parameters of the convolution, This indicates a splicing operation along the channel dimension.

[0024] Optionally, in S3, multi-scale coding, bottleneck handling, and decoding reconstruction specifically include: Parallel processing modules are used as the basic building blocks of each layer of U-Net, and are deployed uniformly in the encoder, bottleneck layer, and decoder; let the... s The input features at each scale level are After processing by the parallel processing module, the following is obtained:

[0025] In the formula: Indicates a parallel processing module. This is the output of the parallel processing module.

[0026] Optionally, in S3, cross-scale skip connection feature fusion is achieved by element-wise addition, specifically as follows: Let the encoder be the first s Layer output is The features of the corresponding layer of the decoder after upsampling are: Then the fusion features for:

[0027] In the formula: This indicates an element-wise addition operation.

[0028] Optionally, in S3, the window attention mode is dynamically switched according to the resolution of the feature maps at each level, specifically as follows: In the encoding path, bottleneck path and decoding path of U-Net, a global feature modeling scheduling mechanism related to the feature map resolution is introduced to adaptively adjust the global feature modeling mode according to the spatial size of the current feature layer. Let the first s The feature map spatial resolution at each scale level is: The preset window size is M Then the selection rules for global feature modeling mode Defined as:

[0029] In the formula: This indicates that the window has multiple heads for self-attention. This indicates a multi-head self-attention mechanism in the shift window.

[0030] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a U-Net image denoising method based on local perception and global feature modeling, which has the following beneficial effects: (1) Enhanced collaborative modeling capability of local and global features: This invention organically integrates the local perception capability of convolution with the global modeling capability of window self-attention through a dual-path parallel processing module, overcoming the problems of limited receptive field of convolution or excessive computational overhead of attention mechanism in traditional methods. It can effectively suppress noise while better preserving the edge texture and overall structural information of the image.

[0031] (2) Improved adaptability of feature allocation and modeling method: The present invention adopts a learnable channel rearrangement and diversion mechanism, which enables the network to adaptively adjust the allocation ratio of local and global features according to the characteristics of the input image; at the same time, a resolution-aware attention scheduling strategy is introduced to dynamically switch the window attention mode according to the feature map size, thereby achieving a balance between modeling ability and computational efficiency.

[0032] (3) Multi-scale feature fusion efficiency optimization: This invention uses element-wise addition to replace traditional channel splicing to achieve skip connection. While maintaining the information transmission function between the encoder and decoder, it avoids the redundant expansion of the channel dimension and effectively reduces the number of network parameters and computational complexity.

[0033] (4) Comprehensive improvement of denoising performance and visual quality: Through the above optimization design, the present invention can effectively balance detail restoration and structure preservation in image denoising tasks, and exhibits good denoising effect and visual quality under various noise levels. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1A flowchart of the U-Net image denoising method based on local perception and global feature modeling provided by the present invention; Figure 2 The algorithm flowchart provided for this invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This invention discloses a U-Net image denoising method based on local perception and global feature modeling, such as... Figure 1 As shown, it includes the following steps: S1. Extract features from the noisy input image to obtain an initial feature map; S2. Input the initial feature map into the parallel processing module. After channel-level learnable splitting, it is divided into a local perception branch and a global feature modeling branch. The local texture and edge detail features, global structure and long-range dependency features of the image are extracted respectively. Then, the features of the two branches are fused and a global residual connection is formed to obtain a fused feature that has both local fine information and global semantic information. S3. Input the fused features into the optimized U-Net architecture for multi-scale encoding, bottleneck processing and decoding reconstruction. In the encoding and decoding process, the cross-scale skip connection feature fusion is achieved by adding elements one by one. The window attention mode is dynamically switched according to the resolution of the feature map at each level, and the denoised clear image is output.

[0038] based on Figure 1 The steps shown will help you understand the following: 1) This embodiment designs a parallel processing module that integrates local perception and global feature modeling capabilities to simultaneously model local texture details and global structural information during image denoising. This module performs a linear transformation on the input features to achieve channel rearrangement and adaptive grouping, and divides the processed features into two parallel feature subspaces, thereby providing a more suitable modeling path for different types of features.

[0039] 2) This embodiment addresses the issues of insufficient global modeling capability and redundant cross-scale feature fusion in traditional U-Net image denoising tasks by proposing three optimizations: First, it replaces traditional convolutional blocks with local-global parallel processing modules as the basic building blocks for each scale layer, giving the network the ability to capture both detailed textures and global structure simultaneously; second, it uses element-wise addition instead of channel splicing to achieve skip connections, avoiding channel dimension expansion while maintaining information transmission; and third, it introduces a resolution-aware attention scheduling strategy, dynamically switching window attention modes according to the feature map size, taking into account both local fine modeling at high-resolution levels and global structural understanding at low-resolution levels.

[0040] Below, in conjunction with Figure 2 The present invention provides a detailed description of the parallel processing module for local perception and global feature modeling and the U-Net architecture optimization method for image denoising described in this embodiment.

[0041] I. Parallel Processing Module for Local Perception and Global Feature Modeling

[0042] This embodiment designs a parallel processing module that integrates local perception and global feature modeling capabilities, and achieves collaborative optimization of convolution and window self-attention through a channel-level learnable diversion mechanism.

[0043] 1. Channel-level learnable routing

[0044] In S2 of this embodiment, the specific operation of channel-level learnable splitting is as follows: Let the initial feature map be ,in H , W , C These represent the height, width, and number of channels of the feature map, respectively. First, a channel-level linear transformation and reconstruction are performed on the initial feature map using a 1×1 pointwise convolution.

[0045] Then the transformed features Divide the data evenly into two sub-features along the channel dimension:

[0046] In the formula: For learnable convolution kernel parameters, This indicates a convolution operation, which enables the network to adaptively learn the optimal recombination method between channels, rather than using a fixed channel splitting strategy; The feature map representing the input local perceptual branch. This represents the feature map of the input global feature modeling branch.

[0047] 2. Local perceptual branch

[0048] In S2 of this embodiment, the local perception branch uses a residual convolution structure to extract local texture and edge detail features. The specific operation is as follows: set up This represents a nonlinear mapping consisting of two 3×3 convolutional layers and a ReLU activation function:

[0049] Output features of the local perceptual branch for:

[0050] In the formula: For convolution kernel parameters, Represents the ReLU activation function. F This represents the input features. Residual connections enable this branch to learn the residual mapping between the input and the target, which is beneficial for gradient propagation and network optimization.

[0051] 3. Global Feature Modeling Branch

[0052] In S2 of this embodiment, the global feature modeling branch uses a multi-head self-attention mechanism based on a shift window to model long-range dependencies. The specific operation is as follows: First, the feature map input to the global feature modeling branch is... Divided into [ H / M ]×[ W / M ] of size M × M The non-overlapping local windows, where M For the window size; for the first i Feature sequences within a window After layer normalization, a query matrix is ​​generated through linear projection. Key matrix Value matrix :

[0053] Attention within the window is calculated as follows:

[0054] To enable information interaction between adjacent windows, this embodiment alternates between conventional window partitioning and shifted window partitioning within consecutive modules. The shifted window is achieved by cyclically shifting the feature map along the horizontal and vertical directions. M Achieved using 2 pixels; The complete transformation of the global feature modeling branch includes a multi-head self-attention layer and a feedforward network layer, both using residual connections:

[0055] In the formula: The projection matrix; For each dimension of attention head, For the number of attention heads; It is a learnable relative position bias matrix used to encode the relative spatial position information between pixel pairs within the window; Presentation layer normalization operation, This represents a two-layer feedforward network, where the hidden layer dimension is expanded to four times the input dimension; X Represents global input features. This represents the intermediate features after multi-head self-attention through the window. This indicates multi-head self-attention operation of the window. This represents the output feature of the global feature modeling branch.

[0056] 4. Dual-path feature fusion

[0057] In S2 of this embodiment, the method for obtaining the fused features is specifically as follows: Output features of the local perceptual branch Output features of the global feature modeling branch After concatenation along the channel dimension, cross-branch feature interaction and fusion are performed through 1×1 convolution, and then combined with the initial feature map. Form a global residual connection:

[0058] In the formula: As a feature of fusion, To fuse the parameters of the convolution, This represents a concatenation operation along the channel dimension. This fusion strategy enables complementary enhancement of local detail features and global structural features, while global residual connections ensure the stability of deep network training.

[0059] II. Optimization of U-Net Architecture for Image Denoising

[0060] This embodiment addresses the limitations of the traditional U-Net architecture in image denoising tasks by optimizing the network structure in three aspects: feature extraction module design, cross-scale feature fusion mechanism, and resolution-adaptive attention scheduling.

[0061] 1. Feature extraction module design optimization

[0062] Traditional U-Net uses pure convolutional blocks as the basic feature extraction units. Limited by the local receptive field of the convolutional kernel, it struggles to effectively model long-range dependencies in images. To address this issue, the "multi-scale encoding, bottleneck handling, and decoding reconstruction" design proposed in S3 of this embodiment is as follows: Parallel processing modules are used as the basic building blocks of each layer of U-Net, and are deployed uniformly in the encoder, bottleneck layer, and decoder; let the... s The input features at each scale level are After processing by the parallel processing module, the following is obtained:

[0063] In the formula: Indicates a parallel processing module. This is the output of the parallel processing module. This design enables the network to have the dual capabilities of local detail extraction and global structure modeling at each scale level, overcoming the limitation of traditional U-Net which relies solely on convolution for feature extraction.

[0064] 2. Optimization of cross-scale feature fusion mechanism

[0065] Traditional U-Net uses channel concatenation to fuse corresponding layer features from the encoder and decoder. This method leads to a sharp increase in the number of feature channels during the decoding stage, increasing the computational burden and the number of parameters. Therefore, in S3, this embodiment uses element-wise addition to achieve cross-scale skip-connection feature fusion, specifically: Let the encoder be the first s Layer output is The features of the corresponding layer of the decoder after upsampling are: Then the fusion features for:

[0066] In the formula: This represents an element-wise addition operation. This strategy maintains the information transmission function of skip connections while avoiding redundant expansion of the channel dimension, ensuring that the number of channels in each layer of the decoder is strictly symmetrical to that of the encoder, thus reducing the number of network parameters and computational complexity.

[0067] 3. Resolution-Adaptive Attention Scheduling Optimization

[0068] Traditional U-Net architectures typically employ the same or static feature modeling methods across different scale levels, making it difficult to simultaneously address the needs of local detail modeling at high resolution and global structure modeling at low resolution. This can easily lead to computational redundancy or modeling failure. Specifically, forcibly using global attention at high resolution levels incurs significant computational overhead, while using a fixed shifting window strategy at low resolution levels may result in incomplete window partitioning due to insufficient feature map size, leading to invalid cross-region modeling operations.

[0069] Therefore, in S3 of this embodiment, the window attention mode is dynamically switched according to the resolution of the feature map at each level, specifically as follows: In the encoding path, bottleneck path and decoding path of U-Net, a global feature modeling scheduling mechanism related to the feature map resolution is introduced to adaptively adjust the global feature modeling mode according to the spatial size of the current feature layer. Let the first s The feature map spatial resolution at each scale level is: The preset window size is M Then the selection rules for global feature modeling mode Defined as:

[0070] In the formula: This indicates that the window has multiple heads for self-attention. This represents multi-head self-attention in a shifted window. When the feature map resolution is high, this is achieved by alternating between adjacent modules. and A cross-window information interaction mechanism is introduced to enhance the feature association modeling capability between different spatial regions. When the feature map resolution is reduced to close to or smaller than the preset window size, a non-shifted local window modeling method is automatically adopted, thereby avoiding the introduction of invalid cross-region modeling operations at low resolution.

[0071] 4. Overall Model Architecture: Multi-scale Unified Feature Modeling Architecture

[0072] In terms of structural design, this embodiment combines the parallel processing modules for local perception and global feature modeling with the proposed resolution-adaptive attention scheduling strategy, systematically embedding them into various scale levels of U-Net. Let the input image resolution be... H × W The number of basic channels is The network consists of four scale levels, and the feature configurations for each level are as follows:

[0073] Parallel processing modules are deployed at each scale level to ensure consistency in feature modeling logic across the encoding, bottleneck, and decoding stages, with the global modeling approach dynamically adjusted only according to changes in feature map resolution. Through this design, the network progressively enhances its perception of global structural information during downsampling and synchronously guides the feature reconstruction process during upsampling, thereby guaranteeing the continuity and consistency of multi-scale features in structural semantics.

[0074] III. Supplement

[0075] The technical solution of this embodiment features a modular design. Each feature modeling module and network structure can be simplified, omitted, or replaced according to actual application requirements, so as to reduce the implementation complexity or adapt to specific application scenarios while maintaining the core technical effect.

[0076] (1) Simplified embodiment of parallel processing module

[0077] In a complete embodiment of the invention, the parallel processing module comprises two parallel paths: a local perception branch and a global feature modeling branch, employing a residual convolutional structure and a shifted window multi-head self-attention mechanism, respectively. In a simplified embodiment, the module structure can be simplified according to the target application scenario: if the target scenario has high real-time requirements and the image structure is relatively simple, the global feature modeling branch can be omitted, retaining only the local perception branch for feature extraction; if the target scenario mainly processes images with obvious repetitive structures or global patterns, the number of convolutional layers in the local perception branch can be simplified. These simplification schemes reduce branch complexity, decrease the number of model parameters, and shorten inference time, while maintaining adaptability to the target scenario.

[0078] (2) Simplified implementation of channel splitting mechanism

[0079] In the complete embodiment of the invention, channel splitting employs learnable 1×1 convolutions to achieve channel rearrangement and adaptive grouping. In a simplified embodiment, the learnable channel rearrangement can be simplified to a fixed channel equal-division strategy, that is, directly dividing the input features into two equal parts along the channel dimension, inputting them into a local perception branch and a global feature modeling branch respectively; furthermore, a fixed-ratio non-uniform partitioning can be used. The above simplified scheme omits the learning process of channel rearrangement, reduces the number of module parameters, and is suitable for application scenarios where the adaptiveness requirements for feature allocation are not high.

[0080] (3) Simplified implementation of U-Net architecture

[0081] In the complete embodiment of this invention, U-Net comprises four scale levels, each deploying parallel processing modules and employing a resolution-adaptive attention scheduling strategy. In a simplified embodiment, the number of network scale levels can be reduced, or parallel processing modules can be deployed only at some scale levels, with traditional convolutional blocks used in the remaining levels; alternatively, the resolution-adaptive scheduling strategy can be omitted, and non-shifted window attention can be uniformly used across all levels. These simplified schemes significantly reduce computational complexity and memory usage by decreasing network depth or module deployment scope, making them suitable for edge deployment scenarios with limited computing resources.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0083] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A U-Net image denoising method based on local perception and global feature modeling, characterized in that, Includes the following steps: S1. Extract features from the noisy input image to obtain an initial feature map; S2. Input the initial feature map into the parallel processing module. After channel-level learnable splitting, it is divided into a local perception branch and a global feature modeling branch. The local texture and edge detail features, global structure and long-range dependency features of the image are extracted respectively. Then, the features of the two branches are fused and a global residual connection is formed to obtain a fused feature that has both local fine information and global semantic information. S3. Input the fused features into the optimized U-Net architecture for multi-scale encoding, bottleneck processing and decoding reconstruction. In the encoding and decoding process, the cross-scale skip connection feature fusion is achieved by adding elements one by one. The window attention mode is dynamically switched according to the resolution of the feature map at each level, and the denoised clear image is output.

2. The U-Net image denoising method based on local perception and global feature modeling according to claim 1, characterized in that, In S2, the specific operation of channel-level learnable splitting is as follows: Let the initial feature map be ,in H , W , C These represent the height, width, and number of channels of the feature map, respectively; the initial feature map is then subjected to channel-level linear transformation and reconstruction using a 1×1 pointwise convolution: Transformed features Divide the data evenly into two sub-features along the channel dimension: In the formula: For learnable convolution kernel parameters, Indicates the convolution operation; The feature map representing the input local perceptual branch. This represents the feature map of the input global feature modeling branch.

3. The U-Net image denoising method based on local perception and global feature modeling according to claim 2, characterized in that, In S2, the local perception branch uses a residual convolution structure to extract local texture and edge detail features. The specific operation is as follows: set up This represents a nonlinear mapping consisting of two 3×3 convolutional layers and a ReLU activation function: Output features of the local perceptual branch for: In the formula: For convolution kernel parameters, Represents the ReLU activation function. F This represents the input features.

4. The U-Net image denoising method based on local perception and global feature modeling according to claim 3, characterized in that, In S2, the global feature modeling branch uses a multi-head self-attention mechanism based on a shifted window to model long-range dependencies. The specific operation is as follows: The feature map of the input global feature modeling branch Divided into [ H / M ]×[ W / M ] of size M × M The non-overlapping local windows, where M For the window size; for the first i Feature sequences within a window After layer normalization, a query matrix is ​​generated through linear projection. Key matrix Value matrix : Attention within the window is calculated as follows: To enable information exchange between adjacent windows, conventional window partitioning and shifted window partitioning are used alternately. Shifted windows are achieved by cyclically shifting the feature map along the horizontal and vertical directions. M Achieved using 2 pixels; The complete transformation of the global feature modeling branch includes a multi-head self-attention layer and a feedforward network layer, both using residual connections: In the formula: The projection matrix; For each dimension of attention head, For the number of attention heads; It is a learnable relative position bias matrix used to encode the relative spatial position information between pixel pairs within the window; Presentation layer normalization operation, This represents a two-layer feedforward network. X Represents global input features. This represents the intermediate features after multi-head self-attention through the window. This indicates multi-head self-attention operation of the window. This represents the output feature of the global feature modeling branch.

5. The U-Net image denoising method based on local perception and global feature modeling according to claim 4, characterized in that, In S2, the specific method for obtaining the fused features is as follows: Output features of the local perceptual branch Output features of the global feature modeling branch After concatenation along the channel dimension, cross-branch feature interaction and fusion are performed through 1×1 convolution, and then combined with the initial feature map. Form a global residual connection: In the formula: As a feature of fusion, To fuse the parameters of the convolution, This indicates a splicing operation along the channel dimension.

6. The U-Net image denoising method based on local perception and global feature modeling according to claim 1, characterized in that, In S3, multi-scale coding, bottleneck handling, and decoding reconstruction are specifically as follows: Parallel processing modules are used as the basic building blocks of each layer of U-Net, and are deployed uniformly in the encoder, bottleneck layer, and decoder; let the... s The input features at each scale level are After processing by the parallel processing module, the following is obtained: In the formula: Indicates a parallel processing module. This is the output of the parallel processing module.

7. The U-Net image denoising method based on local perception and global feature modeling according to claim 1, characterized in that, In S3, cross-scale skip connection feature fusion is achieved by element-wise addition, specifically as follows: Let the encoder be the first s Layer output is The features of the corresponding layer of the decoder after upsampling are: Then the fusion features for: In the formula: This indicates an element-wise addition operation.

8. The U-Net image denoising method based on local perception and global feature modeling according to claim 1, characterized in that, In S3, the window attention mode is dynamically switched according to the resolution of the feature maps at each level, specifically as follows: In the encoding path, bottleneck path and decoding path of U-Net, a global feature modeling scheduling mechanism related to the feature map resolution is introduced to adaptively adjust the global feature modeling mode according to the spatial size of the current feature layer. Let the first s The feature map spatial resolution at each scale level is: The preset window size is M Then the selection rules for global feature modeling mode Defined as: In the formula: This indicates that the window has multiple heads for self-attention. This indicates a multi-head self-attention mechanism in the shift window.

Images