Image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion

Abstract

The invention discloses an image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion, and the device comprises a data preprocessing module which is used for carrying out the pixel normalization processing, color space conversion and size and resolution adjustment of a to-be-denoised image, and obtaining a standard image; the adaptive local convolution enhancement module dynamically adjusts a receptive field of a convolution kernel according to noise characteristics on the standard image, and extracts an original feature map on the standard image; the long and short term dependency modeling module based on beam scanning captures long and short term dependency on the original feature map by using a beam scanning technology, and fuses the long and short term dependency with a standard image to obtain a preliminary de-noised feature map; the multi-level representation information extraction module is used for extracting deep-level de-noising feature maps of different scales in the preliminary de-noising feature maps; and the refined reconstruction module carries out fusion and refined reconstruction on the original feature map and the deep denoising feature map, and outputs a noiseless image. According to the invention, efficient and accurate image denoising processing can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to an image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion. Background Art

[0002] Image denoising is a crucial task in image processing, aiming to restore high-quality original images from noise-contaminated images. Currently, deep learning has demonstrated outstanding performance in image denoising, with Transformer methods based on convolutional neural networks and self-attention mechanisms becoming mainstream. While these techniques have addressed the limitations of traditional denoising algorithms to some extent, they still face significant challenges when processing high-resolution images and complex noisy scenes.

[0003] Convolutional neural networks have been widely used in image denoising due to their powerful local feature extraction capabilities. The convolution layer can effectively restore image details, but its inherent limitation lies in the limited receptive field, which means that convolutional neural networks are insufficient in modeling global dependencies, especially in high-resolution images. It is difficult to capture the dependencies between distant pixels, resulting in defects in the denoising effect in terms of global structure preservation. In addition, the convolution operation is performed in the same way on different channels, and adaptive adjustments cannot be made according to the importance of each channel, resulting in insufficient utilization of channel information and channel redundancy, which reduces the efficiency of convolutional neural networks.

[0004] In contrast, the transformer-based self-attention mechanism excels in image denoising by modeling global pixel dependencies, particularly in preserving global structure and restoring detail. The transformer's self-attention mechanism is able to capture global dependencies in an image through a large receptive field. However, the transformer's main bottleneck is its computational complexity, which grows at O(n²). This makes it particularly difficult to meet the efficiency requirements of practical applications, especially in high-resolution image denoising tasks, due to its extremely high computational resource requirements. Although some studies have attempted to reduce computational overhead through sliding window attention mechanisms, this approach often sacrifices the global receptive field and has not yet fully resolved the trade-off between computational complexity and global dependency modeling.

[0005] In recent years, state-space models (SSMs), particularly the Mamba model, have demonstrated great potential for modeling long-range dependencies through their efficient structured state-space mechanism. The Mamba model utilizes recursive state equations and a parallel scanning algorithm, effectively capturing dependencies in long sequences while also processing global information with linear complexity. Consequently, it excels in one-dimensional sequence tasks such as natural language processing. However, the Mamba model faces new challenges when applied to image processing, particularly image denoising. First, the Mamba model flattens two-dimensional images into one-dimensional sequences for processing, which can cause originally adjacent pixels to become further apart after flattening, leading to local pixel forgetting. Second, to capture long-range dependencies, the Mamba model introduces a large number of hidden states, which not only creates channel redundancy but also compromises the model's computational efficiency and feature representation capabilities.

[0006] In summary, although existing deep learning methods have made some progress in image denoising tasks, the following key issues still exist:

[0007] (1) Local pixel forgetting problem: Due to the limited receptive field, convolutional neural networks are unable to effectively capture the global information of the image. The Mamba model separates spatially adjacent pixels to farther positions during the image flattening process, which weakens the dependence between originally correlated local pixels and easily loses local detail information.

[0008] (2) Channel redundancy: Existing convolutional neural networks and Mamba models process all channels uniformly when processing images, failing to adaptively select and strengthen important channel information, resulting in a waste of computing resources and limiting the denoising effect of the model. There is no adaptive adjustment mechanism for the information processing weights of different channels, resulting in the network failing to fully utilize all effective features.

[0009] (3) The contradiction between global dependency modeling and computational complexity: The transformer-based self-attention mechanism can capture the global dependencies between pixels in a large receptive field, but its computational complexity is extremely high, especially when processing high-resolution images, requiring huge computing resources. Although the Mamba model optimizes the efficiency of global dependency modeling through parallel scanning and state equations, its ability to balance global and local information modeling still needs to be improved.

[0010] (4) Lack of adaptive noise processing capabilities: Existing denoising models often lack the ability to adaptively process noise characteristics. The network parameters are fixed and cannot be dynamically adjusted according to different noise levels and image characteristics, resulting in unsatisfactory denoising effects in complex noise scenes. Summary of the Invention

[0011] In response to the problems existing in the prior art, the present invention provides an image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion to solve the problems of local detail loss, channel redundancy, global dependency modeling and computational complexity balance in existing image denoising technologies, thereby achieving efficient and accurate image denoising processing.

[0012] To achieve the above technical objectives, the present invention adopts the following technical solutions: an image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion, comprising: a data preprocessing module, an adaptive local convolution enhancement module, a long-term and short-term dependency modeling module based on beam scanning, a multi-level representation information extraction module, and a refined reconstruction module;

[0013] The data preprocessing module is used to perform pixel normalization, color space conversion, and size and resolution adjustment on the image to be denoised to obtain a standard image;

[0014] The adaptive local convolution enhancement module dynamically adjusts the receptive field of the convolution kernel according to the noise characteristics of the standard image and extracts the original feature map of the standard image;

[0015] The beam scanning-based long-term and short-term dependency modeling module uses beam scanning technology to capture the long-term and short-term dependencies on the original feature map, and fuses it with the standard image to obtain a preliminary denoised feature map;

[0016] The multi-level representation information extraction module is used to extract deep denoising feature maps of different scales in the preliminary denoising feature map;

[0017] The refined reconstruction module fuses and reconstructs the original feature map and the deep denoising feature map, and outputs a noise-free image.

[0018] Furthermore, the adaptive local convolution enhancement module includes: a noise characteristic analysis unit, a dynamic convolution kernel generation unit, a convolution operation unit and a feature fusion unit;

[0019] The noise characteristic analysis unit analyzes the noise characteristics on the standard image and extracts information on the noise type, intensity and spatial distribution;

[0020] The dynamic convolution kernel generation unit dynamically generates a convolution kernel adapted to the current noise environment based on the noise information extracted by the noise characteristic analysis unit;

[0021] The convolution operation unit performs a convolution operation on the standard image using the generated convolution kernel;

[0022] The feature fusion unit fuses the features after the convolution operation with the standard image to obtain the original feature map on the standard image.

[0023] Furthermore, the dynamic generation process of the convolution kernel includes:

[0024] i. Dynamically adjust the size of the convolution kernel according to the type and intensity of the noise. The greater the noise intensity, the larger the convolution kernel.

[0025] ii. Dynamically adjust the geometry of the convolution kernel according to the spatial distribution of the noise;

[0026] iii. Optimize the weight parameters of the convolution kernel through a learnable parameter adjustment mechanism, and dynamically adjust the weight distribution of the convolution kernel through the attention mechanism.

[0027] Furthermore, the beam scanning-based long-term and short-term dependency modeling module includes: a beam scanning unit, a Mamba long-term dependency modeling unit, a short-term dependency modeling unit, a dependency fusion unit, and a feature integration unit;

[0028] The beam scanning unit dynamically adjusts the scanning beam according to the noise intensity, scans the original feature map, and obtains the scanning feature;

[0029] The Mamba long-range dependency modeling unit inputs the original feature map and the scanned features into the long-range dependency modeling unit to capture the dependency relationship between long-range pixels in the original feature map;

[0030] The short-range dependency modeling unit is used to capture the dependency relationship between short-range pixels in the original feature map;

[0031] The dependency fusion unit uses the attention mechanism to weightedly fuse the dependency between long-distance pixels and the dependency between short-distance pixels to obtain fused dependency information;

[0032] The feature integration unit performs feature fusion on the fusion dependency information and the standard image to obtain a preliminary denoising feature map.

[0033] Furthermore, the specific process of the beam scanning unit scanning the original feature map is:

[0034] A. Design several scanning directions based on various structural and texture information on the original feature map, and generate a corresponding scanning path for each scanning direction;

[0035] B. Dynamically adjust the width and stride of the scanning beam based on the noise intensity information experienced along the scanning path. The greater the noise intensity, the wider the scanning beam width and the smaller the scanning beam stride.

[0036] C. Using the dynamically adjusted scanning beam width and stride, the original feature map is scanned along the scanning path to obtain the scanning features.

[0037] Furthermore, the multi-level representation information extraction module is composed of several multi-scale feature fusion residual state space group modules, each of which is composed of several multi-scale feature fusion residual state space blocks, a first convolution layer and a first residual connection in series; the multi-scale feature fusion residual state space block is used to extract and fuse features of different scales in the input feature map; the first convolution layer is used to adjust the number of channels or spatial size of the feature so that the dimension of the feature is consistent with the input feature map. Figure 1 The first residual connection is used to superimpose the adjusted features and the input features to obtain output features.

[0038] Furthermore, each multi-scale feature fusion residual state space block includes: a first normalization layer, a visual state space module, a second residual connection, a second normalization layer, a second convolutional layer, a dynamic channel attention mechanism and a third residual connection; the first normalization layer, the visual state space module and the second residual connection are connected in series, and the input end of the second residual connection is also connected to the input feature map; the output end of the second residual connection is respectively connected to the input end of the third residual connection and the input end of the second normalization layer, and the second normalization layer, the second convolutional layer, the dynamic channel attention mechanism and the third residual connection are connected in series.

[0039] Furthermore, the visual state space module includes two parallel branches, one of which is used for spatial information processing, including a first linear layer, a deep convolutional layer, a first SiLU activation function, a two-dimensional selective scanning module and a third normalization layer connected in series; the other branch is used for channel information processing, including a second linear layer and a second SiLU activation function connected in series; the results of the spatial information processing and the results of the channel information processing are fused by Hadamard product, and the fused results are then restored to the number of channels through the third linear layer.

[0040] Furthermore, the two-dimensional selective scanning module flattens the input features into a sequence according to the dimensions in the space, scans the sequence according to the set scanning direction, extracts multi-directional feature sequences, performs weighted fusion, and then reshapes them back into a two-dimensional feature form.

[0041] Furthermore, the refined reconstruction module includes: a feature alignment unit, a multi-source feature fusion unit, a reconstruction detail enhancement unit, an image reconstruction decoder and an output mapping and activation unit; the feature alignment unit is used to align the spatial size and match the number of channels of the original feature map and the deep denoising feature map; the multi-source feature fusion unit is used to fuse the original feature map and the deep denoising feature map with spatial size alignment and channel number matching; the reconstruction detail enhancement unit is used to extract high-level features on the fused features; the image reconstruction decoder is used to map the high-level features back to the image space; the output mapping and activation unit maps the high-level features mapped back to the image space to the image pixel space to obtain a noise-free image.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The adaptive local convolution enhancement module in the image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion dynamically adjusts the size, shape, and parameters of the convolution kernel according to the noise characteristic information on the image to enhance the detail recovery capability of the local area of the image. The adaptive local convolution enhancement module overcomes the local pixel forgetting defect of the traditional convolutional neural network by adaptively adjusting the receptive field of the convolution kernel, improves the sensitivity of local pixels, ensures that more detail information can be retained when processing images with different noise levels, and improves the denoising quality of the image;

[0044] (2) The long-term and short-term dependency modeling module based on beam scanning in the image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion of the present invention combines the long-term dependency modeling capability of the Mamba model and innovatively introduces a beam scanning unit that can scan the image from multiple directions, thereby more efficiently capturing the dependency relationship between distant pixels while maintaining the close correlation of local pixels, solving the problem of imbalance between global and local information in traditional methods, and further improving the denoising effect of the image, especially in high-resolution images with complex global structures;

[0045] (3) The multi-scale feature fusion residual state space block in the image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion of the present invention can solve the influence of noise of different scales on the image denoising effect, by extracting local and global features of the image at different scales, and effectively fusing information of different scales through residual connection; at the same time, the dynamic channel attention mechanism can automatically learn the weights of each channel, adaptively select key channels according to the noise characteristics of different channels in the image, reduce the computational overhead of irrelevant or redundant channels, effectively reduce computational redundancy, make the denoising process more efficient, improve the denoising effect, and ensure that important information is fully utilized. Through the collaborative work of multi-scale features, the present invention can handle various types of noise, especially multi-scale complex noise scenes, and achieve a good balance between global structure preservation and local detail recovery;

[0046] In summary, the present invention's image denoising device, based on adaptive local enhancement and dynamic multi-scale-dependent fusion, achieves a balance between image denoising accuracy and efficiency, making it suitable for denoising in a variety of complex scenes. Compared to existing technologies, this invention demonstrates superior image restoration capabilities and computational efficiency when processing high-resolution images and complex noisy scenes, possessing broad application prospects and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of an image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion according to the present invention;

[0048] Figure 2 Schematic diagram of the adaptive local convolution enhancement module in the present invention;

[0049] Figure 3 Schematic diagram of a long-term and short-term dependency modeling module based on beam scanning in the present invention;

[0050] Figure 4 Schematic diagram of the multi-scale feature fusion residual state space block in the present invention;

[0051] Figure 5 Schematic diagram of the visual state space module in the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be further explained below with reference to the accompanying drawings.

[0053] like Figure 1 Schematic diagram of an image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to the present invention, comprising: a data preprocessing module, an adaptive local convolution enhancement module, a long-term and short-term dependency modeling module based on beam scanning, a multi-level representation information extraction module, and a refined reconstruction module;

[0054] The data preprocessing module is used to perform pixel normalization, color space conversion, and size and resolution adjustment on the image to be denoised to obtain a standard image. This ensures the consistency of the standard image, thereby improving the generalization ability of the image denoising device in complex environmental noise and the computational efficiency during reasoning.

[0055] The adaptive local convolution enhancement module dynamically adjusts the receptive field of the convolution kernel according to the noise characteristics of the standard image and extracts the original feature map of the standard image. It can significantly improve the detail recovery ability of the local area of the image, thereby improving the denoising quality of the overall image.

[0056] The long-term and short-term dependency modeling module based on beam scanning uses beam scanning technology to capture the long-term and short-term dependencies on the original feature map, and fuses it with the standard image to obtain a preliminary denoised feature map;

[0057] The multi-level representation information extraction module is used to extract deep denoising feature maps of different scales from the preliminary denoising feature map;

[0058] The refined reconstruction module fuses and reconstructs the original feature map and the deep denoising feature map to output a noise-free image.

[0059] The present invention's image denoising device, based on adaptive local enhancement and dynamic multi-scale dependency fusion, achieves a balance between image denoising accuracy and efficiency, making it suitable for denoising in a variety of complex scenes. Compared to existing technologies, this invention demonstrates superior image restoration capabilities and computational efficiency when processing high-resolution images and complex noisy scenes, possessing broad application prospects and practical value.

[0060] To enhance the adaptability of the image denoising device of the present invention in diverse noise scenarios, an extended training dataset was constructed during the training phase. A variety of data augmentation strategies were implemented on the original dataset D, including the addition of various noise types, such as Gaussian, salt and pepper, and Poisson noise; geometric image transformations such as rotation, flipping, scaling, and cropping; and image color and brightness adjustments, thereby generating a rich set of training samples. Furthermore, advanced enhancement techniques such as cutout, mixup, and random erasure were utilized to simulate realistic and complex noise, further enhancing the robustness of the image denoising device. Furthermore, through data balancing and multi-scale training strategies, the device is able to fully adapt to various noise conditions, improving the denoising effect in practical applications.

[0061] During the inference stage of the image denoising device of the present invention, the data preprocessing module is designed to achieve efficient processing of the input image and ensure a balance between inference speed and denoising effect. First, the data preprocessing module reads and normalizes the input image, and supports color space and format conversion to ensure that the image meets the input requirements of the image denoising device. In order to improve resource utilization, the data preprocessing module converts the calculation accuracy to FP16 to significantly reduce video memory usage and computing overhead, thereby accelerating the inference process. In addition, the batch processing function can process multiple image samples at one time to achieve high throughput and fast response requirements, ensuring the efficiency and stability of the image denoising device in large-scale denoising tasks.

[0062] Through the above multi-level optimization design, the data preprocessing module not only ensures the image quality of the denoising task, but also significantly reduces the consumption of computing resources and enhances the speed and performance of the inference stage, making it suitable for a variety of application scenarios from embedded devices to high-performance computing platforms.

[0063] like Figure 2 The adaptive local convolution enhancement module in the present invention can dynamically adjust the size, shape and parameters of the convolution kernel according to the noise characteristics of the input image, significantly improving the detail recovery ability of the local area of the image, thereby improving the denoising quality of the overall image. It includes: a noise characteristic analysis unit, a dynamic convolution kernel generation unit, a convolution operation unit and a feature fusion unit.

[0064] The noise characteristic analysis unit analyzes the noise characteristics on the standard image and extracts information on the noise type, intensity and spatial distribution; specifically,

[0065] i. Use a classification network to identify the noise type in the input image and determine whether the noise is Gaussian, salt and pepper, Poisson, or other types. Analyze statistical characteristics such as mean, variance, skewness, and kurtosis to further confirm the specific type of noise.

[0066] ii. Quantify the noise intensity level by calculating the signal-to-noise ratio or peak signal-to-noise ratio of the noise image, apply the local noise estimation algorithm to measure the noise intensity in different areas of the image, and obtain the spatial distribution map of the noise intensity;

[0067] iii. Analyze the spatial distribution of noise in the image, identify areas where noise is concentrated and areas where noise is more dispersed, and use clustering algorithms to cluster the noise distribution to determine the noise characteristics of different areas.

[0068] The dynamic convolution kernel generation unit dynamically generates a convolution kernel adapted to the current noise environment based on the noise information extracted by the noise characteristic analysis unit; specifically,

[0069] i. Dynamically adjust the size of the convolution kernel according to the type and intensity of the noise. The greater the noise intensity, the larger the convolution kernel to cover different receptive fields, enhancing denoising capabilities and detail retention capabilities;

[0070] ii. Dynamically adjust the geometric shape of the convolution kernel according to the spatial distribution of noise. For example, an elliptical or diamond convolution kernel is used in areas with linear structures to more flexibly adapt to different structures and textures in the image, thereby more effectively capturing and restoring edge details and avoiding over-smoothing and detail loss.

[0071] iii. Through a learnable parameter adjustment mechanism, the weight parameters of the convolution kernel are optimized to make it more adaptable to the current noise environment, and the weight distribution of the convolution kernel is dynamically adjusted through the attention mechanism to enhance the responsiveness to key areas.

[0072] The convolution operation unit uses the generated convolution kernel to perform convolution operations on the standard image, which can adapt to the details and structures of different scales in the standard image, ensuring that the convolution kernel can effectively capture image details at different scales, achieving detail recovery and noise removal;

[0073] The feature fusion unit fuses the features after the convolution operation with the standard image, and adopts jump connection or residual connection to ensure the effective combination of detail information and structural information on the image. It integrates feature information at different levels through a multi-layer perception mechanism to improve the overall denoising effect. At the same time, it fuses the restored detail information at different scales and adopts a multi-scale feature fusion strategy to ensure the comprehensiveness and consistency of the denoising effect. By combining multi-scale feature maps, denoising features from different scales are integrated through weighted fusion or splicing to improve the overall denoising quality of the image, thereby obtaining the original feature map on the standard image.

[0074] The adaptive local convolution enhancement module of this invention significantly improves the ability to recover detail in local image regions by dynamically adjusting the size, shape, and parameters of the convolution kernel, overcoming the limitations of traditional convolutional neural networks when processing images with varying noise levels. The module boasts high flexibility and adaptability in its technical implementation, maintaining excellent denoising results in variable noise environments. This not only improves the quality and efficiency of image denoising, but also enhances the model's robustness and generalization capabilities, promising broad application prospects and significant market value.

[0075] In one technical solution of the present invention, the adaptive local convolution enhancement module adopts an end-to-end training method to automatically learn and optimize various parameters in the adaptive local convolution enhancement module through the back propagation algorithm, specifically including:

[0076] i. Comprehensively optimize the denoising effect by combining multiple loss functions, including: pixel-level loss function, perceptual loss function, structural similarity loss function, and adversarial loss function. The pixel-level loss function is used to ensure pixel-level denoising accuracy; the perceptual loss function is based on the feature layer of a pre-trained deep network to improve the perceptual quality of the denoised image; the structural similarity loss function ensures the similarity of the denoised image to the original image in structure, brightness, and contrast, maintaining the integrity of the image structure; the adversarial loss function, by combining the idea of a generative adversarial network, uses a discriminator network to improve the realism and detail richness of the denoised image;

[0077] ii. Optimize the parameters of the adaptive local convolution enhancement module through adaptive learning rate scheduling strategy and gradient clipping to ensure the stability and convergence of the adaptive local convolution enhancement module at different training stages;

[0078] iii. Regularization technology is introduced to prevent overfitting of the adaptive local convolution enhancement module, thereby improving its generalization ability on unseen data.

[0079] Through the above-mentioned optimization and adjustment scheme of the parameters of the adaptive local convolution enhancement module, the structure and parameters of the convolution kernel can be flexibly adjusted according to the noise characteristics of the input image, thereby efficiently restoring image details in different noise environments and significantly improving the image denoising quality.

[0080] like Figure 3 The beam-scanning-based long- and short-term dependency modeling module of the present invention includes: a beam scanning unit, a Mamba long-range dependency modeling unit, a short-range dependency modeling unit, a dependency fusion unit, and a feature integration unit. This beam-scanning-based long- and short-term dependency modeling module combines the long-range dependency modeling capabilities of the Mamba model and innovatively incorporates the beam scanning unit. By scanning the image from multiple directions, it can more efficiently capture the dependencies between distant pixels while maintaining the close correlation between local pixels, solving the problem of imbalance between global and local information in traditional methods. This module further improves the denoising effect of the image and is particularly suitable for high-resolution images with complex global structures.

[0081] The beam scanning unit dynamically adjusts the scanning beam according to the noise intensity, scans the original feature map in multiple directions and angles, and realizes efficient long-distance dependency capture to obtain scanning features. Specifically,

[0082] A. Based on the various structural and texture information in the original feature map, several scanning directions are designed, and a corresponding scanning path is generated for each scanning direction to cover the various structural and texture information in the original feature map. The scanning path is a sequence of pixels in the original feature map that traverses the original feature map in a specific direction with a fixed step length.

[0083] B. Dynamically adjust the scanning beam width and stride based on the noise intensity information experienced along the scanning path to adapt to different noise levels and structural complexities. The greater the noise intensity, the wider the scanning beam width and the smaller the scanning beam stride. The beam width determines the pixel range covered by each scanning path. The smaller the scanning beam stride, the higher the scanning accuracy, and the larger the stride, the faster the scanning speed.

[0084] C. Using the dynamically adjusted scanning beam width and stride, the original feature map is scanned along the scanning path to obtain the scanning features.

[0085] In one technical solution of the present invention, multi-core GPU parallel computing is utilized to perform multi-directional scanning simultaneously, thereby improving modeling efficiency and speed. Multi-threading and resource scheduling optimization are also implemented. Specifically, an independent thread or computing unit is allocated to each scanning direction to ensure that multi-directional scanning can be performed simultaneously, thereby reducing the overall computing time. Through dynamic resource scheduling, computing resources are reasonably allocated, computing bottlenecks are avoided, and the overall performance of parallel scanning is improved.

[0086] The Mamba long-range dependency modeling unit inputs the original feature map and the scanned features into the long-range dependency modeling unit. Through the multi-directional beam scanning features, it enhances the ability to capture the dependency relationship between long-distance pixels in the original feature map, thereby capturing global features.

[0087] The short-range dependency modeling unit is used to capture the dependencies between short-range pixels in the original feature map and maintain the close correlation of details. Specifically, a convolutional neural network or a local self-attention mechanism is used to extract features from local areas of the image to capture short-range dependencies.

[0088] The dependency fusion unit uses the attention mechanism to weightedly fuse the dependencies between long-distance pixels and short-distance pixels to obtain fused dependency information;

[0089] The feature integration unit fuses the fusion-dependent information with the standard image using skip connections or residual connections to ensure the comprehensive transmission of information and prevent information loss during the transmission process; then, deconvolution and upsampling are used to restore the spatial size and generate a high-resolution denoising feature map. A detail enhancement module is added to improve the image detail retention capability. High-frequency information extraction and detail restoration further enhance the image detail retention capability, and a preliminary denoising feature map is generated through convolution layers and activation functions.

[0090] The beam-scanning-based long- and short-term dependency modeling module of this invention significantly improves the ability to capture global dependencies in image denoising by combining beam-scanning technology with the Mamba model, resolving the imbalance between global and local information in traditional methods. This module is highly flexible and adaptable, maintaining excellent denoising results in complex and variable noise environments and high-resolution image processing. It improves the quality and efficiency of image denoising, while enhancing the robustness and generalization capabilities of image denoising devices, and has broad application prospects and market value.

[0091] The multi-level representation information extraction module in the present invention is designed to efficiently extract the multi-scale, multi-directional and global features of the preliminary denoising feature map, and realize the accurate modeling and removal of complex noise. It is composed of several multi-scale feature fusion residual state space group modules, each of which is composed of several multi-scale feature fusion residual state space blocks, a first convolution layer and a first residual connection in series; the multi-scale feature fusion residual state space block is used to extract and fuse features of different scales in the input feature map; the first convolution layer is used to adjust the number of channels or spatial size of the feature to make the dimension of the feature consistent with the input feature map. Figure 1 The first residual connection is used to superimpose the adjusted features and input features, retain the original information, and promote the effective transfer of gradients to obtain output features.

[0092] like Figure 4 Each multi-scale feature fusion residual state space block includes: a first normalization layer, a visual state space module, a second residual connection, a second normalization layer, a second convolutional layer, a dynamic channel attention mechanism and a third residual connection; the first normalization layer, the visual state space module and the second residual connection are connected in series, and the input end of the second residual connection is also connected to the input feature map; the output end of the second residual connection is respectively connected to the input end of the third residual connection and the input end of the second normalization layer, the second normalization layer, the second convolutional layer, the dynamic channel attention mechanism and the third residual connection are connected in series, among which the first normalization layer and the second normalization layer can stabilize the distribution of features; the double residual connection can ensure that information in the deep network is not lost and enhance the transmission efficiency of features; the visual state space module uses the state space equation to capture long-distance dependencies globally and enhance the global feature representation; the dynamic channel attention mechanism can adaptively learn the weights of each channel, highlight key features, suppress redundant information and reduce computational overhead.

[0093] like Figure 5The visual state space module contains two parallel branches, one of which is used for spatial information processing, including the first linear layer, deep convolution layer, first SiLU activation function, two-dimensional selective scanning module and third normalization layer connected in series; the other branch is used for channel information processing, including the second linear layer and second SiLU activation function connected in series; the results of spatial information processing are fused with the results of channel information processing by Hadamard product, and then the fused results are restored to the number of channels through the third linear layer, thereby enhancing the expressive ability of features.

[0094] The two-dimensional selective scanning module flattens the input features into a sequence according to the spatial dimensions, scans the sequence according to the set scanning direction, and extracts multi-directional feature sequences. It can make full use of the two-dimensional spatial information of the image, capture structural information and texture features in different directions, thereby enhancing the perception of global features, and then perform weighted fusion to reshape it back into a two-dimensional feature form.

[0095] The multi-level representation information extraction module of the present invention significantly improves the feature extraction and dependency modeling capabilities in image denoising. The module is highly flexible and scalable, and can maintain excellent denoising effects in complex and changeable noise environments and high-resolution image processing, improving the quality and efficiency of image denoising, while enhancing the robustness and generalization ability of the model. It has broad application prospects and significant market value.

[0096] The refined reconstruction module in this paper is designed to perform high-quality fusion and refined reconstruction of features from the adaptive local convolution enhancement module and the multi-level representation information extraction module, generating a clean, sharp, and noise-free output image. By combining multiple feature fusion strategies and reconstruction techniques, the denoising process is highly efficient and the results are of high quality. The refined reconstruction module includes: a feature alignment unit, a multi-source feature fusion unit, a reconstruction detail enhancement unit, an image reconstruction decoder and an output mapping and activation unit; the feature alignment unit is used to align the spatial dimensions of the original feature map and the deep denoising feature map by upsampling and downsampling operations, and adjust the number of channels of the original feature map and the deep denoising feature map through 1×1 convolution to ensure that the features from different sources are consistent in the number of channels, thereby ensuring the uniformity and compatibility of the features; the multi-source feature fusion unit is used to fuse the original feature map and the deep denoising feature map with aligned spatial dimensions and matched channel numbers. Specifically, learnable weights are generated through global average pooling and fully connected layers to perform weighted fusion of features from different sources, highlight key features, suppress redundant information, and add or concatenate the weighted features element by element to form a comprehensive feature representation; the reconstruction detail enhancement unit is used to extract high-level features from the fused features. Specifically, a deep convolutional network is used to extract high-level features through multi-layer convolution operations to enhance image detail information. Information, residual connections are introduced between convolutional layers to ensure the effective transmission of feature information and prevent information loss, and a dynamic channel attention mechanism is applied to further optimize feature representation and improve detail recovery capabilities; the image reconstruction decoder is used to map high-level features back to the image space. Specifically, sub-pixel convolution technology is used to upsample high-level features to high resolution and restore the spatial size of the image. Then, through multi-layer convolution operations, the details and structure of the image are gradually restored to ensure that the reconstructed image maintains high resolution and details; the output mapping and activation unit maps the high-level features mapped back to the image space to the image pixel space. Specifically, 1×1 convolution is used to map the reconstructed image to the required number of channels to generate the final image features. The activation function is used to limit the pixel value range, improve the details and texture quality of the denoised image, ensure that the pixel values of the output image are within the legal range, normalize the output image, and convert the color space from the internal processing format back to the original format to ensure the visual consistency and color accuracy of the output image and obtain a noise-free image.

[0097] In one technical solution of the present invention, the feature fusion part further includes:

[0098] i. By integrating the feature pyramid network, we can fuse dependencies at different scales and ensure the effective integration of multi-level information;

[0099] ii. Introducing a learnable weight mechanism to dynamically adjust weight distribution based on the strength of dependencies. At the same time, it adjusts the weights of long- and short-range dependencies based on the strength of regional dependencies, and combines this with an attention mechanism to achieve fine-grained adjustments.

[0100] iii. Use Dropout and L2 regularization to prevent overfitting and improve the generalization ability of the image denoising device. Through sparse connections and structural constraints, ensure the rationality and effectiveness of dependency modeling.

[0101] In the real-world image denoising task, the present invention used 320 high-resolution images from the SIDD dataset as training data. These images capture complex, real-world noise patterns, providing the image denoising device with learning material specific to real-world noise, enabling it to adapt to the real-world noise distribution in different scenarios. During the testing phase, the SIDD test set and the DND dataset were used to comprehensively evaluate the denoising device's performance in removing real-world noise. The test results demonstrated that the denoising capability of the present image denoising device accurately reflects the structural features of the image, demonstrating its strong denoising capabilities.

[0102] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. An image denoising device based on adaptive local enhancement and dynamic multi-scale dependent fusion, characterized in that: include: Data preprocessing module, adaptive local convolution enhancement module, beam scanning-based long-term and short-term dependency modeling module, multi-level representation information extraction module, and refined reconstruction module; The data preprocessing module is used to perform pixel normalization, color space conversion, and size and resolution adjustment on the image to be denoised to obtain a standard image; The adaptive local convolution enhancement module dynamically adjusts the receptive field of the convolution kernel according to the noise characteristics of the standard image and extracts the original feature map of the standard image; The beam scanning-based long-term and short-term dependency modeling module uses beam scanning technology to capture the long-term and short-term dependencies on the original feature map, and fuses it with the standard image to obtain a preliminary denoised feature map; The multi-level representation information extraction module is used to extract deep denoising feature maps of different scales in the preliminary denoising feature map; The refined reconstruction module fuses and reconstructs the original feature map and the deep denoising feature map, and outputs a noise-free image.

2. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 1, characterized in that: The adaptive local convolution enhancement module includes: a noise characteristic analysis unit, a dynamic convolution kernel generation unit, a convolution operation unit and a feature fusion unit; The noise characteristic analysis unit analyzes the noise characteristics on the standard image and extracts information on the noise type, intensity and spatial distribution; The dynamic convolution kernel generation unit dynamically generates a convolution kernel adapted to the current noise environment based on the noise information extracted by the noise characteristic analysis unit; The convolution operation unit performs a convolution operation on the standard image using the generated convolution kernel; The feature fusion unit fuses the features after the convolution operation with the standard image to obtain the original feature map on the standard image.

3. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 2, characterized in that: The dynamic generation process of the convolution kernel includes: i. Dynamically adjust the size of the convolution kernel according to the type and intensity of the noise. The greater the noise intensity, the larger the convolution kernel. ii. Dynamically adjust the geometry of the convolution kernel according to the spatial distribution of the noise; iii. Optimize the weight parameters of the convolution kernel through a learnable parameter adjustment mechanism, and dynamically adjust the weight distribution of the convolution kernel through the attention mechanism.

4. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 2, characterized in that: The long-term and short-term dependency modeling module based on beam scanning includes: a beam scanning unit, a Mamba long-term dependency modeling unit, a short-term dependency modeling unit, a dependency fusion unit and a feature integration unit; The beam scanning unit dynamically adjusts the scanning beam according to the noise intensity, scans the original feature map, and obtains the scanning feature; The Mamba long-range dependency modeling unit inputs the original feature map and the scanned features into the long-range dependency modeling unit to capture the dependency relationship between long-range pixels in the original feature map; The short-range dependency modeling unit is used to capture the dependency relationship between short-range pixels in the original feature map; The dependency fusion unit uses the attention mechanism to weightedly fuse the dependency between long-distance pixels and the dependency between short-distance pixels to obtain fused dependency information; The feature integration unit performs feature fusion on the fusion dependency information and the standard image to obtain a preliminary denoising feature map.

5. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 4, characterized in that: The specific process of the beam scanning unit scanning the original feature map is as follows: A. Design several scanning directions based on various structural and texture information on the original feature map, and generate a corresponding scanning path for each scanning direction; B. Dynamically adjust the width and stride of the scanning beam based on the noise intensity information experienced along the scanning path. The greater the noise intensity, the wider the scanning beam width and the smaller the scanning beam stride. C. Using the dynamically adjusted scanning beam width and stride, the original feature map is scanned along the scanning path to obtain the scanning features.

6. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 4, characterized in that: The multi-level representation information extraction module is composed of several multi-scale feature fusion residual state space group modules, each of which is composed of several multi-scale feature fusion residual state space blocks, a first convolutional layer and a first residual connection in series; the multi-scale feature fusion residual state space block is used to extract and fuse features of different scales in the input feature map; the first convolutional layer is used to adjust the number of channels or spatial size of the feature so that the dimension of the feature is consistent with the input feature map; the first residual connection is used to superimpose the adjusted features and the input features to obtain output features.

7. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 6, characterized in that: Each multi-scale feature fusion residual state space block includes: a first normalization layer, a visual state space module, a second residual connection, a second normalization layer, a second convolutional layer, a dynamic channel attention mechanism and a third residual connection; the first normalization layer, the visual state space module and the second residual connection are connected in series, and the input end of the second residual connection is also connected to the input feature map; the output end of the second residual connection is respectively connected to the input end of the third residual connection and the input end of the second normalization layer, and the second normalization layer, the second convolutional layer, the dynamic channel attention mechanism and the third residual connection are connected in series.

8. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 7, characterized in that: The visual state space module includes two parallel branches, one of which is used for spatial information processing, including a first linear layer, a deep convolutional layer, a first SiLU activation function, a two-dimensional selective scanning module and a third normalization layer connected in series; the other branch is used for channel information processing, including a second linear layer and a second SiLU activation function connected in series; the results of the spatial information processing are fused with the results of the channel information processing by Hadamard product, and the fused results are then restored to the number of channels through the third linear layer.

9. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 8, characterized in that: The two-dimensional selective scanning module flattens the input features into a sequence according to the dimensions in the space, scans the sequence according to the set scanning direction, extracts multi-directional feature sequences, performs weighted fusion, and then reshapes them back into a two-dimensional feature form.

10. The image denoising device based on adaptive local enhancement and dynamic multi-scale dependency fusion according to claim 6, characterized in that: The refined reconstruction module includes: a feature alignment unit, a multi-source feature fusion unit, a reconstruction detail enhancement unit, an image reconstruction decoder and an output mapping and activation unit; the feature alignment unit is used to align the spatial dimensions and match the number of channels of the original feature map and the deep denoising feature map; the multi-source feature fusion unit is used to fuse the original feature map and the deep denoising feature map with spatial dimension alignment and channel number matching; the reconstruction detail enhancement unit is used to extract high-level features on the fused features; the image reconstruction decoder is used to map the high-level features back to the image space; the output mapping and activation unit maps the high-level features mapped back to the image space to the image pixel space to obtain a noise-free image.

Images