A method and system for deblurring Mamba images with nonlocal information compensation

Abstract

This application proposes a nonlocal information compensation-based Mamba image deblurring method and system, relating to the field of computer vision technology. The method includes: extracting features from a degraded image containing blurred features to obtain initial shallow features; using the initial shallow features as input parameters, performing n iterations of first-level processing, followed by deblurring to obtain a first restored feature map; the first-level processing includes deblurring and downsampling; the deblurring is performed using an improved Mamba module and an enhanced FFN module; performing n iterations of second-level processing on the first restored feature map, followed by deblurring to obtain a second restored feature map; the second-level processing includes deblurring and upsampling; convolving the second restored feature map to generate a residual feature map, which is then added to the degraded image to obtain the restored image. This application's solution achieves high-quality restoration of blurred images while effectively controlling computational and storage overhead.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for deblurring Mamba images using non-local information compensation. Background Technology

[0002] Image deblurring aims to recover a sharp image from a blurred one, and this task is receiving widespread attention due to the proliferation of various cameras and handheld imaging devices. This task is extremely challenging because, given only a blurred image, it's impossible to accurately determine the specific mechanism of blur formation or directly obtain the potentially sharp image. Blurring often originates from complex and diverse sources, such as camera shake, object motion, defocusing, and atmospheric turbulence. Different types of blur may also be coupled, making the image degradation process highly ill-posed, which places higher demands on restoration algorithms.

[0003] In recent years, convolutional neural networks (CNNs) and visual Transformers have achieved remarkable results in image restoration. CNNs excel in feature extraction, but their limitations in capturing long-range dependencies and non-local structures are due to the local receptive field of the convolutional kernel. Visual Transformers, with their self-attention mechanism, can better model long-range dependencies and the non-local characteristics of the input content, thus showing great potential in image restoration tasks. However, the computational complexity of visual Transformer schemes increases quadratically with image resolution, resulting in extremely high computational and storage costs for high-resolution images, making it difficult to meet the needs of practical applications. As the precision of modern imaging equipment continues to improve, the resolution of acquired images is becoming increasingly higher, causing Transformer models to face significant performance bottlenecks and memory consumption problems when processing large-scale high-resolution images. Summary of the Invention

[0004] In view of this, this application proposes a non-local information compensation method and system for deblurring Mamba images.

[0005] Firstly, this application provides a non-local information compensation method for deblurring Mamba images, including:

[0006] Convolutional layers are used to extract features from degraded images containing blurred features to obtain initial shallow features;

[0007] The initial shallow features are used as input parameters, and the first iteration is performed n times. The features after the nth first iteration are then deblurred to obtain a first restored feature map. The first iteration includes deblurring and downsampling the input parameters sequentially. The deblurring is accomplished by an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a first nonlocal compensation branch and a selective SSM branch set in parallel and incorporating a selective scanning mechanism. Before entering the selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module. The enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel. The second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights.

[0008] The first restored feature map is used as the input parameter, and the second iteration is performed n times. The feature after the nth second iteration is then deblurred to obtain the second restored feature map. The second iteration includes: deblurring and upsampling the input parameter in sequence.

[0009] The second restored feature map is processed by a convolutional layer to generate a residual feature map, and the residual feature map is added to the degraded image to obtain the restored image.

[0010] In one embodiment, the improved Mamba module further includes a first feature enhancement module and a normalization processing module;

[0011] The feature enhancement module is used to perform a linear mapping operation on the input features, adjust the original channel dimension to a preset extended dimension, introduce depthwise separable convolution to perform convolution calculation on each channel independently, map the features after convolution calculation through a non-linear activation function, and output the mapped features to the first non-local compensation branch and the selective SSM branch; the feature enhancement module is also used to output the modulated features after non-linear activation of the features after the linear mapping operation to the normalization processing module.

[0012] The normalization processing module is used to fuse the output features of the first nonlocal compensation branch and the selective SSM branch, perform layer normalization processing, and interact the layer normalized features with the modulation features element by element. The feature dimension of the interacted features is compressed back to the original number of channels through a linear mapping layer.

[0013] In one embodiment, the first nonlocal compensation branch includes: a frequency domain conversion module, a Gaussian low-pass filter module, and a spatial domain conversion module;

[0014] The frequency domain conversion module is used to convert the input features from the spatial domain to the frequency domain representation to obtain frequency domain features;

[0015] The Gaussian low-pass filter module is used to modulate the frequency domain features to obtain modulated nonlocal features;

[0016] The spatial domain transformation module is used to restore the spatial domain representation of the features after low-pass filtering through inverse Fourier transform.

[0017] In one embodiment, the enhanced FFN module further includes: a second feature enhancement module, a fusion module, and a dual-branch frequency domain enhancement module;

[0018] The second feature enhancement module is used to perform pointwise feature mapping on the input features to adjust the channel dimension and enhance the feature expressive power, and to use depthwise separable convolution to perform local feature extraction, introducing local spatial information separately for each channel to obtain enhanced local features;

[0019] The fusion module is used to segment the enhanced local features along the channel dimension. After the two segments are processed by a nonlinear activation function, one feature is used as a gate signal and multiplied element-wise with the other feature. The two features are then spliced ​​together along the channel direction to form the fused output feature.

[0020] The dual-branch frequency domain enhancement module is used to process the fused output features using convolutional layers, and then divides the processed features into sub-blocks in the spatial dimension and inputs them in parallel into the second nonlocal compensation branch and the frequency domain discriminant branch. The outputs of the two branches are subjected to inverse Fourier transform to return to the spatial domain, and the sub-blocks in the spatial domain are sequentially spliced ​​back to the original size. The spliced ​​results of the two branches are then added and fused.

[0021] In one embodiment, the input features in the frequency domain discriminative branch are directly multiplied element-wise with the learnable weights to discriminate and retain the target frequency domain information; the second nonlocal compensation branch first extracts nonlocal information through a low-pass filter, and then multiplies it with the learnable weights.

[0022] In one embodiment, the cutoff frequency ratio in the Gaussian low-pass filter module is a learnable parameter.

[0023] In one embodiment, the number of deblurring processes corresponds to the same number for the i-th first iteration and the n-i+1-th second iteration.

[0024] In one embodiment, the first iterative process includes: performing multiple deblurring processes on the input parameters sequentially, and downsampling the features after the multiple deblurring processes.

[0025] In one embodiment, the output feature of the i-th first iteration is used as the skip connection feature, which is used to fuse with the input feature in the (n-i+1)-th level second iteration.

[0026] Secondly, this application also provides a Mamba image deblurring system with non-local information compensation, including: a feature extraction module, an encoder module, a decoder module and an image reconstruction module;

[0027] The feature extraction module is used to extract features from the degraded image containing blurred features using a convolutional layer to obtain initial shallow features;

[0028] The encoder module includes n cascaded encoder layers and a deblurring module. The encoder layers are used to perform a first iterative processing on the input features. The encoder module is used to process the initial shallow features to obtain a first recovered feature map. The first iterative processing includes: sequentially deblurring and downsampling the input parameters. The deblurring is accomplished by an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a first nonlocal compensation branch and a selective SSM branch set in parallel and incorporating a selective scanning mechanism. Before entering the selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module. The enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel. The second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights.

[0029] The decoder module includes n cascaded decoder layers and a deblurring module. The decoder layers are used to perform a second iterative processing on the input features. The decoder module is used to process the first recovered feature map to obtain a second recovered feature map. The second iterative processing includes: sequentially performing deblurring and upsampling on the input features.

[0030] The image reconstruction module is used to process the second restored feature map using a convolutional layer to generate a residual feature map, and then adds the residual feature map to the degraded image to obtain the restored image.

[0031] The nonlocal information compensation-based Mamba image deblurring method proposed in this application has the following advantages over related techniques:

[0032] 1. The Mamba image deblurring method of this application extracts initial shallow features through convolutional layers, and then achieves efficient modeling while ensuring linear complexity through n iterations of the first iteration, which includes deblurring and downsampling operations. The downsampling operation effectively reduces the spatial dimension of the high-resolution image, and the improved linear complexity of the Mamba module in the deblurring process alleviates the performance bottleneck and memory consumption problem of the visual Transformer's computational complexity increasing with the square of the resolution from the source. Then, the image resolution is gradually restored through n iterations of the second iteration, which includes deblurring and upsampling operations. The reconstruction error is further corrected by adding and fusing the residual feature map and the degraded image. Finally, high-quality restoration of the blurred image is achieved while effectively controlling the computational and storage overhead, so that the restored image has both clear local details and good structural consistency.

[0033] 2. Deblurring is achieved through an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a parallel first nonlocal compensation branch and a selective SSM branch. Supported by the selective scanning mechanism, the parameters of the first nonlocal compensation branch can be learned by the Gaussian low-pass filter module to explicitly and adaptively extract nonlocal features at different scales, and then perform targeted compensation and enhancement. At the same time, the selective SSM branch ensures the capture of local information, which not only enhances the model's perception of nonlocal structures and context modeling ability, but also overcomes the shortcomings of traditional methods in utilizing nonlocal information insufficiently. The second branch of the enhanced FFN module... The nonlocal compensation branch extracts nonlocal information in the frequency domain through adaptive low-pass filtering and dynamically modulates it with learnable discriminative weights. This not only suppresses high-frequency noise interference but also works in conjunction with the frequency domain discriminative branch to balance local details and nonlocal structures, solving the problems of traditional methods being sensitive to high-frequency interference and struggling to balance the two types of information. Therefore, through deblurring, the Mamba image deblurring method of this application can explicitly extract and compensate for nonlocal features, while effectively balancing nonlocal structure enhancement and local detail restoration, avoiding overfitting of the model to local information, and significantly improving the image restoration quality in complex blurred scenes. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating a Mamba image deblurring method with non-local information compensation in one embodiment of this application.

[0036] Figure 2 This is a schematic diagram of the deblurring process in one embodiment of this application;

[0037] Figure 3 This is a schematic diagram of the network architecture corresponding to the Mamba image deblurring method in one embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the processing flow of the improved Mamba module in one embodiment of this application;

[0039] Figure 5 This is a schematic diagram of the processing flow of the enhanced FFN module in one embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the adaptive low-pass filtering process in the first non-local compensation branch in one embodiment of this application;

[0041] Figure 7 This is a schematic diagram of the structure of a Mamba image deblurring system with non-local information compensation in one embodiment of this application. Detailed Implementation

[0042] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0043] In some embodiments, such as Figure 1 As shown, this application provides a non-local information compensation method for deblurring Mamba images, which includes the following steps S101 to S104.

[0044] S101: Use convolutional layers to extract features from degraded images with blurred features to obtain initial shallow features.

[0045] The initial shallow feature extraction stage uses a 3×3 convolutional layer to project the input image, mapping the number of channels to a specified feature dimension while maintaining the input spatial resolution. This stage primarily captures the image's basic texture and edge information, serving as input for subsequent deep feature extraction.

[0046] S102: Using the initial shallow features as input parameters, perform n iterations of the first iteration processing, and then deblur the features after the nth iteration to obtain the first restored feature map. The first iteration processing includes: sequentially deblurring and downsampling the input parameters; the deblurring is accomplished by an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a first nonlocal compensation branch and a selective SSM (State Space Model) branch set in parallel and incorporating a selective scanning mechanism. Before entering the selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module. The enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel. The second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights.

[0047] The deblurring process can be described as follows: Figure 2 As shown, this can be accomplished sequentially by improving the Mamba module and enhancing the FFN module. The cutoff frequency ratio in the Gaussian low-pass filter module can be a learnable parameter. By setting the cutoff frequency ratio in the low-pass filter module as a learnable parameter, it can be automatically optimized and adjusted through gradient backpropagation and parameter update mechanisms during neural network training, thereby adaptively adjusting the frequency retention range and smoothing intensity. The creation process of the Gaussian low-pass filter includes: calculating the distance from each point in the frequency domain to the center based on the input feature size, determining the standard deviation of the Gaussian function according to the learnable cutoff frequency ratio, and generating a Gaussian filter mask. The filtering process of the Gaussian low-pass filter module can include: performing a fast Fourier transform on the input image and performing frequency domain centering, multiplying the generated Gaussian low-pass filter with the frequency domain feature map to achieve frequency domain filtering, and finally restoring the spatial domain feature map through an inverse Fourier transform.

[0048] It is understandable that the improved Mamba module, based on the traditional Mamba structure, sets up a parallel nonlocal compensation branch to enhance the modeling capability of nonlocal information. Before entering the selective scanning mechanism, the branch first performs frequency domain modulation on the input features through an adaptive Gaussian low-pass filter to suppress local high-frequency interference and highlight the nonlocal low-frequency structure. Then, it uses the selective scanning mechanism for recursive updates and enhancements, compensating for the deficiency of low utilization of nonlocal context information in the traditional Mamba backbone, enhancing the ability to describe nonlocal relationships of the overall features, and realizing the effective fusion of local and nonlocal information. This allows for the modeling of long-range dependencies and the enhancement of nonlocal information, providing high-quality feature representations for the subsequent FFN module.

[0049] The enhanced FFN module addresses the shortcomings of traditional FFN modules, which rely solely on spatial domain convolution or pointwise mapping and struggle to simultaneously model both local details and non-local information. By designing a branch in the dual-branch frequency domain enhancement structure of the FFN module to extract non-local information in the frequency domain using an adaptive low-pass filter, and then multiplying it by learnable discriminative weights for modulation, it suppresses high-frequency noise interference while enhancing useful non-local structural information. This is then fused with the local branch, achieving collaborative modeling of local details and non-local structural information, significantly improving the detail clarity and structural consistency of the reconstruction results. Based on this, deblurring can enhance the representation of non-local information while preserving local details. Combining deblurring with downsampling during the iteration process can gradually reduce the spatial size of the feature map, expanding the receptive field and extracting contextual information at different scales, thereby obtaining multi-level, multi-scale deep features and providing effective support for subsequent decoding and recovery processes.

[0050] It should be noted that the downsampling operation can be achieved through the following steps: First, the spatial size of the feature map is reduced to half by using bilinear interpolation. This interpolation method maintains the smoothness and continuity of the image by weighted averaging of adjacent pixels, avoiding jagged edges or artifacts caused by size changes. Then, the number of feature channels is doubled through a convolutional layer.

[0051] S103: Using the first restored feature map as input parameters, perform n second iterations, and deblur the features after the nth second iteration to obtain the second restored feature map. The second iteration process includes: sequentially deblurring and upsampling the input parameters.

[0052] It is understandable that the second iteration includes deblurring and upsampling operations. Therefore, the progressive upsampling operations of n second iterations can gradually restore the spatial resolution of the feature map. During the spatial resolution restoration process, the restored decoded features are processed layer by layer through deblurring, allowing the features generated during decoding to be progressively enhanced at multiple stages. This reduces blurring effects while maintaining the integrity of texture information and structural features. Through n second iterations, the input features can be enhanced to highlight structural information and restore detailed features, providing stable and high-quality feature support for the final image reconstruction.

[0053] In application, the output features of the first iteration (i-th stage) can be used as skip connection features. These skip connection features are then fused with the input features of the second iteration (n-i+1-th stage). This establishes skip connections with the feature maps from the first iteration after each upsampling stage, thereby achieving the fusion of shallow detail information and deep semantic information. It should be noted that the value of i ranges from 1 to n.

[0054] It should also be noted that the upsampling operation can be achieved through the following steps: First, the spatial size of the feature map is doubled using bilinear interpolation. This interpolation method uses the weighted average of adjacent pixels to generate new pixel values, thereby effectively maintaining the smooth transition of the image structure during the magnification process. Then, the number of feature channels is reduced by half through a convolutional layer.

[0055] S104: The second restored feature map is processed by a convolutional layer to generate a residual feature map, and the residual feature map is added to the degraded image to obtain the restored image.

[0056] In the application, the second restored feature map can be processed by a convolutional layer with a kernel size of 3×3 to generate a residual image of the same size as the degraded image. Then, by adding the residual image pixel-by-pixel to the input blurred image, a high-quality deblurred image with complete detail restoration and sharp edges is finally obtained. The network architecture corresponding to the non-local information compensation Mamba image deblurring method of this application can be as follows: Figure 3 As shown, a symmetrical U-shaped network architecture based on a hierarchical encoder-decoder can be adopted.

[0057] The aforementioned Mamba image deblurring method extracts initial shallow features using convolutional layers. Then, through n iterations of deblurring and downsampling operations, it achieves efficient modeling while maintaining linear complexity. Downsampling effectively reduces the spatial dimension of high-resolution images. Combined with improvements to the linear complexity of the Mamba module in the deblurring process, it alleviates the performance bottleneck and memory consumption problem of the visual Transformer's computational complexity increasing with the square of resolution. Subsequently, a second iteration of n iterations, including deblurring and upsampling operations, gradually restores the image resolution. The addition and fusion of residual feature maps and degraded images further corrects reconstruction errors. Ultimately, while effectively controlling computational and storage overhead, it achieves high-quality restoration of blurred images, ensuring that the restored images possess both clear local details and good structural consistency.

[0058] In some embodiments, the improved Mamba module further includes a first feature enhancement module and a normalization processing module.

[0059] The feature enhancement module is used to perform linear mapping operations on the input features, adjust the original channel dimension to the preset extended dimension, introduce depthwise separable convolution to perform convolution calculations on each channel independently, map the features after convolution calculation through a non-linear activation function, and output the mapped features to the first non-local compensation branch and the selective SSM branch; the feature enhancement module is also used to output the modulated features after non-linear activation of the features after linear mapping operations to the normalization processing module.

[0060] The normalization module is used to fuse the output features of the first nonlocal compensation branch and the selective SSM branch, perform layer normalization, and then interact the layer-normalized features with the modulation features element by element. The feature dimension of the interacted features is compressed back to the original number of channels through a linear mapping layer.

[0061] Improving the processing flow of Mamba modules can be achieved as follows: Figure 4 As shown, during the processing of the feature enhancement module, the input features... First, a linear transformation is performed. Reduce the number of channels from the original C Expand to 4C To enhance the expressive power of features and the flexibility of subsequent processing, the expanded features are divided equally along the channel dimension, denoted as follows: and This process can be represented as:

[0062]

[0063] Subsequently, the obtained features Input to a 3×3 depth separable convolutional layer To extract local spatial information and through The activation function performs a nonlinear mapping, thereby enhancing the expressive power and nonlinear modeling ability of the features, resulting in an updated feature representation:

[0064]

[0065] This step not only expands the channels of the input features but also captures spatial context information using depthwise convolution and enhances the network's nonlinear modeling capabilities under the action of the activation function, thus laying the foundation for subsequent feature integration. This lays the foundation for interaction and integration.

[0066] The feature enhancement module will process the features When input into the parallel branching structure of the improved Mamba module, one of the branches retains its original characteristics. No processing is performed to preserve complete local information. Since the nonlocal structures are scattered and the signal is weak, to avoid overfitting local information by the selective SSM branch, another branch uses a parameter-learnable Gaussian low-pass filter module to perform frequency domain modulation on the features. This suppresses local high-frequency interference and highlights the nonlocal low-frequency structures, thereby adaptively extracting nonlocal features at different scales for subsequent targeted compensation of nonlocal information. Subsequently, the low-pass filtered features are restored to their spatial domain representation through inverse Fourier transform to obtain the nonlocal features. Nonlocal features Parallel processing with the original feature branch is performed to effectively handle both local and non-local information, providing enhanced feature representations for subsequent dynamic parameter calculations and feature updates. This process can be represented as:

[0067]

[0068] It's important to note that in the improved Mamba module, the parameters are not static but calculated based on the input sequence. This dynamic calculation allows the improved Mamba module to adapt its behavior according to the specific circumstances of the input sequence, thus providing more refined data understanding and processing. (State transition matrix) and jump connection parameters Set as learnable dynamic parameters, and then according to and Three parameters are generated by passing through a linear layer: time step. Input projection matrix and output projection matrix For the original feature branch, a one-dimensional convolution with a kernel size of 7 is set after the linear layer to introduce local spatial context when generating parameters, enhancing the model's ability to perceive the multi-dimensional spatial features of the input sequence. The two parallel branches employ independent parameter generation paths, which can be represented as:

[0069]

[0070]

[0071] Wherein, the state transition matrix Used to control the recursive process of the hidden state between time steps; jump connection parameters Used to preserve the direct contribution of the input during the update process, ensuring the stability of information flow; input projection matrix Used to map input features to the latent state space; output projection matrix Used to transform hidden states into output features; time step This is used to adjust the rate and granularity of state updates, thereby enabling adaptive modeling of different input sequences.

[0072] After obtaining the above parameters, the state update function is used. The features are dynamically evolved. The state update function S6 is implemented based on a selective scan mechanism, the core idea of ​​which is to utilize parameters. The hidden states of the input sequence are recursively updated to preserve local dependencies while modeling long-range relationships. Then, the output features of the two branches are processed. and The fusion is achieved by summing element by element, and the fusion characteristics are obtained. The specific form is as follows:

[0073]

[0074]

[0075]

[0076] The state update function is essentially a recursive state evolution process. Specifically, it takes a sequence as input. Based on the hidden state of the previous position and current input Recursively calculate the hidden state at the current position. Then, a linear transformation is performed based on the hidden state at the current position and the current input to obtain the output at the current position. Its mathematical form can be expressed as:

[0077]

[0078]

[0079] Where parameters and Discretization is performed using the zero-order preserve (ZOH) rule, and a time-scale parameter is introduced. Specifically, this is achieved through the following formula:

[0080]

[0081] Therefore, the output not only contains the long-term accumulated dependency features of the hidden state, but also retains the direct response of the input, effectively balancing the modeling ability of local information and long-range relationships. By introducing a selective scanning mechanism, the state update function... It can adaptively adjust the contributions of different channels and time steps during sequence propagation, thereby taking into account both non-local dependencies and local details, and achieving efficient dynamic evolution and expression enhancement of input features.

[0082] Since blur typically exhibits global diffusion characteristics in space, the correlation between non-local regions plays a crucial role in the reconstruction of a sharp image. However, local convolution or linear scanning struggles to explicitly capture such relationships, necessitating the separate extraction and modeling of non-local information. This application's improved Mamba module effectively compensates for the main structure's insufficient utilization of non-local contextual information by introducing a first non-local compensation branch.

[0083] After the first nonlocal compensation branch and the selective SSM branch output the corresponding features to the normalization processing module, the normalization processing module fuses the output features and updates the fused features. First perform layer normalization , and features After nonlinear activation The results are then multiplied element-wise to enhance the non-linear interaction between channels. Finally, a linear transformation is applied to reduce the number of channels from... 2C Map back to the original C , to obtain output features :

[0084]

[0085] This embodiment introduces a parallel branch structure into the traditional Mamba module and combines it with a parameter-learnable Gaussian low-pass filter module to achieve adaptive extraction of nonlocal features at different scales, thereby compensating for the model's inability to model nonlocal dependencies. By combining dynamic parameter calculation and a selective scanning mechanism, the features not only retain local input information but also effectively capture long-range dependencies, making feature evolution more adaptable and expressive. Through this innovative design, this embodiment enhances the ability to model nonlocal information while maintaining controllable computational complexity, providing a new technical solution for high-quality image deblurring.

[0086] In some embodiments, the first nonlocal compensation branch includes: a frequency domain conversion module, a Gaussian low-pass filter module, and a spatial domain conversion module.

[0087] The frequency domain transformation module converts the input features from the spatial domain to the frequency domain, obtaining frequency domain features. The Gaussian low-pass filter module modulates the frequency domain features, obtaining modulated nonlocal features. The spatial domain transformation module restores the low-pass filtered features to their spatial domain representation using an inverse Fourier transform.

[0088] In applications, the adaptive low-pass filtering process of the first nonlocal compensation branch can be as follows: Figure 5 As shown, the frequency domain transformation module performs a two-dimensional Fourier transform on the input feature map. At this point, the low-frequency components are located at the four corners of the spectrum. Then, a spectrum center shifting operation moves the low-frequency components to the center of the spectrum, converting the spatial domain signal to a more processable frequency domain representation, allowing different frequency components of the image to be processed independently. Subsequently, a learnable Gaussian low-pass filter module is adaptively generated based on the feature map size. The parameters are automatically optimized through backpropagation during training. The filter's center is located at the shifted spectrum center. By calculating the distance from each frequency point to the center and constructing Gaussian weights accordingly, low-frequency regions are preserved while high-frequency regions are attenuated. The filter strength is controlled by a trainable parameter that determines the standard deviation of the Gaussian function, thus affecting the cutoff frequency range. This allows the model to automatically learn the optimal low-pass characteristics during training, and each channel's parameters are independent, facilitating adaptive adjustment of different channel features and frequency response optimization.

[0089] Next, the Gaussian low-pass filter module is multiplied point-by-point with the centered spectral features to complete frequency domain filtering. Then, an inverse center shift operation is performed. Finally, the result is mapped back to the spatial domain through an inverse Fourier transform module, resulting in a smoothed and modulated nonlocal feature representation. This process not only effectively suppresses noise and local texture interference but also highlights nonlocal structural information, providing a more stable and robust feature input for subsequent nonlocal information processing.

[0090] In some embodiments, the enhanced FFN module further includes: a second feature enhancement module, a fusion module, and a dual-branch frequency domain enhancement module.

[0091] The second feature enhancement module performs pointwise feature mapping on the input features to adjust the channel dimensions and enhance feature expressive power. It also uses depthwise separable convolution to perform local feature extraction, introducing local spatial information separately for each channel to obtain enhanced local features.

[0092] The fusion module is used to segment the enhanced local features along the channel dimension. After the two segments are processed by a nonlinear activation function, one feature is used as a gating signal and multiplied element-wise with the other feature. The two features are then concatenated along the channel direction to form the fused output feature.

[0093] The enhanced FFN module's processing flow can be as follows: Figure 6 As shown, the dual-branch frequency domain enhancement module is used to process the fused output features using convolutional layers. The processed features are divided into sub-blocks in the spatial dimension and then input into the second nonlocal compensation branch and the frequency domain discriminant branch in parallel. The outputs of the two branches are subjected to inverse Fourier transform to return to the spatial domain. The sub-blocks in the spatial domain are sequentially spliced ​​back to the original size, and the spliced ​​results of the two branches are added and fused.

[0094] During the second feature enhancement module processing, a 1×1 convolutional layer is used. Input features The number of channels is expanded to 6 times the original number to increase feature expressiveness. Then, the expanded features are input into a 3×3 depthwise separable convolution. This further captures local spatial information while maintaining the independence between channels. Next, the fusion module divides the features into two equal parts along the channel dimension, denoted as... and This process can be represented as:

[0095]

[0096] To enhance inter-channel interaction capabilities, this embodiment employs an improved gating mechanism—CrossGate. Specifically, it uses features... After nonlinear activation function After processing, it is combined with another set of features. Perform element-wise multiplication; simultaneously, assign features... through After activation, with features Element-wise multiplication is performed. Finally, the two interaction results are concatenated along the channel dimension to obtain the output feature. This mechanism achieves complementary information exchange between features by performing nonlinear activation and cross-fusion on the two sets of features after partitioning, thereby enhancing feature representation capabilities and improving model performance without increasing computational cost. Therefore, the above process can be further expressed as:

[0097]

[0098] Subsequently, the dual-branch frequency domain enhancement module is passed through a 1×1 convolutional layer. Will The number of channels is reduced back to the number of input channels, achieving feature dimension compression and information integration while maintaining the integrity of the fused local and non-local feature representations and saving computational overhead. Then, the dimensionality-reduced feature map is divided into non-overlapping sub-blocks to improve adaptability to images of different sizes. This process can be represented as:

[0099]

[0100] Divide the feature sub-blocks in the middle The inputs are fed into a dual-branch structure of the enhanced FFN module for parallel processing to enhance frequency domain information. The final result of the frequency domain discriminative branch is then obtained. Results of the second nonlocal compensation branch The fusion is achieved by summing element by element, resulting in an output feature that integrates local detail representations and non-local information. This process can be represented as:

[0101]

[0102] In some embodiments, the input features in the frequency domain discriminative branch are directly multiplied element-wise with the learnable weights to discriminate and retain the target frequency domain information; the second nonlocal compensation branch first extracts nonlocal information through a low-pass filter and then multiplies it with the learnable weights.

[0103] In applications, for the frequency domain discriminative branch, a real-valued Fast Fourier Transform (RFFT) is first performed on the feature sub-blocks to the frequency domain. Then, learnable frequency domain weight parameters are used to discriminatively enhance the local structural information, preserving features useful for image deblurring and suppressing useless information. These frequency domain weight parameters are channel-independent, allowing independent adjustment of the frequency response for each channel, and are automatically optimized through backpropagation during training. Finally, an inverse real-valued Fourier Transform (IRFFT) is performed to the spatial domain, and the feature sub-blocks are merged sequentially to return to their original size. This branch can be represented as:

[0104]

[0105] In the second nonlocal compensation branch, by specifically processing the nonlocal information within each sub-block, the low-frequency nonlocal structural features of each sub-block can be highlighted, avoiding interference from local details, thereby enhancing the model's ability to perceive the overall structure. Specifically, an adaptive low-pass filter is introduced after the fast Fourier transform to suppress local high-frequency noise components and highlight low-frequency nonlocal structures, thereby enhancing the modeling ability of nonlocal dependencies. Then, learnable parameters are used for frequency domain enhancement. This branch can be represented as:

[0106]

[0107] It should be noted that after the frequency domain transformation is completed, this embodiment generates a Gaussian low-pass mask corresponding to the input feature size on the frequency plane of the real-valued Fast Fourier Transform. The mask is constructed based on the frequency coordinates in the horizontal and vertical directions, and the radial distance to the origin of the spectrum is obtained by calculating the sum of the squares of the frequency components. Considering that the upper left and lower left corner regions of the RFFT spectrum structure contain DC components and low-frequency information in the vertical direction, this embodiment generates and retains the Gaussian low-pass response only in these two regions to weaken high-frequency components and highlight low-frequency features. Compared with conventional FFT methods, this application makes full use of the inherent conjugate symmetry of RFFT, reducing the calculation range of the Gaussian low-pass mask from the complete spectrum to half, significantly reducing the computational complexity while maintaining the same low-pass filtering effect.

[0108] The standard deviation of the Gaussian low-pass mask is adaptively controlled by a learnable parameter to adjust the filter's cutoff frequency and transition bandwidth. This parameter is used independently for each channel, facilitating channel-level adaptive filtering and frequency optimization. When this parameter is small, the filter response is concentrated in the upper left and lower left corners of the spectrum, effectively filtering out low-frequency components. As the parameter increases, the frequency range covered by the filter expands to retain more mid- and low-frequency information. After obtaining the spectrum of the input features via RFFT, the Gaussian mask is multiplied point-by-point with the spectrum to achieve smooth attenuation of high-frequency energy and compensation enhancement of low-frequency structural information. Finally, the spatial domain is recovered through inverse real-valued fast Fourier transform, thus obtaining the nonlocal feature representation selected by the low-pass filter.

[0109] This embodiment constructs a dual-branch frequency domain enhancement structure. On the one hand, in the frequency domain discriminative branch, sub-block partitioning and frequency domain weight adjustment are used to enhance the expression of local details. On the other hand, in the second nonlocal compensation branch, a low-pass filter is combined to highlight the nonlocal low-frequency structure and strengthen the nonlocal modeling capability, thereby achieving efficient fusion of local and nonlocal features and improving the image deblurring effect.

[0110] In some embodiments, the first iterative process includes: performing multiple deblurring processes on the input parameters sequentially, and downsampling the features after multiple deblurring processes.

[0111] It is understandable that multiple deblurring processes are sequentially performed on the input features. Each deblurring process relies on the improved Mamba module and the enhanced FFN module. A low-pass filter module with learnable parameters iteratively extracts non-local features at different scales and suppresses high-frequency noise. Simultaneously, local branches are used to accurately capture detailed information. Multiple iterations gradually remove blurry components and interference signals from the features, continuously optimizing the feature's discriminability and effectiveness, avoiding the problem of insufficient feature optimization that may occur with a single deblurring process. After completing multiple deblurring processes and obtaining high-quality purified features, a downsampling operation is performed on these features. By using a reasonable downsampling kernel, the spatial resolution of the feature map is reduced, and the number of pixels is decreased, while retaining the key structural features and effective information enhanced by multiple deblurring processes. This fundamentally alleviates the enormous computational and storage pressure brought by high-resolution images, ensuring that the key information required for core restoration is not lost while reducing dimensionality.

[0112] In some embodiments, the number of deblurring processes corresponds to the same number for the i-th first iteration and the (n-i+1)-th second iteration.

[0113] In this design, the i-th first iteration and the (n-i+1)-th second iteration have the same number of deblurring operations. This symmetrical design ensures a precise correspondence between feature dimensionality reduction and dimensionality enhancement, and a dynamic matching of optimization efforts. The core of the i-th first iteration is to gradually refine information during the feature dimensionality reduction stage. Its input is the feature map from the preceding process. By performing the same number of deblurring operations as the corresponding second iteration, it ensures that the features corresponding to each round of dimensionality reduction have undergone sufficient optimization before the downsampling operation is performed to reduce the feature dimensionality, thus avoiding the loss of useful information due to dimensionality reduction. The corresponding (n-i+1)th level second iteration process focuses on gradually restoring resolution during the feature upscaling stage. The input is the feature map from the previous second iteration. Through the same number of deblurring operations, it can accurately inherit the optimization intensity of the first iteration of dimensionality reduction. While performing upsampling operations to improve feature resolution, it continuously deepens non-local structure modeling and local detail supplementation, compensating for feature blurring or structural distortion that may occur during dimensionality upscaling. This ensures both the continuity and symmetry of feature processing throughout the entire iterative process, and avoids insufficient optimization due to insufficient deblurring at a certain stage or computational redundancy caused by excessive deblurring. Ultimately, through this symmetrical matching processing strategy, the high-quality features purified in the dimensionality reduction stage are fully restored and extended in the dimensionality upscaling stage, providing strong support for restoring the detail clarity and structural consistency of the image.

[0114] Based on the same inventive concept, such as Figure 7 As shown, this application also provides a Mamba image deblurring system with non-local information compensation, including: a feature extraction module 11, an encoder module 12, a decoder module 13, and an image reconstruction module 14.

[0115] The feature extraction module 11 is used to extract features from the degraded image containing blurred features using a convolutional layer to obtain initial shallow features;

[0116] The encoder module 12 includes n cascaded encoder layers and a deblurring module. The encoder layers are used to perform a first iterative processing on the input features. The encoder module 12 is used to process the initial shallow features to obtain a first recovered feature map. The first iterative processing includes: sequentially deblurring and downsampling the input parameters. The deblurring is accomplished by an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a first nonlocal compensation branch and a selective SSM branch set in parallel and introducing a selective scanning mechanism. Before entering the selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module. The enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel. The second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights.

[0117] The decoder module 13 includes n cascaded decoder layers and a deblurring module. The decoder layers are used to perform a second iterative process on the input features. The decoder module 13 is used to process the first recovered feature map to obtain a second recovered feature map. The second iterative process includes: performing deblurring and upsampling on the input features in sequence.

[0118] The image reconstruction module 14 is used to process the second restored feature map using a convolutional layer to generate a residual feature map, and then add the residual feature map to the degraded image to obtain the restored image.

[0119] It should be noted that the Mamba image deblurring system with non-local information compensation provided in this application embodiment and the Mamba image deblurring method with non-local information compensation provided in this application embodiment are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned Mamba image deblurring method with non-local information compensation, and the repeated parts will not be described again.

[0120] Based on the above embodiments, to verify the effectiveness of the non-local information compensation Mamba image deblurring system, we trained and tested it on multiple synthetic and real-world datasets, including commonly used GoPro, HIDE, and RealBlur datasets. These datasets cover various forms of blur degradation caused by camera shake, object motion, and real-world shooting scenarios, ensuring both the diversity of training data and enhancing the model's generalization ability in complex real-world scenes.

[0121] During the training phase, this embodiment employs a joint optimization strategy using L1 loss and frequency domain loss. L1 loss measures the difference between the predicted result and the real image in pixel space, ensuring the accuracy of overall image restoration. Frequency domain loss constrains the consistency of the predicted and real images in spectral distribution, effectively improving the model's ability to recover details and texture information. The two losses are weighted and summed with coefficients of 1 and 0.1 to form the final optimization objective function. Through backpropagation, this loss function continuously guides the updating and optimization of model parameters, ultimately improving the overall performance of the model.

[0122] Regarding the optimization strategy, this embodiment employs the AdamW optimizer (β1=0.9, β2=0.9, weight decay is...). Compared to the traditional Adam optimizer, AdamW effectively suppresses overfitting by introducing a weight decay mechanism, further improving model convergence stability and generalization ability. The initial learning rate is set to... And a cosine annealing scheduling strategy is used to gradually decay to This strategy maintains a large learning rate in the early stages of training to accelerate convergence, while gradually reducing the learning rate in the later stages of training to ensure the stability and precision of the results.

[0123] In terms of data augmentation, this embodiment employs random flipping and rotation, random saturation adjustment, and other methods to augment the image, including horizontal and vertical flipping, as well as 90° rotation. These strategies effectively increase the diversity of training samples, preventing the model from overfitting on limited data, thereby significantly improving the model's robustness and generalization ability.

[0124] Regarding data cropping, during training, a 128×128 pixel image patch is randomly cropped from the input image to improve training efficiency and enhance sample diversity. A fixed random seed is used to ensure the reproducibility of training results. During testing, the input image is divided into multiple non-overlapping 384×384 pixel image patches and fed into the nonlocal information compensation Mamba module to ensure prediction consistency and detail integrity under a large receptive field. Then, the recovery results of each patch are merged and restored to the original image size. Finally, the original size is input into the frequency domain enhancement FFN module.

[0125] The entire training and testing process was completed on a single GPU. During training, the batch size was set to 6, and the total number of iterations was 400,000. This hardware configuration, combined with PyTorch's parallel computing capabilities, provided ample computing resources for model training, ensuring the efficiency and stability of the training process.

[0126] In terms of network structure, this embodiment provides a symmetrical U-shaped network architecture based on a hierarchical encoder-decoder, which is the network architecture corresponding to the Mamba image deblurring system with non-local information compensation. In one example, the complete image deblurring process can be as follows:

[0127] First, input the degraded image. After passing through a 3×3 convolution, feature mapping is performed to obtain shallow features. Next, the module enters the encoder section, sequentially passing through the first, second, and third encoding layers. Each layer stacks 6, 6, and 12 deblurring modules respectively to progressively extract multi-scale deep semantic features. Downsampling modules are placed between the encoding layers to progressively reduce the spatial resolution of the feature maps and expand the channel dimension, thereby obtaining a more global feature representation and resulting in the encoded feature representation. The number of channels here Set it to 48.

[0128] The specific implementation process can be referred to the description of the foregoing embodiments, and will not be repeated here. Through the synergistic effect of the two, the feature representation capability can be significantly improved without significantly increasing the computational overhead.

[0129] Then, for the decoder section, the decoder and encoder are set up symmetrically, and the output of the encoder... The image is sequentially passed through the first, second, and third decoding layers. Upsampling modules are placed between the decoding layers to restore the spatial resolution of the feature maps layer by layer. Each decoding layer also stacks multiple deblurring modules, symmetrical in number to the encoder, to gradually restore the image's detailed information, resulting in the decoded feature representation. .

[0130] Secondly, skip connections are set between the corresponding encoding and decoding layers to add and fuse shallow features with deep features, thereby enhancing the ability to recover details and avoiding the gradient vanishing problem.

[0131] Finally, in the reconstruction phase, at the end of the decoding process... A 3×3 convolutional layer is used to generate the residual image. The network's reconstruction performance is further enhanced by adding the residual image to the input image through residual connections, resulting in the final deblurred image. By using the residual learning mechanism, the model only needs to learn the difference between the blurred image and the clear image, which improves both the convergence speed of training and the final restoration effect.

[0132] In summary, the image deblurring system based on a hierarchical encoder-decoder structure provided in this embodiment can preserve local details and compensate for and enhance non-local information while ensuring multi-scale feature extraction. This structure not only exhibits high accuracy and stability on synthetic datasets but also demonstrates good robustness and generalization ability on real-world scene data, thus achieving superior performance compared to existing methods in image restoration tasks.

[0133] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described Mamba image deblurring method with non-local information compensation.

[0134] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0135] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0136] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the Mamba image deblurring method with non-local information compensation described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0137] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0138] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A Mamba image deblurring method with non-local information compensation, characterized in that, include: Convolutional layers are used to extract features from degraded images containing blurred features to obtain initial shallow features; The initial shallow features are used as input parameters, and the first iteration is performed n times. The features after the nth first iteration are then deblurred to obtain a first restored feature map. The first iteration includes deblurring and downsampling the input parameters sequentially. The deblurring is accomplished by an improved Mamba module and an enhanced FFN module. The improved Mamba module includes a first nonlocal compensation branch and a selective SSM branch set in parallel and incorporating a selective scanning mechanism. Before entering the selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module. The enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel. The second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights. The first restored feature map is used as the input parameter, and the second iteration is performed n times. The feature after the nth second iteration is then deblurred to obtain the second restored feature map. The second iteration includes: deblurring and upsampling the input parameter in sequence. The second restored feature map is processed by a convolutional layer to generate a residual feature map, and the residual feature map is added to the degraded image to obtain the restored image.

2. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The improved Mamba module also includes a first feature enhancement module and a normalization processing module; The feature enhancement module is used to perform linear mapping operation on the input features, adjust the original channel dimension to a preset extended dimension, introduce depthwise separable convolution to perform convolution calculation on each channel independently, map the features after convolution calculation through a non-linear activation function, and output the mapped features to the first non-local compensation branch and the selective SSM branch. The feature enhancement module is also used to perform nonlinear activation on the features after the linear mapping operation and output the modulated features to the normalization processing module. The normalization processing module is used to fuse the output features of the first nonlocal compensation branch and the selective SSM branch, perform layer normalization processing, and interact the layer normalized features with the modulation features element by element. The feature dimension of the interacted features is compressed back to the original number of channels through a linear mapping layer.

3. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The first nonlocal compensation branch includes: a frequency domain conversion module, a Gaussian low-pass filter module, and a spatial domain conversion module; The frequency domain conversion module is used to convert the input features from the spatial domain to the frequency domain representation to obtain frequency domain features; The Gaussian low-pass filter module is used to modulate the frequency domain features to obtain modulated nonlocal features; The spatial domain transformation module is used to restore the spatial domain representation of the features after low-pass filtering through inverse Fourier transform.

4. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The enhanced FFN module further includes: a second feature enhancement module, a fusion module, and a dual-branch frequency domain enhancement module; The second feature enhancement module is used to perform pointwise feature mapping on the input features to adjust the channel dimension and enhance the feature expressive power, and to use depthwise separable convolution to perform local feature extraction, introducing local spatial information separately for each channel to obtain enhanced local features; The fusion module is used to segment the enhanced local features along the channel dimension. After the two segments are processed by a nonlinear activation function, one feature is used as a gate signal and multiplied element-wise with the other feature. The two features are then spliced ​​together along the channel direction to form the fused output feature. The dual-branch frequency domain enhancement module is used to process the fused output features using convolutional layers, and then divides the processed features into sub-blocks in the spatial dimension and inputs them in parallel into the second nonlocal compensation branch and the frequency domain discriminant branch. The outputs of the two branches are subjected to inverse Fourier transform to return to the spatial domain, and the sub-blocks in the spatial domain are sequentially spliced ​​back to the original size. The spliced ​​results of the two branches are then added and fused.

5. The Mamba image deblurring method with non-local information compensation as described in claim 4, characterized in that, The input features in the frequency domain discriminant branch are directly multiplied element-wise with the learnable weights to discriminate and retain the target frequency domain information; the second nonlocal compensation branch first extracts nonlocal information through a low-pass filter and then multiplies it with the learnable weights.

6. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The cutoff frequency ratio in the Gaussian low-pass filter module is a learnable parameter.

7. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The number of deblurring operations corresponds to the same number of iterations for the first iteration at level i and the second iteration at level n-i+1.

8. The Mamba image deblurring method with non-local information compensation as described in claim 7, characterized in that, The first iterative process includes: performing multiple deblurring processes on the input parameters in sequence, and downsampling the features after multiple deblurring processes.

9. The Mamba image deblurring method with non-local information compensation as described in claim 1, characterized in that, The output features of the first iteration of the i-th process are used as jump connection features, which are then fused with the input features in the second iteration of the (n-i+1)-th process.

10. A Mamba image deblurring system with nonlocal information compensation, characterized in that, include: The module consists of a feature extraction module, an encoder module, a decoder module, and an image reconstruction module. The feature extraction module is used to extract features from the degraded image containing blurred features using a convolutional layer to obtain initial shallow features; The encoder module includes n cascaded encoder layers and a deblurring module. The encoder layers are used to perform a first iterative processing on the input features. The encoder module is used to process the initial shallow features to obtain a first recovered feature map; the first iterative processing includes: deblurring and downsampling the input parameters sequentially; the deblurring is completed by an improved Mamba module and an enhanced FFN module, the improved Mamba module includes a first nonlocal compensation branch and a selective SSM branch set in parallel and introducing a selective scanning mechanism, the first nonlocal compensation branch adaptively extracts nonlocal features at different scales through a parameter-learnable Gaussian low-pass filter module before entering the selective scanning mechanism, the enhanced FFN module includes a second nonlocal compensation branch and a frequency domain discriminant branch set in parallel, the second nonlocal compensation branch extracts nonlocal information in the frequency domain using an adaptive low-pass filter module and performs dynamic modulation through learnable discriminant weights; The decoder module includes n cascaded decoder layers and a deblurring module. The decoder layers are used to perform a second iterative processing on the input features. The decoder module is used to process the first recovered feature map to obtain a second recovered feature map. The second iterative processing includes: sequentially performing deblurring and upsampling on the input features. The image reconstruction module is used to process the second restored feature map using a convolutional layer to generate a residual feature map, and then adds the residual feature map to the degraded image to obtain the restored image.

Images