Image deblurring method based on rotation perception multidimensional attention and fuzzy sensitive adaptive distribution mechanism

Abstract

The invention discloses an image deblurring method based on a rotation perception multi-dimensional attention and blurring sensitivity self-adaptive distribution mechanism, aiming at the defects of the prior art in the aspects of complex blurring processing and image reality sense improvement, and belongs to the technical field of image processing, and the method comprises the following steps: preprocessing a collected picture to obtain a training set; training an image deblurring model by using the training set; and performing image deblurring by using the trained image deblurring model. According to the invention, by designing a plurality of innovative modules and combining fuzzy degree adaptive selection, kernel estimation and dynamic weight distribution driven by a physical model, a multi-dimensional rotation perception attention mechanism, spectrum reconstruction anti-noise deblurring and detail enhancement of perception loss optimization, the image deblurring effect and processing efficiency are effectively improved; the method is especially suitable for processing blurred images with high complexity and high resolution.

Description

Technical Field

[0001] This invention relates to image processing technology, specifically to an image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism. Background Technology

[0002] Receptance Weighted Key Value (RWKV) model: RWKV is a deep learning architecture that combines the advantages of Transformer and RNN, employing an acceptance weight mechanism to adjust the influence of key values ​​on the model output. It achieves highly parallelized training and efficient inference, making it suitable for long sequence processing.

[0003] WKV Attention Mechanism: The WKV (Weighted Key-Value) attention mechanism is the core component of the RWKV model. It achieves selective attention to information through weighted key-value pairs, avoiding the dot product calculation in traditional attention mechanisms and improving computational efficiency.

[0004] In the field of image deblurring, image blurring is often caused by factors such as motion during shooting, lighting conditions, and dynamic changes of objects, exhibiting varying degrees of blur and directional characteristics. Currently, image deblurring methods typically rely on deep learning methods such as Convolutional Neural Networks (CNNs), Transformers (self-attention mechanisms), and Mamba models based on State-Space Models (SSMs). While these techniques have achieved some success in image deblurring, they still face numerous challenges in complex blurred scenes.

[0005] Traditional CNN models are widely used in image deblurring tasks, particularly showing good results in removing simple motion blur and slight blur. CNNs extract spatial features of images through local convolution operations. However, the inherent kernel size of CNNs limits the receptive field of the model, making it difficult to effectively capture blur dependencies spanning large distances in an image, especially when processing high-resolution images. Although multi-scale networks and deep U-Net structures can partially alleviate this problem, they still struggle to achieve accurate detail restoration when dealing with complex blur structures.

[0006] To compensate for the shortcomings of CNNs, the self-attention mechanism of Transformers has been introduced into deblurring tasks. Transformers possess a global self-attention mechanism, which can better model long-range dependencies in images, improving the quality of blurred information recovery to some extent. However, the computational complexity of the Transformer self-attention mechanism increases quadratically with the increase of input image resolution, making it inefficient in high-resolution image processing. To address this, many Transformer variants reduce computational overhead by limiting the computational scope of attention (e.g., local window attention or sparse attention), but this also leads to a reduction in the receptive field and the loss of long-range information, thus affecting the overall comprehensiveness of the deblurring effect.

[0007] As an alternative solution, the Mamba model, based on the SSM structure, models long-range dependencies with linear computational complexity, combining efficiency and long-range modeling capabilities, making it suitable for processing large-scale image data. The advantage of the Mamba model lies in its ability to capture global features without requiring window partitioning operations similar to Transformers. However, the unidirectional sequence modeling characteristic of the Mamba model limits its performance in multi-directional blurring, making it difficult to comprehensively capture blur information in all directions of the image, thus affecting the stability and accuracy of the deblurring effect.

[0008] Furthermore, the Receptance Weighted Key Value (RWKV) model, with its advantages in Natural Language Processing (NLP), is gradually entering the field of image processing. RWKV achieves long-distance dependency modeling with linear complexity through the WKV attention mechanism, while introducing a Token Shift layer to enhance the capture of local features. However, RWKV was originally designed for one-dimensional sequence processing, and its direct application to two-dimensional image deblurring tasks encounters difficulties. To address this, Vision-RWKV improves the RWKV model by introducing bidirectional WKV and a four-directional Token Shift mechanism, extending it to the image processing domain. However, Vision-RWKV still has shortcomings in handling complex multi-directional blurring, making it difficult to achieve ideal results in fine-grained blur removal.

[0009] Despite significant progress in image deblurring technology, the following key issues remain in practical applications:

[0010] Lack of adaptive processing mechanism based on blur level: Existing deblurring models mostly adopt fixed network structure and processing strategy, and fail to flexibly adjust the deblurring strategy according to the blur level of the image. As a result, the model lacks adaptability when processing different types and degrees of blur, which affects the deblurring effect and processing efficiency.

[0011] The accuracy of fuzzy information restoration is insufficient: existing methods struggle to accurately estimate the direction and degree of fuzziness, especially in complex fuzzy scenes, where the estimation of the fuzzy kernel lacks effectiveness. Since accurate estimation of the fuzzy kernel is fundamental to fuzz removal, the shortcomings of existing technologies in this area lead to inadequate fuzz restoration.

[0012] Limitations of multi-directional blur processing: In complex scenes, blur in an image may exist in multiple directions and to varying degrees. Existing models struggle to fully and flexibly adapt to these multi-directional blur features, resulting in incomplete image detail recovery and insufficient blur removal.

[0013] Lack of noise-resistant deblurring capabilities in the spectral domain: Most existing deblurring methods operate directly in the spatial domain, failing to fully utilize spectral domain information for noise-resistant deblurring. Spatial domain operations often struggle to accurately separate high-frequency and low-frequency information, resulting in the loss of some high-frequency details and texture features during deblurring.

[0014] The visual effects of detail enhancement are lacking: Although existing techniques can remove blur to some extent, the restored images often fall short in terms of detail realism and visual quality. The lack of comprehensive perceptual optimization for gradient, texture, and content loss leads to unsatisfactory detail enhancement results. Summary of the Invention

[0015] This invention addresses the shortcomings of existing deblurring methods in handling complex multi-directional blurring and restoring high-precision blur information, which leads to unstable deblurring results, low efficiency, and significant deficiencies in visual detail enhancement. It provides an image deblurring method based on rotation-aware multidimensional attention and blur sensitivity adaptive allocation mechanism, combined with multi-dimensional innovative modules, to improve the adaptability, accuracy, and detail restoration effect of deblurring.

[0016] To achieve the above objectives, the present invention adopts the following technical solution:

[0017] An image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism includes the following steps:

[0018] The collected images are preprocessed to obtain the training set;

[0019] Train an image deblurring model using the training set;

[0020] Image deblurring is performed using a trained image deblurring model.

[0021] To optimize the above technical solution, the specific measures also include:

[0022] Furthermore, the preprocessing of the acquired images specifically includes:

[0023] The system normalizes and converts the color space of the collected images, and automatically scales and crops them according to their actual size.

[0024] Furthermore, the image deblurring model specifically includes: a shallow feature extraction module, a blur sensitivity evaluation module, a dynamic weight allocation module, a high-dimensional feature decoupling and extraction module based on a hierarchical structure, an image restoration module, and a detail enhancement module based on perceptual loss optimization;

[0025] The shallow feature extraction module is used to extract shallow features from the input image;

[0026] The blur sensitivity evaluation module is used to evaluate the blur degree of the input image and obtain a blur sensitivity map;

[0027] The dynamic weight allocation module is used to weight shallow features using a fuzzy sensitivity map to obtain weighted features;

[0028] The high-dimensional feature decoupling and extraction module based on hierarchical structure is used to extract deep features from weighted features;

[0029] The image restoration module is used to generate a residual image based on depth features, and the residual image is added to the input image to obtain a deblurred image;

[0030] The detail enhancement module based on perceptual loss optimization is used to construct a loss function. The perceptual loss function is used to calculate the difference between the deblurred image and the original image in the high-level feature space, guiding the image deblurring model to refine local details while maintaining the overall structural consistency.

[0031] Furthermore, the process of evaluating the blur level of the input image to obtain a blur sensitivity map specifically involves:

[0032] Using the Sobel operator on shallow features F S The gradients in the horizontal and vertical directions are calculated using the following formulas:

[0033] G x =Sobel x (F S ),G y =Sobel y (F S )

[0034] In the formula, G x It is the gradient in the horizontal direction, G y It is the gradient in the vertical direction, Sobel x It is the horizontal Sobel operator, Sobel y It is a Sobel operator in the vertical direction;

[0035] Calculate the gradient magnitude G mag :

[0036]

[0037] Based on the gradient magnitude, the degree of ambiguity is evaluated, and the ambiguity sensitivity map M is obtained:

[0038]

[0039] in, and These are the minimum and maximum values ​​of the gradient magnitude, respectively.

[0040] Furthermore, the weighting of shallow features using the fuzzy sensitivity map to obtain weighted features is specifically as follows:

[0041] F W =F S ⊙M

[0042] In the formula, F W As a weighted feature, F S For shallow features, M is a fuzzy sensitivity map, and ⊙ represents element-wise pointwise multiplication.

[0043] Furthermore, the high-dimensional feature decoupling and extraction module based on the hierarchical structure adopts a four-level U-shaped encoder-decoder architecture, with each level consisting of a multi-dimensional rotated spectrum reconstruction RWKV block;

[0044] The first-level encoder processes the weighted features by reconstructing RWKV blocks from N1 multidimensional rotated spectra, obtaining the first-level encoded feature F1, expressed by the following formula:

[0045]

[0046] In the formula, R-RWKV represents the multidimensional rotational spectrum reconstruction RWKV block, and F W Indicates weighted features;

[0047] Using a 1×1 convolutional layer Conv 1×1 Downsampling is performed using pixel rearrangement operations, as shown in the following formula:

[0048] F down1 =PixelUnshuffle(Conv 1×1 (F1))

[0049] In the formula, PixelUnshuffle represents the pixel rearrangement operation, F down1 This represents the first-level downsampling feature;

[0050] The second-stage encoder reconstructs the RWKV block from the first-stage downsampled feature F using N2 multidimensional rotating spectra. down1 After processing, the second-level coding feature F2 is obtained, and the formula is expressed as follows:

[0051]

[0052] The second-level encoded feature F2 is downsampled to obtain the second-level downsampled feature F. down2 ;

[0053] The third-level encoder reconstructs the RWKV block from the second-level downsampled feature F using N3 multidimensional rotating spectra. down2 The process yields the third-level coding feature F3. Then, a downsampling operation is performed on the third-level coding feature F3 to obtain the third-level downsampled feature F. down3 ;

[0054] The fourth-level encoder reconstructs the RWKV block from the third-level downsampled feature F using N4 multidimensional rotational spectra. down3 The bottleneck feature F is obtained through processing. bottleneck ;

[0055] Bottleneck feature F bottleneck The input decoder is then subjected to upsampling and feature fusion, as detailed below:

[0056] In the third-level decoder, upsampling is performed using pixel rearrangement and a 1×1 convolutional layer, as shown in the following formula:

[0057] F up3 ′=Conv 1×1 (Pixelshuffle(F bottleneck ))

[0058] In the formula, PixelUnshuffle represents the pixel rearrangement operation, and Conv 1×1 F represents a 1×1 convolutional layer. up3 ′ represents the third-level upsampling feature;

[0059] The third-level upsampled features and the third-level encoded features are skipped together, as shown in the following formula:

[0060] F concat3 =Concat(F up3 ′,F3)

[0061] In the formula, F concat3 This indicates the third-level concatenation feature; Concat represents the concatenation operation.

[0062] Feature fusion is performed by reconstructing RWKV blocks using N3 multidimensional rotated spectra:

[0063] Fdecoder3 =R-RWKV N3 (F concat3 )

[0064] In the formula, F decoder3 Indicates the third-level decoding feature;

[0065] The third-level decoded features are input into the second-level decoder. In the second-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the second-level decoded features F. decoder2 ;

[0066] The second-level decoded features are input into the first-level decoder. In the first-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the first-level decoded features F. decoder1 ;

[0067] The first-level decoded features are input to the final-level decoder. The final-level decoder uses a multidimensional rotated spectrum to reconstruct the RWKV block on the first-level decoded features F. decoder1 The depth feature F is obtained by refining the feature. D .

[0068] Furthermore, the structure of the multidimensional rotating spectrum reconstruction RWKV block includes a spatial mixing module and a channel mixing module. The input and output of the spatial mixing module are residually connected, and the input and output of the channel mixing module are residually connected.

[0069] The principle of the spatial mixing module is as follows:

[0070] Layer normalization is performed on the input feature X, as shown in the following formula:

[0071] X norm =LN(X)

[0072] In the formula, X norm For normalized features, LN is layer normalization;

[0073] Normalized feature X norm Input a 360° omnidirectional labeling displacement layer, output the displacement feature X s ;

[0074] For the feature X after displacement s Linear projection to obtain the receiving gate R s Key K s Sum V s The formula is as follows:

[0075]

[0076] In the formula, These are the learnable weight matrices;

[0077] The global attention wkv is calculated using the recurrent WKV attention mechanism, as shown in the following formula:

[0078] wkv=Re-WKV(K s V s )

[0079] In the formula, Re-WKV is the recurrent WKV attention mechanism;

[0080] The attention result is modulated using the Sigmoid function to obtain the output O of the spatial mixing module. s :

[0081]

[0082] in, Let σ(·) be the weight matrix of the output projection, and let σ(·) be the Sigmoid function.

[0083] The principle of the channel mixing module is as follows:

[0084] The output O of the spatial mixing module s The input feature X of the spatial mixing module is added to the input feature X of the channel mixing module and then used as the input of the channel mixing module. For input Perform layer normalization, input the normalized features into the omnidirectional labeled displacement layer, and output the displacement features X. c ;

[0085] For the feature X after displacement c Linear projection to obtain the receiving gate R c and key K c :

[0086] K c =X c W K

[0087] In the formula, and W K All are learnable weight matrices;

[0088] The key K is processed using the squared ReLU activation function. c And obtain the value V by linear projection. c :

[0089]

[0090] In the formula, Let be the weight matrix, and use ReLU(·). 2 It is the squared ReLU activation function;

[0091] The output O of the channel mixing module is obtained by receiving gate modulation.c :

[0092]

[0093] In the formula, σ(·) is the Sigmoid function. This is the weight matrix;

[0094] The output O of the channel mixing module c Input of the channel mixing module Adding them together yields the output X of the multidimensional rotational spectrum reconstruction RWKV block. out .

[0095] Furthermore, the omnidirectional labeling displacement layer is a multi-branch convolutional structure, with each branch corresponding to a different receptive field. Branch 1 is a 1×1 depthwise convolution, branch 2 is a 3×3 depthwise convolution, branch 3 is a 5×5 depthwise convolution, and branch 4 is an identity mapping. The outputs of each branch are weighted and fused.

[0096]

[0097] Where, α i For learnable fusion weights, X os This indicates the features of the input omnidirectional labeled displacement layer, Branch i (·) represents the i-th branch, and Omni-Shift(·) represents the omnidirectional marking displacement layer.

[0098] Furthermore, the recurrent WKV attention mechanism specifically involves recursively applying bidirectional WKV attention in different scanning directions, as detailed below:

[0099] Set the number of loops M and the set of scan directions D, where D = {horizontal, vertical};

[0100] Initialize wkv (0) =V s wkv (0) For initial global attention, V s Represents the value;

[0101] For loop counts j = 1 to M, perform the following steps sequentially:

[0102] Select the scan direction d for the j-th scan. j ∈D

[0103] According to d j Change key K s The global attention wkv in the (j-1)th iteration (j-1) The order of arrangement;

[0104] Applying bidirectional WKV attention, we obtain the global attention wkv for the j-th iteration.(j) :

[0105] wkv (j) =Bi-WKV(K s wkv (j-1) )

[0106] In the formula, Bi-WKV(·) represents bidirectional WKV attention;

[0107] When the maximum number of iterations is reached, output the global attention wkv for the Mth iteration. (M) wkv serves as the final global attention.

[0108] Furthermore, the loss function includes reconstruction loss, gradient loss, texture loss, and content loss.

[0109] The beneficial effects of this invention are:

[0110] 1. Significantly improves image deblurring effect: Through multidimensional rotation-aware attention mechanism and adaptive kernel estimation, the model can more accurately capture and process blurring of different directions and degrees, significantly improving the clarity and detail of deblurred images.

[0111] 2. Enhance the model's adaptability and robustness: The adaptive module selection mechanism and dynamic weight allocation strategy enable the model to flexibly adjust processing methods and parameters according to the blur characteristics of different images, adapt to diverse blurry scenes, and improve the overall robustness of the system.

[0112] 3. Optimize computational efficiency and inference speed: Through a comprehensive displacement marking mechanism, structural reparameterization technology, and FP16 precision calculation, the model maintains high performance while significantly reducing computational complexity and resource consumption, making it suitable for efficient operation on resource-constrained devices.

[0113] 4. Improve the model's generalization ability: Multi-level data augmentation and expansion strategies allow the model to be exposed to various fuzzy types and complex scenarios during training, improving its generalization ability in practical applications and ensuring excellent performance in different environments.

[0114] 5. Provides a high-quality visual experience: The detail enhancement module based on perceptual loss optimization generates deblurred images that are more visually natural and richer in detail by comprehensively optimizing gradient, texture and content loss, significantly improving the user's visual experience and satisfaction.

[0115] 6. Real-time evaluation and feedback control: The real-time evaluation and feedback control mechanism ensures that the image processing effect meets expectations each time. The processing strategy is dynamically adjusted through a closed-loop control process to further improve the image restoration quality and ensure the reliability and stability of the system. Attached Figure Description

[0116] Figure 1 This is a structural diagram of the image deblurring model proposed in this invention.

[0117] Figure 2 A schematic diagram of the RWKV block for multidimensional rotational spectrum reconstruction.

[0118] Figure 3 This is a schematic diagram of the recurrent WKV attention mechanism. Detailed Implementation

[0119] The invention will now be described in further detail with reference to the accompanying drawings.

[0120] In one embodiment, the present invention proposes an image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism, comprising the following steps:

[0121] The collected images are preprocessed to obtain the training set. The preprocessing includes normalizing and color space conversion of the collected images, and automatically scaling and cropping them according to their actual size.

[0122] The image deblurring model is trained using the training set; the structure of the image deblurring model is as follows: Figure 1 Specifically, it includes: a shallow feature extraction module, a fuzziness sensitivity evaluation module, a dynamic weight allocation module, a high-dimensional feature decoupling and extraction module based on a hierarchical structure, an image restoration module, and a detail enhancement module based on perceptual loss optimization;

[0123] The shallow feature extraction module is used to extract the input image I. LQ Shallow features;

[0124] Use a 3×3 convolutional layer for I LQ Perform convolution to obtain shallow features C represents the number of channels (set to 64), and H and W represent the image height and width, respectively. The convolution operation can be represented as:

[0125] F S =Conv 3×3 (I LQ )

[0126] Among them, Conv 3×3 This indicates a convolution operation with a kernel size of 3×3.

[0127] The blur sensitivity evaluation module is used to evaluate the blur level of the input image and obtain a blur sensitivity map; specifically:

[0128] Using the Sobel operator on shallow features F S The gradients in the horizontal and vertical directions are calculated using the following formulas:

[0129] G x =Sobel x (F S ),G y =Sobel y (F S )

[0130] In the formula, G x It is the gradient in the horizontal direction, G y It is the gradient in the vertical direction, Sobel x It is the horizontal Sobel operator, Sobel y It is a Sobel operator in the vertical direction;

[0131] Calculate the gradient magnitude G mag :

[0132]

[0133] Based on the gradient magnitude, the degree of ambiguity is evaluated, and the ambiguity sensitivity map M is obtained:

[0134]

[0135] in, and These are the minimum and maximum values ​​of the gradient magnitude, respectively.

[0136] The dynamic weight allocation module is used to weight shallow features using a fuzzy sensitivity map to obtain weighted features; specifically:

[0137] F W =F S ⊙M

[0138] In the formula, F W As a weighted feature, F S For shallow features, M is a fuzzy sensitivity map, and ⊙ represents element-wise pointwise multiplication.

[0139] The high-dimensional feature decoupling and extraction module based on the hierarchical structure is used to extract deep features from the weighted features; the high-dimensional feature decoupling and extraction module based on the hierarchical structure adopts a four-level U-shaped encoder-decoder architecture, and each level is composed of multi-dimensional rotated spectrum reconstruction RWKV blocks;

[0140] The first-level encoder processes the weighted features by reconstructing RWKV blocks from N1 multidimensional rotated spectra, obtaining the first-level encoded feature F1, expressed by the following formula:

[0141]

[0142] In the formula, R-RWKV represents the multidimensional rotational spectrum reconstruction RWKV block, and FW Indicates weighted features;

[0143] Using a 1×1 convolutional layer Conv 1×1 Downsampling is performed using pixel rearrangement operations, as shown in the following formula:

[0144] F down1 =PixelUnshuffle(Conv 1×1 (F1))

[0145] In the formula, PixelUnshuffle represents the pixel rearrangement operation, F down1 This represents the first-level downsampling feature;

[0146] The second-stage encoder reconstructs the RWKV block from the first-stage downsampled feature F using N2 multidimensional rotating spectra. down1 After processing, the second-level coding feature F2 is obtained, and the formula is expressed as follows:

[0147]

[0148] The second-level encoded feature F2 is downsampled to obtain the second-level downsampled feature F. down2 ;

[0149] The third-level encoder reconstructs the RWKV block from the second-level downsampled feature F using N3 multidimensional rotating spectra. down2 The process yields the third-level coding feature F3. Then, a downsampling operation is performed on the third-level coding feature F3 to obtain the third-level downsampled feature F. down3 ;

[0150] The fourth-level encoder (bottleneck layer) reconstructs the RWKV block from the third-level downsampled feature F using N4 multidimensional rotational spectra. down3 The bottleneck feature F is obtained through processing. bottleneck ;

[0151] Bottleneck feature F bottleneck The input decoder is then subjected to upsampling and feature fusion, as detailed below:

[0152] In the third-level decoder, upsampling is performed using pixel rearrangement and a 1×1 convolutional layer, as shown in the following formula:

[0153] F up3 ′=Conv 1×1 (Pixelshuffle(F bottleneck ))

[0154] In the formula, PixelUnshuffle represents the pixel rearrangement operation, and Conv 1×1 F represents a 1×1 convolutional layer. up3′ represents the third-level upsampling feature;

[0155] The third-level upsampled features and the third-level encoded features are skipped together, as shown in the following formula:

[0156] F concat3 =Concat(F up3 ′,F3)

[0157] In the formula, F concat3 This indicates the third-level concatenation feature; Concat represents the concatenation operation.

[0158] Feature fusion is performed by reconstructing RWKV blocks using N3 multidimensional rotated spectra:

[0159] F decoder3 =R-RWKV N3 (F concat3 )

[0160] In the formula, F decoder3 This indicates the third-level decoding feature;

[0161] The third-level decoded features are input into the second-level decoder. In the second-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the second-level decoded features F. decoder2 ;

[0162] The second-level decoded features are input into the first-level decoder. In the first-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the first-level decoded features F. decoder1 ;

[0163] The first-level decoded features are input to the final-level decoder. The final-level decoder uses a multidimensional rotated spectrum to reconstruct the RWKV block on the first-level decoded features F. decoder1 The feature F is refined to obtain the depth feature. D .

[0164] The structure of the multidimensional rotating spectrum reconstruction RWKV block includes a spatial mixing module and a channel mixing module. The input and output of the spatial mixing module are residually connected, and the input and output of the channel mixing module are also residually connected.

[0165] The principle of the spatial mixing module is as follows:

[0166] Layer normalization is performed on the input feature X, as shown in the following formula:

[0167] X norm =LN(X)

[0168] In the formula, X norm For normalized features, LN is layer normalization;

[0169] Normalized feature X normInput a 360° omnidirectional labeling displacement layer, output the displacement feature X s ;

[0170] For the feature X after displacement s Linear projection to obtain the receiving gate R s Key K s Sum V s The formula is as follows:

[0171]

[0172] In the formula, These are the learnable weight matrices;

[0173] The global attention wkv is calculated using the recurrent WKV attention mechanism, as shown in the following formula:

[0174] wkv=Re-WKV(K s V s )

[0175] In the formula, Re-WKV is the recurrent WKV attention mechanism;

[0176] The attention result is modulated using the Sigmoid function to obtain the output O of the spatial mixing module. s :

[0177]

[0178] in, Let σ(·) be the weight matrix of the output projection, and let σ(·) be the Sigmoid function.

[0179] The principle of the channel mixing module is as follows:

[0180] The output O of the spatial mixing module s The input feature X of the spatial mixing module is added to the input feature X of the channel mixing module and then used as the input of the channel mixing module. For input Perform layer normalization, input the normalized features into the omnidirectional labeled displacement layer, and output the displacement features X. c ;

[0181] For the feature X after displacement c Linear projection to obtain the receiving gate R c and key K c :

[0182] K c =X c W K

[0183] In the formula, and W KAll are learnable weight matrices;

[0184] The key K is processed using the squared ReLU activation function. c And obtain the value V by linear projection. c :

[0185]

[0186] In the formula, Let be the weight matrix, and use ReLU(·). 2 It is the squared ReLU activation function;

[0187] The output O of the channel mixing module is obtained by receiving gate modulation. c :

[0188]

[0189] In the formula, σ(·) is the Sigmoid function. This is the weight matrix;

[0190] The output O of the channel mixing module c Input of the channel mixing module Adding them together yields the output X of the multidimensional rotational spectrum reconstruction RWKV block. out .

[0191] The omnidirectional labeling displacement layer is a multi-branch convolutional structure, with each branch corresponding to a different receptive field. Branch 1 is a 1×1 depthwise convolution, branch 2 is a 3×3 depthwise convolution, branch 3 is a 5×5 depthwise convolution, and branch 4 is an identity mapping. The outputs of each branch are weighted and fused.

[0192]

[0193] Where, α i For learnable fusion weights, X os This indicates the features of the input omnidirectional labeled displacement layer, Branch i (·) represents the i-th branch, and Omni-Shift(·) represents the omnidirectional marking displacement layer.

[0194] The recurrent WKV attention mechanism specifically involves recursively applying bidirectional WKV attention in different scanning directions, as detailed below:

[0195] Set the number of loops M and the set of scan directions D, where D = {horizontal, vertical};

[0196] Initialize wkv (0) =V s wkv (0) For initial global attention, V s Represents the value;

[0197] For loop counts j = 1 to M, perform the following steps sequentially:

[0198] Select the scan direction d for the j-th scan. j ∈D

[0199] According to d j Change key K s The global attention wkv in the (j-1)th iteration (j-1) The order of arrangement;

[0200] Applying bidirectional WKV attention, we obtain the global attention wkv for the j-th iteration. (j) :

[0201] wkv (j) =Bi-WKV(K s wkv (j-1) )

[0202] In the formula, Bi-WKV(·) represents bidirectional WKV attention;

[0203] When the maximum number of iterations is reached, output the global attention wkv for the Mth iteration. (M) wkv serves as the final global attention.

[0204] The image restoration module is used to generate a residual image based on depth features. The residual image is added to the input image to obtain a deblurred image; the depth features F D The residual image I is generated through a 3×3 convolutional layer. R :

[0205] I R =Conv 3×3 (F D )

[0206] Residual image I R With input image I LQ Adding them together yields the restored high-quality image:

[0207]

[0208] In the formula, To deblur the image.

[0209] The detail enhancement module based on perceptual loss optimization is used to construct the loss function, which includes reconstruction loss, gradient loss, texture loss, and content loss. The perceptual loss function calculates the difference between the deblurred image and the original image in the high-level feature space, guiding the image deblurring model to refine local details while maintaining overall structural consistency.

[0210] The loss function is as follows:

[0211] L total =λ reconstruction L reconstruction +λ grad L grad +λ texture L texture +λ content L content

[0212] In the formula, L total For the total loss, λ reconstruction To reconstruct the weights of the loss, L reconstruction For reconstruction loss, λ grad L represents the weights of the gradient loss. grad For gradient loss, λ texture L represents the weights of the texture loss. texture For texture loss, λ content L is the weight for content loss. content This results in content loss.

[0213] Calculate the gradient of the total loss with respect to the model parameters θ:

[0214]

[0215] Update model parameters using an optimization algorithm (Adam optimizer):

[0216]

[0217] Where η is the learning rate.

[0218] The entire training set is traversed, and multiple rounds of training are performed until the image deblurring model converges.

[0219] Image deblurring is performed using a trained image deblurring model.

[0220] The effectiveness of this invention has been verified through a series of rigorous experimental designs and data comparison analyses. The specific steps are as follows:

[0221] First, blurred images were deblurred on three widely used deblurring datasets—RealBlur-J, RealBlur-R, and REDS—to generate high-resolution reconstructed images. Then, the deblurred images were compared one-to-one with the corresponding original high-resolution images, and two key metrics—Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)—were used to quantify the model's performance. These two metrics comprehensively evaluate the deblurring effect from the perspectives of image quality and structural consistency, respectively.

[0222] Next, the deblurring performance of this invention was systematically compared with existing mainstream deblurring methods on the same RealBlur-J, RealBlur-R, and REDS datasets. The experimental results are detailed in Tables 1, 2, and 3. The results clearly show that the model of this invention achieves significant improvements in both PSNR and SSIM metrics, demonstrating superior deblurring performance compared to existing methods. These experimental results fully demonstrate the excellent performance and practical application value of this invention in image deblurring tasks.

[0223] Table 1 shows the performance comparison of each method on the RealBlur-R dataset.

[0224] SRN MPRNet MAXIM DeepRFT+ Stripformer FFTformer This invention PSNR 38.65 39.31 39.45 40.01 39.84 40.11 40.24 SSIM 0.965 0.972 0.962 0.973 0.974 0.975 0.974

[0225] Table 2 shows the performance comparison of each method on the RealBlur-J dataset.

[0226] SRN MPRNet MAXIM DeepRFT+ Stripformer FFTformer This invention PSNR 31.38 31.76 32.84 32.63 32.48 32.62 32.92 SSIM 0.909 0.922 0.935 0.933 0.929 0.933 0.933

[0227] Table 3 shows the performance comparison of each method on the REDS dataset.

[0228] MPRNet HINet MAXIM NAFNet64 This invention PSNR 28.79 28.83 28.93 29.09 29.20 SSIM 0.811 0.862 0.865 0.867 0.868

[0229] In summary, the method proposed in this invention not only demonstrates leading performance on mainstream datasets, but also provides strong technical support and reference for further research in the field of image deblurring.

[0230] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. An image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism, characterized in that, Includes the following steps: The collected images are preprocessed to obtain the training set; Train an image deblurring model using the training set; Image deblurring is performed using a trained image deblurring model.

2. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 1, characterized in that, The preprocessing of the acquired images specifically involves: The system normalizes and converts the color space of the collected images, and automatically scales and crops them according to their actual size.

3. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 1, characterized in that, The image deblurring model specifically includes: a shallow feature extraction module, a blur sensitivity evaluation module, a dynamic weight allocation module, a high-dimensional feature decoupling and extraction module based on a hierarchical structure, an image restoration module, and a detail enhancement module based on perceptual loss optimization. The shallow feature extraction module is used to extract shallow features from the input image; The blur sensitivity evaluation module is used to evaluate the blur degree of the input image and obtain a blur sensitivity map; The dynamic weight allocation module is used to weight shallow features using a fuzzy sensitivity map to obtain weighted features; The high-dimensional feature decoupling and extraction module based on hierarchical structure is used to extract deep features from weighted features; The image restoration module is used to generate a residual image based on depth features, and the residual image is added to the input image to obtain a deblurred image; The detail enhancement module based on perceptual loss optimization is used to construct a loss function. The perceptual loss function is used to calculate the difference between the deblurred image and the original image in the high-level feature space, guiding the image deblurring model to refine local details while maintaining the overall structural consistency.

4. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 3, characterized in that, The process of evaluating the blur level of the input image to obtain a blur sensitivity map is as follows: Using the Sobel operator on shallow features F S The gradients in the horizontal and vertical directions are calculated using the following formulas: G x =Sobel x (F S ),G y =Sobel y (F S ) In the formula, G x It is the gradient in the horizontal direction, G y It is the gradient in the vertical direction, Sobel x It is a horizontal Sobel operator, Sobel y It is a Sobel operator in the vertical direction; Calculate the gradient magnitude G mag : Based on the gradient magnitude, the degree of ambiguity is evaluated, and the ambiguity sensitivity map M is obtained: in, and These are the minimum and maximum values ​​of the gradient magnitude, respectively.

5. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 3, characterized in that, The process of weighting shallow features using a fuzzy sensitivity map to obtain weighted features is as follows: F W =F S ⊙M In the formula, F W As a weighted feature, F S For shallow features, M is a fuzzy sensitivity map, and ⊙ represents element-wise pointwise multiplication.

6. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 3, characterized in that, The high-dimensional feature decoupling and extraction module based on hierarchical structure adopts a four-level U-shaped encoder-decoder architecture, with each level consisting of a multi-dimensional rotated spectrum reconstruction RWKV block; The first-level encoder processes the weighted features by reconstructing RWKV blocks from N1 multidimensional rotated spectra, obtaining the first-level encoded feature F1, expressed by the following formula: In the formula, R-RWKV represents the multidimensional rotational spectrum reconstruction RWKV block, and F W Indicates weighted features; Using a 1×1 convolutional layer Conv 1×1 Downsampling is performed using pixel rearrangement operations, as shown in the following formula: F down1 =PixelUnshuffle(Conv 1×1 (F1)) In the formula, PixelUnshuffle represents the pixel rearrangement operation, F down1 This represents the first-level downsampling feature; The second-stage encoder reconstructs the RWKV block from the first-stage downsampled feature F using N2 multidimensional rotating spectra. down1 After processing, the second-level coding feature F2 is obtained, and the formula is expressed as follows: The second-level encoded feature F2 is downsampled to obtain the second-level downsampled feature F. down2 ; The third-level encoder reconstructs the RWKV block from the second-level downsampled feature F using N3 multidimensional rotating spectra. down2 The process yields the third-level coding feature F3. Then, a downsampling operation is performed on the third-level coding feature F3 to obtain the third-level downsampled feature F. down3 ; The fourth-level encoder reconstructs the RWKV block from the third-level downsampled feature F using N4 multidimensional rotational spectra. down3 The bottleneck feature F is obtained through processing. bottleneck ; Bottleneck feature F bottleneck The input decoder is then subjected to upsampling and feature fusion, as detailed below: In the third-level decoder, upsampling is performed using pixel rearrangement and a 1×1 convolutional layer, as shown in the following formula: F up3 ′=Conv 1×1 (Pixelshuffle(F bottleneck )) In the formula, PixelUnshuffle represents the pixel rearrangement operation, and Conv 1×1 F represents a 1×1 convolutional layer. up3 ′ represents the third-level upsampling feature; The third-level upsampled features and the third-level encoded features are skipped together, as shown in the following formula: F concat3 =Concat(F up3 ′,F3) In the formula, F concat3 This indicates the third-level concatenation feature; Concat represents the concatenation operation. Feature fusion is performed by reconstructing RWKV blocks using N3 multidimensional rotated spectra: F decoder3 =R-RWKV N3 (F concat3 ) In the formula, F decoder3 Indicates the third-level decoding feature; The third-level decoded features are input into the second-level decoder. In the second-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the second-level decoded features F. decoder2 ; The second-level decoded features are input into the first-level decoder. In the first-level decoder, upsampling, skip connections, and feature fusion are repeated to obtain the first-level decoded features F. decoder1 ; The first-level decoded features are input to the final-level decoder. The final-level decoder uses a multidimensional rotated spectrum to reconstruct the RWKV block on the first-level decoded features F. decoder1 The feature F is refined to obtain the depth feature. D .

7. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 6, characterized in that, The structure of the multidimensional rotating spectrum reconstruction RWKV block includes a spatial mixing module and a channel mixing module. The input and output of the spatial mixing module are residually connected, and the input and output of the channel mixing module are also residually connected. The principle of the spatial mixing module is as follows: Layer normalization is performed on the input feature X, as shown in the following formula: X norm =LN(X) In the formula, X norm For normalized features, LN is layer normalization; Normalized feature X norm Input a 360° omnidirectional labeling displacement layer, output the displacement feature X s ; For the feature X after displacement s Linear projection to obtain the receiving gate R s Key K s Sum V s The formula is as follows: In the formula, These are the learnable weight matrices; The global attention wkv is calculated using the recurrent WKV attention mechanism, as shown in the following formula: wkv=Re-WKV(K s ,V s ) In the formula, Re-WKV is the recurrent WKV attention mechanism; The attention result is modulated using the Sigmoid function to obtain the output O of the spatial mixing module. s : in, Let σ(·) be the weight matrix of the output projection, and let σ(·) be the Sigmoid function. The principle of the channel mixing module is as follows: The output O of the spatial mixing module s The input feature X of the spatial mixing module is added to the input feature X of the channel mixing module and then used as the input of the channel mixing module. For input Perform layer normalization, input the normalized features into the omnidirectional labeled displacement layer, and output the displacement features X. c ; For the feature X after displacement c Linear projection to obtain the receiving gate R c and key K c : K c =X c W K In the formula, and W K All are learnable weight matrices; The key K is processed using the squared ReLU activation function. c And obtain the value V by linear projection. c : In the formula, Let be the weight matrix, and use ReLU(·). 2 It is the squared ReLU activation function; The output O of the channel mixing module is obtained by receiving gate modulation. c : In the formula, σ(·) is the Sigmoid function. This is the weight matrix; The output O of the channel mixing module c Input of the channel mixing module Adding them together yields the output X of the multidimensional rotational spectrum reconstruction RWKV block. out .

8. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 7, characterized in that, The omnidirectional labeling and displacement layer is a multi-branch convolutional structure, with each branch corresponding to a different receptive field. Branch 1 is a 1×1 depthwise convolution, branch 2 is a 3×3 depthwise convolution, branch 3 is a 5×5 depthwise convolution, and branch 4 is an identity mapping. The outputs of each branch are weighted and fused. Where, α i For learnable fusion weights, X os This indicates the features of the input omnidirectional labeled displacement layer, Branch i (·) represents the i-th branch, and Omni-Shift(·) represents the omnidirectional marking displacement layer.

9. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 7, characterized in that, The cyclic WKV attention mechanism specifically involves recursively applying bidirectional WKV attention in different scanning directions, as detailed below: Set the number of loops M and the set of scan directions D, where D = {horizontal, vertical}; Initialize wkv (0) =V s wkv (0) For initial global attention, V s Represents the value; For loop counts j = 1 to M, perform the following steps sequentially: Select the scan direction d for the j-th scan. j ∈D According to d j Change key K s The global attention wkv in the (j-1)th iteration (j-1) The order of arrangement; Applying bidirectional WKV attention, we obtain the global attention wkv for the j-th iteration. (j) : wkv (j) =Bi-WKV(K s ,wkv (j-1) ) In the formula, Bi-WKV(·) represents bidirectional WKV attention; When the maximum number of iterations is reached, output the global attention wkv for the Mth iteration. (M) wkv serves as the final global attention.

10. The image deblurring method based on rotation-aware multidimensional attention and blur-sensitive adaptive allocation mechanism as described in claim 3, characterized in that, The loss function includes reconstruction loss, gradient loss, texture loss, and content loss.

Images