Image super-resolution reconstruction method and system based on progressive frequency domain guided adaptive sparse attention

Abstract

A method and system for image super-resolution reconstruction based on progressively frequency-domain guided adaptive sparse attention, belonging to the fields of image processing and computer vision technology. This invention aims to solve the problems of high computational complexity of attention, lack of adaptive resource allocation, insufficient utilization of high-frequency priors in the frequency domain, and accumulation and loss of inter-layer sparse information in existing methods. The method includes: preprocessing a low-resolution image and extracting shallow features; concatenating multiple progressively frequency-domain guided adaptive sparse reconstruction blocks to progressively interact and transform the shallow features, generating deep reconstruction features; fusing deep and shallow features using global residuals, and then upsampling to reconstruct and output a high-resolution image. This invention introduces a frequency-domain prior extraction branch, an adaptive sparse prediction branch, and a progressively sparse support set propagation mechanism, which can adaptively allocate attention computational resources, reduce computational redundancy, and improve high-frequency detail recovery and reconstruction quality. It is applicable to image processing.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology. Background Technology

[0002] Image super-resolution reconstruction aims to recover high-resolution images from low-resolution images. It has important application value in fields such as intelligent security, remote sensing imaging, medical imaging, industrial inspection, video enhancement, and mobile terminal imaging. By performing super-resolution reconstruction on low-resolution images, the spatial resolution and detail representation of images can be improved, thereby providing a higher quality data foundation for subsequent tasks such as target recognition, structural analysis, defect detection, and visual understanding.

[0003] Existing image super-resolution reconstruction methods mainly include traditional interpolation-based methods and deep learning-based methods. While traditional interpolation-based methods are simple to implement and have low computational overhead, they struggle to effectively recover high-frequency textures and edge details, and are prone to blurring. Convolutional neural network-based reconstruction methods achieve significant improvements in reconstruction quality by learning the nonlinear mapping relationship between low-resolution and high-resolution images end-to-end, but their ability to model long-distance dependencies and global context information is limited by the local convolutional receptive field. Transformer-based image super-resolution methods enhance global feature interaction capabilities through self-attention mechanisms, exhibiting good performance in texture restoration and structural reconstruction. However, they typically require large-scale correlation calculations of features within a window or globally. As feature resolution increases, their computational and storage overhead increases significantly, limiting the efficiency of their application in high-resolution image reconstruction scenarios.

[0004] Meanwhile, most existing methods employ fixed dense attention or fixed sparse attention strategies, which make it difficult to adaptively allocate computational resources according to the texture complexity and structural information importance of different image regions. This can easily lead to computational redundancy or insufficient retention of key information. Furthermore, they do not make full use of high-frequency prior information in the frequency domain and fail to fully explore the important guiding role of high-frequency responses such as edges and textures in super-resolution reconstruction. This can result in problems such as oversmoothing, texture loss, or artifact enhancement during detail restoration. In addition, although some sparse attention methods can reduce computation, they are prone to accumulating information loss during inter-layer propagation due to the fixed or insufficiently updated sparse support set, which can affect the expression of deep features and the final reconstruction effect. Summary of the Invention

[0005] This invention aims to overcome the technical problems of existing image super-resolution reconstruction methods, such as high computational complexity of attention, lack of adaptive resource allocation, insufficient utilization of high-frequency priors in the frequency domain, and accumulation and loss of sparse information between layers. It provides an image super-resolution reconstruction method and system based on progressive frequency domain guided adaptive sparse attention.

[0006] The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention as described in this invention includes:

[0007] Step 1: Obtain the low-resolution image to be reconstructed and preprocess the low-resolution image;

[0008] Step 2: Perform convolutional feature extraction on the preprocessed low-resolution image to obtain shallow features;

[0009] Step 3: By connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series, progressive feature interaction and transformation are performed on the shallow features to generate deep reconstruction features;

[0010] Each progressive frequency domain guided adaptive sparse reconstruction block consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is then transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block.

[0011] Step 4: Use the global residual fusion method to fuse the deep reconstruction features and the shallow features, and then upsample and reconstruct the fused features to output a high-resolution reconstructed image.

[0012] Furthermore, in this invention, the method for preprocessing the low-resolution image in step one is as follows:

[0013] The low-resolution image is filled with reflection to adapt the image size to the preset window size; mean correction and numerical range normalization are performed on the filled image to obtain the preprocessed low-resolution image.

[0014] Furthermore, in this invention, the method for obtaining shallow features in step two is as follows:

[0015] Convolution operations are used to map low-resolution images into shallow features.

[0016] Furthermore, in this invention, in step three, each of the progressive frequency domain guided adaptive sparse transform layers includes a first normalization unit, a linear projection unit, a window attention unit, a first residual connection unit, a second normalization unit, a gated convolutional feedforward network unit, and a second residual connection unit connected in sequence.

[0017] The first normalization unit is used to perform layer normalization processing on the input features to obtain stable distribution features;

[0018] The linear projection unit is used to perform a linear mapping on the normalized distribution features of the input, and to query the features Q, key features K, and value features V.

[0019] The window attention unit is used to perform window-based adaptive sparse attention calculation on the query features, key features, and value features to obtain attention aggregation features;

[0020] The first residual connection unit is used to perform a residual connection between the attention aggregation feature and the input of the linear projection unit and then output the result to the second normalization unit.

[0021] The second normalization unit is used to perform layer normalization processing on the output of the first residual connection unit;

[0022] The gated convolutional feedforward network unit is used to perform nonlinear transformation and local detail modeling on the normalized distribution features to obtain enhanced features;

[0023] The second residual connection unit is used to add the enhanced features to the output of the first residual connection unit to obtain the output of the current asymptotic frequency domain guided adaptive sparse transform layer.

[0024] Furthermore, in this invention, the enhanced feature is:

[0025] ;

[0026] in, This represents the output characteristics of a gated convolutional feedforward network unit. Represents the second linear mapping. This indicates local modeling in depthwise convolution. This represents the gated activation function. This represents element-wise multiplication. and This represents the two feature branches obtained after branching the output of the first linear mapping.

[0027] Furthermore, in this invention, the window attention unit includes a frequency domain prior extraction branch, an adaptive sparse prediction branch, a progressive prior fusion module, a Top-k sparse selection module, and an attention aggregation module;

[0028] The frequency domain prior extraction branch is used for,

[0029] Channel compression is performed on the input features to obtain a single-channel feature map;

[0030] High-pass filtering is performed on the single-channel feature map to obtain high-frequency response information;

[0031] The high-frequency response information is used to generate a frequency steering fraction, and a frequency domain steering bias term is constructed based on the frequency steering fraction.

[0032] The progressive prior fusion module is used for,

[0033] The attention distribution corresponding to the sparse support set of the current layer is fused with the attention distribution corresponding to the sparse support set of the previous layer to obtain the fused attention distribution of the sparse support set of the current layer.

[0034] The adaptive sparse prediction branch is used for,

[0035] Based on the sparse support set attention distribution of the current layer, construct the attention log terms after fusion for the current window;

[0036] Using the attention logarithm of the current window fusion, query feature Q, key feature K, attention distribution of the sparse support set of the current layer, frequency domain guided bias term and asymptotic prior bias term, the window attention formula is used to calculate the target attention weight of the current asymptotic frequency domain guided adaptive sparse transform layer, and the number of weights retained corresponding to the target attention weights is taken as the number of attention retained in the current window.

[0037] The attention window position bias term is a bias term that determines the relative spatial relationship between the pixel feature positions within the encoding window;

[0038] The Top-k sparse screening module is used for,

[0039] The number of weights retained is subjected to Top-k sparse filtering (the top k highest probability samples) to obtain the sparse support set of the current asymptotic frequency domain guided adaptive sparse transform layer.

[0040] The attention aggregation module is used for,

[0041] Attention aggregation features are obtained by using the sparse support set of the current asymptotic frequency domain-guided adaptive sparse transform layer and the value feature V to perform attention aggregation calculation.

[0042] Furthermore, in this invention, the formula for the frequency-guided fraction of high-frequency response information generation is:

[0043] ;

[0044] in, Frequency-guided fractions This represents the two-dimensional features obtained by recovering the features from the input sequence. This indicates a channel compression operation. This indicates a high-pass filtering operation. Indicates the flattening operation. This indicates a normalization operation.

[0045] Furthermore, in this invention, the attention logarithm of the current window after fusion is:

[0046] ;

[0047] in, Indicates the first The attention logarithm after layer fusion Indicates the first The current attention of the layer is on several items. This represents the attention distribution of the sparse support set in the previous layer. Indicates the prior fusion weights. Indicates a stable term. This represents the logarithmic transformation function, used to compress the numerical range of the prior distribution and improve fusion stability.

[0048] Furthermore, in this invention, the formula for calculating window attention is:

[0049] ;

[0050] in, Indicates attention weights, Representing feature dimension, Indicates the position offset term. This represents the frequency-domain guided bias term constructed based on the frequency-guided fraction. This represents the asymptotic prior bias term. Indicates frequency domain guiding weights, Let Q represent the exponential normalization function, and let Q represent the query feature. Represents the transpose of the key feature, wherein, the , and It constitutes the logarithmic terms of attention. The three bias components.

[0051] Furthermore, in this invention, the high-pass filtering operation is implemented using gradient magnitude extraction based on the Sobel operator, second-order high-frequency response extraction based on the Laplacian operator, or high-frequency response extraction based on a learnable high-pass convolution kernel.

[0052] Furthermore, in this invention, the number of attention-retained elements in the current window is:

[0053] ;

[0054] ;

[0055] in, This represents the probability distribution of each candidate sparsity level. This represents a multilayer perceptron. Indicates the number of targets whose attention is retained. Represents a predefined set of sparse levels. This refers to the operation of determining the number of target attention to retain from the preset sparse level set according to preset selection rules; Represents the sequence features within the window. Indicates global average pooling. Indicates global max pooling. This represents window-level frequency-guided statistics. Presentation layer identifier embedding.

[0056] The preset sparsity level set includes at least two of 16, 32, 64, and 128, and the dynamic selection is implemented using the maximum probability selection method or the Gumbel-Softmax discrete sampling method.

[0057] Furthermore, in this invention, the sparse support set of the current asymptotic frequency domain guided adaptive sparse transform layer obtained by the Top-k sparse screening module is:

[0058] ;

[0059] in, Indicates the first The updated sparse support set. Indicates the first The number of attention points retained for each layer. This indicates selecting the top from the attention distribution. The filtering operation consists of the element corresponding to the largest response or its index.

[0060] An image super-resolution reconstruction system based on asymptotically frequency-domain guided adaptive sparse attention, comprising:

[0061] Preprocessing module and image super-resolution reconstruction network;

[0062] The preprocessing module is used to acquire the low-resolution image to be reconstructed and perform preprocessing.

[0063] The image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature reconstruction module, a feature fusion module, and an upsampling reconstruction module;

[0064] The shallow feature extraction module is used to perform convolutional feature extraction on the input low-resolution image to obtain shallow features.

[0065] The deep feature reconstruction module generates deep reconstructed features by connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series and progressively interacting and transforming shallow features.

[0066] Each progressive frequency domain guided adaptive sparse reconstruction block consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is then transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block.

[0067] The feature fusion module uses global residual fusion to fuse the deep reconstruction features and shallow features, and then uses the upsampling reconstruction module to reassemble pixels to obtain a high-resolution reconstructed image.

[0068] This invention introduces a frequency domain prior extraction branch, an adaptive sparse prediction branch, and a progressively sparse support set propagation mechanism into the image super-resolution reconstruction network. This enables the network to adaptively allocate attention computational resources based on the high-frequency response information and structural complexity of different regions of the input image, while maintaining effective sparse support set update capabilities during inter-layer propagation. Compared to existing technologies, this invention can improve the recovery of high-frequency details such as edges and textures while reducing attention computational redundancy and storage overhead, enhancing the expression of deep features, and mitigating problems such as over-smoothing, texture loss, and information accumulation and loss. Therefore, it improves the quality and computational efficiency of image super-resolution reconstruction and has promising application prospects. Attached Figure Description

[0069] Figure 1 This is a flowchart of the method described in this invention;

[0070] Figure 2 A block diagram illustrating the principle of progressive frequency domain-guided adaptive sparse reconstruction blocks;

[0071] Figure 3 Block diagram of the principle of guiding the adaptive sparse transform layer in the asymptotic frequency domain;

[0072] Figure 4A block diagram illustrating the principle of an adaptive sparse window attention unit guided by the asymptotic frequency domain. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0074] Specific implementation method one: Refer to Figures 1 to 4 This embodiment specifically describes the image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention, which includes:

[0075] Step 1: Obtain the low-resolution image to be reconstructed and preprocess the low-resolution image;

[0076] Step 2: Perform convolutional feature extraction on the preprocessed low-resolution image to obtain shallow features;

[0077] Step 3: By connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series, progressive feature interaction and transformation are performed on the shallow features to generate deep reconstruction features;

[0078] Each progressive frequency domain guided adaptive sparse reconstruction block consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is then transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block.

[0079] Step 4: Use the global residual fusion method to fuse the deep reconstruction features and the shallow features, and then upsample and reconstruct the fused features to output a high-resolution reconstructed image.

[0080] In the field of image super-resolution reconstruction technology, a high-resolution image refers to an image whose spatial resolution is improved relative to the input low-resolution image. The resolution improvement factor is determined by the magnification factor of the image super-resolution reconstruction network, and the specific magnification factor can be set according to the actual application scenario, including but not limited to 2x, 3x, or 4x.

[0081] Furthermore, in this embodiment, the method for preprocessing the low-resolution image in step one is as follows:

[0082] The low-resolution image to be reconstructed is filled with reflection to make the image size fit the preset window size; mean correction and numerical range normalization are performed on the filled image to obtain the preprocessed low-resolution image.

[0083] Furthermore, in this embodiment, the method for obtaining shallow features in step two is as follows:

[0084] Convolution operations are used to map low-resolution images into shallow features.

[0085] Furthermore, in this embodiment, in step three, each of the progressive frequency domain guided adaptive sparse transform layers includes a first normalization unit, a linear projection unit, a window attention unit, a first residual connection unit, a second normalization unit, a gated convolutional feedforward network unit, and a second residual connection unit connected in sequence.

[0086] The first normalization unit is used to perform layer normalization processing on the input features to obtain stable distribution features;

[0087] The linear projection unit is used to perform a linear mapping on the normalized distribution features of the input, and to query the features Q, key features K, and value features V.

[0088] The window attention unit is used to perform window-based adaptive sparse attention calculation on the query features, key features, and value features to obtain attention aggregation features;

[0089] The first residual connection unit is used to perform a residual connection between the attention aggregation feature and the input of the linear projection unit and then output the result to the second normalization unit.

[0090] The second normalization unit is used to perform layer normalization processing on the output of the first residual connection unit;

[0091] The gated convolutional feedforward network unit is used to perform nonlinear transformation and local detail modeling on the normalized distribution features to obtain enhanced features;

[0092] The second residual connection unit is used to add the enhanced features to the output of the first residual connection unit to obtain the output of the current asymptotic frequency domain guided adaptive sparse transform layer.

[0093] Furthermore, in this embodiment, the enhanced feature is:

[0094] ;

[0095] in, This represents the output characteristics of a gated convolutional feedforward network unit. Represents the second linear mapping. This indicates local modeling in depthwise convolution. This represents the gated activation function. This represents element-wise multiplication. and This represents the two feature branches obtained after branching the output of the first linear mapping.

[0096] Furthermore, in this embodiment, the window attention unit includes a frequency domain prior extraction branch, an adaptive sparse prediction branch, a progressive prior fusion module, a Top-k sparse filtering module, and an attention aggregation module;

[0097] The frequency domain prior extraction branch is used for,

[0098] Channel compression is performed on the input features to obtain a single-channel feature map;

[0099] High-pass filtering is performed on the single-channel feature map to obtain high-frequency response information;

[0100] The high-frequency response information is used to generate a frequency steering fraction, and a frequency domain steering bias term is constructed based on the frequency steering fraction.

[0101] The progressive prior fusion module is used for,

[0102] The attention distribution corresponding to the sparse support set of the current layer is fused with the attention distribution corresponding to the sparse support set of the previous layer to obtain the fused attention distribution of the sparse support set of the current layer.

[0103] The adaptive sparse prediction branch is used to construct the attention logarithm of the current window fusion based on the sparse support set attention distribution of the current layer.

[0104] Using the attention logarithm of the current window fusion, query feature Q, key feature K, attention distribution of the sparse support set of the current layer, frequency domain guided bias term and asymptotic prior bias term, the window attention formula is used to calculate the target attention weight of the current asymptotic frequency domain guided adaptive sparse transform layer, and the number of weights retained corresponding to the target attention weights is taken as the number of attention retained in the current window.

[0105] The attention window position bias term is a bias term that determines the relative spatial relationship between the pixel feature positions within the encoding window;

[0106] The Top-k sparse filtering module is used to perform Top-k sparse filtering on the number of weights to retain (sampling the top k highest probability samples) and obtain the sparse support set of the current asymptotic frequency domain guided adaptive sparse transform layer.

[0107] The attention aggregation module is used for,

[0108] Attention aggregation features are obtained by using the sparse support set of the current asymptotic frequency domain-guided adaptive sparse transform layer and the value feature V to perform attention aggregation calculation.

[0109] Furthermore, in this embodiment, the formula for the frequency-guided fraction used to generate high-frequency response information is:

[0110] ;

[0111] in, Frequency-guided fractions This represents the two-dimensional features obtained by recovering the features from the input sequence. This indicates a channel compression operation. This indicates a high-pass filtering operation. Indicates the flattening operation. This indicates a normalization operation;

[0112] Furthermore, in this embodiment, the attention logarithm of the current window after fusion is:

[0113] ;

[0114] in, Indicates the first The attention logarithm after layer fusion Indicates the first The current attention of the layer is on several items. This represents the attention distribution of the sparse support set in the previous layer. Indicates the prior fusion weights. Indicates a stable term. This represents the logarithmic transformation function, used to compress the numerical range of the prior distribution and improve fusion stability;

[0115] Furthermore, in this embodiment, the formula for calculating window attention is:

[0116] ;

[0117] in, Indicates attention weights, Representing feature dimension, Indicates the position offset term. This represents the frequency-domain guided bias term constructed based on the frequency-guided fraction. This represents the asymptotic prior bias term. Indicates frequency domain guiding weights, Let Q represent the exponential normalization function, and let Q represent the query feature. Represents the transpose of the key feature, wherein, the , and It constitutes the logarithmic terms of attention. The three bias components.

[0118] Furthermore, in this embodiment, the high-pass filtering operation is implemented using gradient magnitude extraction based on the Sobel operator, second-order high-frequency response extraction based on the Laplacian operator, or high-frequency response extraction based on a learnable high-pass convolution kernel.

[0119] Furthermore, in this embodiment, the number of attention-retaining elements in the current window is:

[0120] ;

[0121] ;

[0122] in, This represents the probability distribution of each candidate sparsity level. This represents a multilayer perceptron. Indicates the number of targets whose attention is retained. Represents a predefined set of sparse levels. This refers to the operation of determining the number of target attention to retain from the preset sparse level set according to preset selection rules; Represents the sequence features within the window. Indicates global average pooling. Indicates global max pooling. This represents window-level frequency-guided statistics. Presentation layer identifier embedding.

[0123] The preset sparsity level set includes at least two of 16, 32, 64, and 128, and the dynamic selection is implemented using the maximum probability selection method or the Gumbel-Softmax discrete sampling method.

[0124] Furthermore, in this embodiment, the sparse support set of the current asymptotic frequency domain guided adaptive sparse transform layer obtained by the Top-k sparse screening module is:

[0125] ;

[0126] in, Indicates the first The updated sparse support set. Indicates the first The number of attention points retained for each layer. This indicates selecting the top from the attention distribution. The filtering operation consists of the element corresponding to the largest response or its index.

[0127] Specific Implementation Method Two: The image super-resolution reconstruction system based on progressive frequency domain guided adaptive sparse attention described in this implementation method includes:

[0128] Preprocessing module and image super-resolution reconstruction network;

[0129] The preprocessing module is used to acquire the low-resolution image to be reconstructed and perform preprocessing.

[0130] The image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature reconstruction module, a feature fusion module, and an upsampling reconstruction module;

[0131] The shallow feature extraction module is used to perform convolutional feature extraction on the input low-resolution image to obtain shallow features.

[0132] The deep feature reconstruction module generates deep reconstructed features by connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series and progressively interacting and transforming shallow features.

[0133] Each progressive frequency domain guided adaptive sparse reconstruction block consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is then transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block.

[0134] The feature fusion module uses global residual fusion to fuse the deep reconstruction features and shallow features, and then uses the upsampling reconstruction module to reassemble pixels to obtain a high-resolution reconstructed image.

[0135] Specific implementation: An image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention includes the following steps:

[0136] S1. Obtain the low-resolution image to be reconstructed and preprocess the low-resolution image.

[0137] The specific implementation methods of preprocessing include:

[0138] S1.1. Obtain the low-resolution image to be reconstructed, denoted as... .

[0139] S1.2. Perform a reflection fill operation on the low-resolution image to adapt its spatial dimensions to a preset window size; assuming the original image size is... The default window size is The size of the filled image is denoted as . ,in and All are window sizes Integer multiples of. Among them, the preset window size. Typically 16×16 or 32×32;

[0140] S1.3. Perform mean correction and numerical range normalization on the filled image to obtain the network input image. Its expression can be represented as:

[0141] ;

[0142] in, This represents the low-resolution image after padding. This represents the mean term. This represents the scaling factor.

[0143] S2. Input the preprocessed low-resolution image into the shallow feature extraction module to extract shallow features.

[0144] The specific implementation method for extracting shallow features is as follows:

[0145] S2.1. Transfer the input image Input to the shallow feature extraction module, and obtain shallow features through convolution operations. It can be represented as:

[0146] ;

[0147] Among them, it means express Convolution mapping operation.

[0148] S2.2. The shallow features Before being input into the deep feature reconstruction module, the two-dimensional features are mapped to serialized features through a serialization embedding unit. It can be represented as:

[0149] ;

[0150] in, This represents a serialization embedding operation that flattens, transposes, and normalizes two-dimensional features.

[0151] S3. Input the shallow features into the deep feature reconstruction module for feature reconstruction to obtain deep reconstructed features.

[0152] The specific implementation method for obtaining deep reconstruction features:

[0153] S3.1. The deep feature reconstruction module is composed of multiple progressively frequency-domain guided adaptive sparse reconstruction blocks connected in series. Let the first... The inputs and outputs of each asymptotically frequency-domain guided adaptive sparse reconstruction block are respectively and Then we have:

[0154] ;

[0155] in, Indicates the first Asymptotic frequency domain-guided adaptive sparse reconstruction blocks Indicates the number of rebuild blocks.

[0156] S3.2. Each asymptotically frequency-domain guided adaptive sparse reconstruction block includes multiple asymptotically frequency-domain guided adaptive sparse transform layers and convolutional layers. The block-level output can be represented as:

[0157] ;

[0158] in, Indicates the first Within each reconstruction block, a cascaded mapping of multiple asymptotic frequency domain-guided adaptive sparse transform layers is performed. This indicates the convolution fusion operation.

[0159] S3.3. Each asymptotically frequency-domain guided adaptive sparse transform layer includes a first normalization unit, a linear projection unit, a window attention unit, a first residual connection unit, a second normalization unit, a gated convolutional feedforward network unit, and a second residual connection unit; let the input features be... The output of the transformation layer can then be expressed as:

[0160] ;

[0161] ;

[0162] in, This represents window attention mapping. This represents the mapping of a gated convolutional feedforward network. and This indicates a normalization operation.

[0163] S3.4. The window attention unit includes a frequency domain prior extraction branch, which performs channel compression and high-pass filtering on the two-dimensional features recovered from the input serialized features to obtain a frequency-guided score. It is represented as:

[0164] ;

[0165] in, This indicates a channel compression operation. This indicates a high-pass filtering operation. Indicates the flattening operation. This represents a normalization operation; furthermore, the high-pass filtering operation can be implemented using the Sobel operator, the Laplacian operator, or a learnable high-pass convolution kernel.

[0166] The window attention satisfies the following relation:

[0167] ;

[0168] in, Indicates attention weights, Representing feature dimension, Indicates the position offset term. This represents the frequency-domain guided bias term constructed based on the frequency-guided fraction. This represents the asymptotic prior bias term. Indicates frequency domain guiding weights, This represents the exponential normalization function;

[0169] S3.5. The window attention unit further includes an adaptive sparse prediction branch, used to dynamically predict the number of attention-retaining elements based on the sequence features within the window, the frequency-guided score, and the layer identifier information. Let the window features be... Then the predicted feature can be expressed as:

[0170] ;

[0171] in, Indicates global average pooling. Indicates global max pooling. This represents window-level frequency-guided statistics. Presentation layer identifier embedding;

[0172] Number of target attention retentions It can be represented as:

[0173] ;

[0174] in, This represents the probability distribution of each candidate sparsity level. This represents a multilayer perceptron. Represents a predefined set of sparse levels. This refers to the operation of determining the number of target attention to retain from the preset sparse level set according to preset selection rules;

[0175] S3.6. The window attention unit adopts a progressively sparse support set propagation mechanism, reusing the sparse support set calculated by the previous layer window attention within the same reconstruction block, and fusing the previous layer attention distribution as a priori term with the current layer attention logarithm term, which can be expressed as:

[0176] ;

[0177] in, Indicates the first The attention logarithm after layer fusion Indicates the first The current attention of the layer is on several items. This represents the attention distribution corresponding to the sparse support set of the previous layer. Indicates the prior fusion weights. Indicates a stable term. This represents the logarithmic transformation function, used to compress the numerical range of the prior distribution and improve fusion stability;

[0178] The updated sparse support set of the current layer can be represented as:

[0179] ;

[0180] in, Indicates the first The updated sparse support set. Indicates the first The number of attention points retained for each layer. This indicates selecting the top from the attention distribution. The filtering operation consists of the element corresponding to the largest response or its index.

[0181] Furthermore, to mitigate the cumulative loss of information between layers, dense attention recall can be performed according to preset layer intervals to reset or expand the sparse support set.

[0182] S3.7. The gated convolutional feedforward network unit is used to enhance the representation of local details and the ability to model nonlinearities. Its output can be expressed as:

[0183] ;

[0184] in, Represents the second linear mapping. This indicates local modeling in depthwise convolution. This represents the gated activation function. This represents element-wise multiplication. and This represents the two feature branches obtained after branching the output of the first linear mapping.

[0185] S4. The deep reconstruction features are fused with the shallow features, and the resolution is improved by the upsampling reconstruction module to output a high-resolution reconstructed image.

[0186] Specific implementation method:

[0187] S4.1. Reconstruct a two-dimensional feature map from the serialized features output by the deep feature reconstruction module using the spatial restoration unit. And perform convolutional fusion;

[0188] S4.2. Perform global residual fusion on the fused deep features and shallow features to obtain the fused features. It can be represented as:

[0189] ;

[0190] in, Indicates shallow features. Indicates the convolution fusion operation;

[0191] S4.3. Input the fused features into the upsampling reconstruction module, and output a high-resolution image through pixel recombination. It can be represented as:

[0192] ;

[0193] in, This indicates a pixel recombining upsampling operation.

[0194] In one specific embodiment, the magnification factor of the image super-resolution reconstruction network can be 2x, 3x, or 4x, and the window size, number of reconstruction blocks, sparsity level set, and channel dimension can be adjusted according to the actual application scenario. This invention is not limited to the specific parameter settings described above; any equivalent substitutions and modifications made without departing from the technical concept of this invention should fall within the protection scope of this invention.

[0195] To better demonstrate the beneficial effects of the present invention, the following experiments were conducted:

[0196] The equipment used in this experiment was a server equipped with an NVIDIA RTX 4090 GPU. To comprehensively evaluate the reconstruction quality, we used two metrics: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), both calculated on the luminance channel (Y channel) of the YCbCr color space. A ×4 magnification experiment was conducted on five publicly available datasets (EDSR, CARN, IMDN, SMSR, and NGswin). The experimental results are shown in Table 1. The test results for the comparative methods all used the experimental results of the publicly available standard datasets for each method. Our method achieved the best results in both PSNR and SSIM metrics.

[0197] Table 1. Comparison of test metrics between our method and some existing methods on five public datasets.

[0198]

[0199] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. An image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention, characterized in that, include: Step 1: Obtain the low-resolution image to be reconstructed and preprocess the low-resolution image; Step 2: Perform convolutional feature extraction on the preprocessed low-resolution image to obtain shallow features; Step 3: By connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series, progressive feature interaction and transformation are performed on the shallow features to generate deep reconstruction features; Each of the progressive frequency domain guided adaptive sparse reconstruction blocks consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block; Step 4: Use the global residual fusion method to fuse the deep reconstruction features and the shallow features, and then upsample and reconstruct the fused features to output a high-resolution reconstructed image.

2. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention as described in claim 1, characterized in that, In step one, the method for preprocessing the low-resolution image is as follows: The low-resolution image to be reconstructed is filled with reflection to make the image size fit the preset window size; mean correction and numerical range normalization are performed on the filled image to obtain the preprocessed low-resolution image.

3. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 1 or 2, characterized in that, In step three, each of the progressive frequency domain guided adaptive sparse transform layers includes a first normalization unit, a linear projection unit, a window attention unit, a first residual connection unit, a second normalization unit, a gated convolutional feedforward network unit, and a second residual connection unit connected in sequence. The first normalization unit is used to perform layer normalization processing on the input features to obtain stable distribution features; The linear projection unit is used to perform a linear mapping on the normalized distribution features of the input, and to query the features Q, key features K, and value features V. The window attention unit is used to perform window-based adaptive sparse attention calculation on the query features, key features, and value features to obtain attention aggregation features; The first residual connection unit is used to perform a residual connection between the attention aggregation feature and the input of the linear projection unit and then output the result to the second normalization unit. The second normalization unit is used to perform layer normalization processing on the output of the first residual connection unit; The gated convolutional feedforward network unit is used to perform nonlinear transformation and local detail modeling on the normalized distribution features to obtain enhanced features; The second residual connection unit is used to add the enhanced features to the output of the first residual connection unit to obtain the output of the current asymptotic frequency domain guided adaptive sparse transform layer.

4. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 3, characterized in that, The enhanced feature is: ； in, This represents the output characteristics of a gated convolutional feedforward network unit. Represents the second linear mapping. This indicates local modeling in depthwise convolution. This represents the gated activation function. This represents element-wise multiplication. and This represents the two feature branches obtained after branching the output of the first linear mapping.

5. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 3, characterized in that, The window attention unit includes a frequency domain prior extraction branch, an adaptive sparse prediction branch, a progressive prior fusion module, a Top-k sparse selection module, and an attention aggregation module; The frequency domain prior extraction branch is used for, Channel compression is performed on the input features to obtain a single-channel feature map; High-pass filtering is performed on the single-channel feature map to obtain high-frequency response information; The high-frequency response information is used to generate a frequency steering fraction, and a frequency domain steering bias term is constructed based on the frequency steering fraction. The progressive prior fusion module is used for, The attention distribution corresponding to the sparse support set of the current layer is fused with the attention distribution corresponding to the sparse support set of the previous layer to obtain the fused attention distribution of the sparse support set of the current layer. The adaptive sparse prediction branch is used for, Based on the sparse support set attention distribution of the current layer, construct the attention log terms after fusion for the current window; Using the attention logarithm of the current window fusion, query feature Q, key feature K, attention distribution of the sparse support set of the current layer, frequency domain guided bias term and asymptotic prior bias term, the window attention formula is used to calculate the target attention weight of the current asymptotic frequency domain guided adaptive sparse transform layer, and the number of weights retained corresponding to the target attention weights is taken as the number of attention retained in the current window. The attention window position bias term is a bias term that determines the relative spatial relationship between the pixel feature positions within the encoding window; The Top-k sparse screening module is used for, The number of weights retained is subjected to Top-k sparse filtering (the top k highest probability samples) to obtain the sparse support set of the current asymptotic frequency domain guided adaptive sparse transform layer. The attention aggregation module is used for, Attention aggregation features are obtained by using the sparse support set of the current asymptotic frequency domain-guided adaptive sparse transform layer and the value feature V to perform attention aggregation calculation.

6. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 5, characterized in that, The formula for generating the frequency-guided fraction of high-frequency response information is: ； in, Frequency-guided fractions This represents the two-dimensional features obtained by recovering the features from the input sequence. This indicates a channel compression operation. This indicates a high-pass filtering operation. Indicates the flattening operation. This indicates a normalization operation.

7. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention as described in claim 5, characterized in that, The current attention logarithm after window merging is: ； in, Indicates the first The attention logarithm after layer fusion Indicates the first The current attention of the layer is on several items. This represents the attention distribution of the sparse support set in the previous layer. Indicates the prior fusion weights. Indicates a stable term. This represents the logarithmic transformation function.

8. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 5, characterized in that, The formula for calculating window attention is: ； in, Indicates attention weights, Representing feature dimension, Indicates the position offset term. This represents the frequency-domain guided bias term constructed based on the frequency-guided fraction. This represents the asymptotic prior bias term. Indicates frequency domain guiding weights, Let Q represent the exponential normalization function, and let Q represent the query feature. Represents the transpose of the key feature, wherein, the , and It constitutes the logarithmic terms of attention. The three bias components.

9. The image super-resolution reconstruction method based on progressive frequency domain guided adaptive sparse attention according to claim 6, characterized in that, The number of attention-retaining elements in the current window is: ； ； in, This represents the probability distribution of each candidate sparsity level. This represents a multilayer perceptron. Indicates the number of targets whose attention is retained. Represents a predefined set of sparse levels. This refers to the operation of determining the number of target attention to retain from the preset sparse level set according to preset selection rules; Represents the sequence features within the window. Indicates global average pooling. Indicates global max pooling. This represents window-level frequency-guided statistics. Presentation layer identifier embedding.

10. An image super-resolution reconstruction system based on progressive frequency domain guided adaptive sparse attention, characterized in that, The system includes: Preprocessing module and image super-resolution reconstruction network; The preprocessing module is used to acquire the low-resolution image to be reconstructed and perform preprocessing. The image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature reconstruction module, a feature fusion module, and an upsampling reconstruction module; The shallow feature extraction module is used to perform convolutional feature extraction on the input low-resolution image to obtain shallow features. The deep feature reconstruction module generates deep reconstructed features by connecting multiple progressive frequency domain-guided adaptive sparse reconstruction blocks in series and progressively interacting and transforming shallow features. Each progressive frequency domain guided adaptive sparse reconstruction block consists of multiple cascaded progressive frequency domain guided adaptive sparse transform layers and a convolutional layer. The multiple transform layers sequentially perform adaptive sparse attention calculation based on frequency domain prior guidance on the input features. In each transform layer, the attention distribution corresponding to the sparse support set passed from the previous layer is used as prior information and fused with the calculation result of the current layer. The fused result is then transformed to obtain the output features of the transform layer. The output features of all transform layers are passed to the convolutional layer for fusion, and then added to the input of the reconstruction block through residual connections to obtain the output of the current reconstruction block. The feature fusion module uses global residual fusion to fuse the deep reconstruction features and shallow features, and then uses the upsampling reconstruction module to reassemble pixels to obtain a high-resolution reconstructed image.

Images