An Image Super-Resolution Reconstruction Method and System Based on Adaptive Aggregation Sensing

By employing multi-scale large-kernel convolutional attention, shared attention weight matrix reuse, and local window cyclic shifting interaction, combined with partial convolutional gating, the problems of limited receptive field and high computational complexity of self-attention in convolutional networks are solved, achieving efficient image super-resolution reconstruction.

CN123118364APending Publication Date: 2026-11-17HANGZHOU TUMEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611619545.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing convolutional networks have limited receptive fields for their kernels, requiring stacked, deeper networks to establish spatial dependencies over long distances. Standard self-attention has high computational complexity, and window attention limits global structure modeling, resulting in poor image super-resolution reconstruction performance.

Method used

By employing multi-scale large-kernel convolutional attention, shared attention weight matrix reuse, local window cyclic shift interaction, and partial convolutional gating, we enhance the ability to model contextual information of distant regions through shallow feature extraction, deep feature fusion, and upsampling reconstruction, thereby reducing computational redundancy and improving computational efficiency.

Benefits of technology

While avoiding high computational overhead, it enhances the ability to model contextual information in distant regions, improves the quality and efficiency of image super-resolution reconstruction, preserves basic structural and color information, and enhances the reconstruction capabilities of edge and texture details.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118364A_ABST
    Figure CN123118364A_ABST
Patent Text Reader

Abstract

This invention provides an image super-resolution reconstruction method and system based on adaptive aggregation perception, belonging to the field of image data processing technology. The method first extracts shallow features through convolution, and then extracts deep features through cascaded residual groups. In the residual group, a hybrid efficient attention module modulates the features of each channel group with multi-scale large-kernel depth convolution and dilated depth convolution, expanding the effective receptive field; the attention weight matrix of the first module is reused between consecutive modules, reducing redundant computation; a low-resolution self-attention module alternately uses regular windows and shifted windows to establish cross-window information interaction; a partially convolutional gated feedforward module applies spatial convolution only to some channels, directly retaining bypass channels to reduce computational redundancy. Finally, a high-resolution image is output by superimposing long-range residuals and global residuals, preserving the basic image structure while enhancing the quality of texture detail reconstruction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to an image super-resolution reconstruction method and system based on adaptive aggregation sensing. Background Technology

[0002] Image super-resolution reconstruction aims to recover the corresponding high-resolution image from a low-resolution image. Low-resolution images typically contain low-frequency structural and color information of the target scene, but high-frequency textures, fine edges, and local details are easily lost during acquisition, compression, or transmission. To recover these high-frequency details, deep neural networks are widely used to learn the mapping relationship between low-resolution and high-resolution images.

[0003] Existing convolutional networks possess good local modeling capabilities, but their convolutional kernels have limited receptive fields, typically requiring deep networks to establish spatial dependencies over long distances. Transformer-based image super-resolution methods can capture long-range dependencies through self-attention mechanisms, but the computational complexity of standard self-attention increases significantly with image resolution. While window attention reduces complexity, hard window partitioning limits information interaction between windows, affecting global structure modeling.

[0004] In view of this, there is an urgent need for an image super-resolution reconstruction method and system based on adaptive aggregation sensing, in order to at least address the above-mentioned shortcomings. Summary of the Invention

[0005] One of the objectives of this invention is to provide an image super-resolution reconstruction method and system based on adaptive aggregation sensing, so as to solve the problems mentioned in the background art.

[0006] The image super-resolution reconstruction method based on adaptive aggregation sensing provided in this embodiment of the invention includes: Step S1: Obtain the low-resolution image to be reconstructed, and perform convolution mapping on the low-resolution image through the shallow feature extraction module to obtain shallow features; Step S2: Input the shallow features into the deep feature extraction backbone composed of multiple concatenated residual groups; each residual group processes the input features of its own residual group in the order of hybrid efficient attention module, low-resolution self-attention module and part of convolutional gated feedforward module, and outputs the output features of its own residual group through residual connection; Step S3: In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features. Step S4: For a shared sequence consisting of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates the shared attention weight matrix based on query features and key features. Subsequent hybrid efficient attention modules perform feature aggregation based on their respective input generated value features and reuse the shared attention weight matrix. When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated. Step S5: Perform in-window self-attention calculation alternately between the regular local window and the shifted local window using a low-resolution self-attention module, and apply an attention mask to the shifted local window to establish information interaction between adjacent windows; Step S6: Divide the expanded feature channels into unprocessed channels and bypass channels through a partial convolutional gated feedforward module. Perform deep convolutional gated processing on the unprocessed channels, fuse the gated processing results with the bypass channels, and add the residuals with the module input to obtain deep features. Step S7: Refine the deep features and perform long-range residual fusion with the shallow features. Improve the spatial resolution through the upsampling reconstruction module and perform global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.

[0007] Preferably, the shallow feature extraction in step S1 satisfies: ; in, Indicates a low-resolution image. This indicates a shallow feature extraction module that uses convolutional mapping. This indicates shallow features.

[0008] Preferably, the deep feature extraction backbone includes The cascaded residual sets; for the ... One set of residuals, Its processing satisfies: ; ; ; in, and They represent the first Input and output features of each residual set; and For the first Intermediate features of each residual set; Representation layer normalization; , and They represent the first The residual sets contain a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module.

[0009] Preferably, step S3 includes: mixing the input features of the efficient attention module. Divided into Channel group and for the first Execute in one channel group: ; ; in, Indicates the first Spatial attention response of each channel group; This represents a 1×1 pointwise convolution; Indicates the kernel size as void ratio And maintains the same spatial dimensions with open-hole depth convolution; Indicates the kernel size as And depthwise convolution that maintains the same spatial dimensions; Indicates the first Attention output of each channel group; This represents element-wise multiplication; Indicates the number of channel groups; where at least two channel groups correspond to or different.

[0010] Preferably, the attention outputs of each channel group are concatenated and mapped to obtain multi-scale contextual features: ; in, Represents multi-scale contextual features; This represents a 1×1 convolution output mapping; This indicates splicing along the channel dimension.

[0011] Preferably, the shared attention weight matrix in step S4 is generated and reused in the following manner: The first in the shared sequence The input features of the hybrid efficient attention module are flattened into a window sequence. Query features are generated by the first hybrid efficient attention module. Bond features Each hybrid high-efficiency attention module generates value features. ;Will , and Rearranged as ,in And calculate: ; ; in, Indicates the number of window sequences in the batch; This indicates the number of tags in each window sequence; Indicates the channel dimension; Indicates the number of attention heads; Represents the feature dimensions of each attention head; Represents the shared attention weight matrix; Indicates relative position offset; Indicates the window attention mask; Indicates the first Shared attention aggregation results of several hybrid high-efficiency attention modules; This indicates the number of hybrid, efficient attention modules contained in the shared sequence; Shared sequence by The system consists of three consecutively configured hybrid efficient attention modules with identical spatial dimensions, channel dimensions, and window partitioning parameters for the input features; the first hybrid efficient attention module calculates a shared attention weight matrix, and the second to third modules... Each hybrid efficient attention module reuses a shared attention weight matrix; the shared attention weight matrix is ​​calculated on a local window sequence or a downsampled low-resolution sequence, and is recalculated when entering the next shared sequence or when any of the spatial size, channel dimension, or window partitioning parameters changes.

[0012] Preferably, step S5 includes: dividing the input features of the low-resolution self-attention module into... Local windows, each containing One marker; the shift of a regular local window is 0, while the shift of a shifted local window is 0 in both the height and width directions. Generate query features for each local window. Key features Sum value characteristics And calculate: ; in, Represents the local attention weight matrix; Represents the feature dimensions of each attention head; Indicates the relative position offset of a local window; This represents the attention mask for the shift window, which is set to 0 at positions where connections are allowed and to negative infinity at positions where connections are blocked. This represents the local self-attention output; it performs window restoration on the local self-attention output and performs reverse shifting when using a shifted local window.

[0013] Preferably, the partial convolutional gated feedforward module in step S6 satisfies: ; ; ; in, This represents the input features of a partial convolutional gated feedforward module; express Intermediate features obtained after layer normalization and linear mapping; This represents a 1×1 convolutional mapping used for channel expansion. Indicates division along the channel dimension; Indicates the channel to be processed; Indicates a bypass route; Indicates a feature branch; Indicates a gated branch; Represents gated branch convolution mapping; Indicates the gating processing result; It is a feature branch Nonlinear activation; This represents a 3×3 depthwise convolution; It is the Sigmoid activation function; This represents a 1×1 convolutional mapping used to recover the channel dimension. This represents the output characteristics of a partially convolutionally gated feedforward module; the channel expansion factor is 2. and The ratio of the number of channels is 1:1 or 1:3.

[0014] Preferably, step S7 satisfies: ; in, Indicates the characteristics of long-range residual fusion; Indicates will Mapped to Refined convolutions with the same spatial dimensions and channel dimensions; These are shallow features; For deep features; Represents a high-resolution image; This represents an upsampling reconstruction module consisting of convolutional mapping and subpixel rearrangement; This indicates bicubic interpolation upsampling; The input image is low resolution; the upsampling reconstruction module has a magnification factor of 2, 3 or 4, and when the magnification factor is 4, two-level 2x subpixel rearrangement is used.

[0015] The image super-resolution reconstruction system based on adaptive aggregation sensing provided in this embodiment of the invention includes: The shallow feature extraction module is used to acquire the low-resolution image to be reconstructed, and to perform convolution mapping on the low-resolution image to obtain shallow features. The deep feature extraction module is used to input shallow features into the deep feature extraction backbone, which is composed of multiple concatenated residual groups. Each residual group processes its input features in the order of a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module, and outputs its output features through residual connections. In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features. For a shared sequence consisting of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates the shared attention weight matrix based on query features and key features. Subsequent hybrid efficient attention modules perform feature aggregation based on their respective input generated value features and reuse the shared attention weight matrix. When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated. Intra-window self-attention computation is performed alternately between regular local windows and shifted local windows using a low-resolution self-attention module, and an attention mask is applied to the shifted local window to establish information interaction between adjacent windows. The extended feature channels are divided into unprocessed channels and bypass channels by a partially convolutional gated feedforward module. Deep convolutional gate processing is performed on the unprocessed channels, the gate processing results are fused with the bypass channels, and the residuals are added to the module input to obtain deep features. The residual fusion module is used to refine deep features and perform long-range residual fusion with shallow features. It improves spatial resolution through the upsampling reconstruction module and performs global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.

[0016] The beneficial effects of this invention are as follows: This invention aggregates spatial contexts of different ranges through multi-scale large-kernel convolutional attention, expanding the effective receptive field with convolutional feature processing. This enhances the modeling ability of contextual information in distant regions while avoiding the high computational overhead of standard global self-attention in original high-resolution whole images. The shared attention weight matrix is ​​calculated by the first hybrid efficient attention module in the shared sequence and reused by subsequent hybrid efficient attention modules, which reduces the repeated calculation of query features, key features, and attention weight matrices, improving the computational efficiency of deep feature extraction. By performing cyclic shift operations on the input feature map, alternating between regular local windows and shifted local windows, and using attention masks to block connections at non-homologous locations, an information interaction path between windows is established. At the same time, selective enhancement of local spatial information in some channels is performed through partial convolutional gating feedforward modules, reducing the computational redundancy caused by spatial convolution of all channels while preserving the features of bypass channels. Multi-level feature transfer paths are formed through intra-group residuals, long-range residuals, and global residuals, enabling deep networks to learn high-frequency residual information between low-resolution and high-resolution images. This helps to maintain basic structure and color information and enhance the reconstruction ability of edge and texture details.

[0017] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a comparison diagram of the effective receptive fields of different architectures in embodiments of the present invention; Figure 2 This is a diagram of the image super-resolution reconstruction network framework in an embodiment of the present invention; Figure 3 This is a structural diagram of the hybrid high-efficiency attention module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the shared attention weight matrix reuse mechanism in an embodiment of the present invention; Figure 5 This is a structural diagram of a portion of the convolutional gated feedforward module in an embodiment of the present invention. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] See Figure 1 and Figure 2 This invention addresses the problems of limited effective receptive field of convolutional networks, high computational complexity of standard self-attention, and limited range of local window attention interaction. It constructs an image super-resolution reconstruction process executed by a processor, which reconstructs low-resolution images into high-resolution images through multi-scale large kernel convolutional attention, reuse of shared attention weight matrix (i.e. shared attention score), local window cyclic shift interaction, and partial convolutional gating.

[0022] This invention provides an image super-resolution reconstruction method based on adaptive aggregation sensing, such as... Figure 2 As shown, it includes: Step S1: Obtain the low-resolution image to be reconstructed, and perform convolution mapping on the low-resolution image through the shallow feature extraction module to obtain shallow features.

[0023] The processor receives the low-resolution image to be reconstructed from the image acquisition device via an image data interface. Alternatively, the low-resolution image can be read from local storage, external storage media, or an image database; subsequently, the low-resolution image is mapped from the pixel space to the feature space using a shallow feature extraction module. ; in, This refers to the low-resolution image. This indicates a shallow feature extraction module that uses convolutional mapping. This represents the shallow features. In one specific embodiment, the shallow feature extraction module is: A single-layer convolution maps a three-channel RGB image to... Feature map of the channel.

[0024] Step S2: Input the shallow features into the deep feature extraction backbone composed of multiple cascaded residual groups; each residual group processes the input features of its own residual group in the order of hybrid efficient attention module, low-resolution self-attention module and part of convolutional gated feedforward module, and outputs the output features of its own residual group through residual connection.

[0025] shallow features Input the deep feature extraction backbone. The deep feature extraction backbone includes... The cascaded residual sets; Each residual group sequentially processes the input features through a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module. Each sub-module employs a pre-normalized residual structure (i.e., the input is first normalized by the layer, then transformed by the sub-module, and finally added to the sub-module's input residual). ; ; ; in, and They represent the first Input and output features of each residual set; and For the first Intermediate features of each residual set; Representation layer normalization; , and They represent the first The residual sets contain a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module; in one specific implementation, the residual sets include a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module. .

[0026] Step S3: In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features.

[0027] See Figure 3 The hybrid efficient attention module includes a multi-scale large kernel convolutional attention path (left side) and a shared window attention path (right side). The two outputs are added together to obtain the module output.

[0028] Multi-scale large-kernel convolutional attention path: for input features Divide it along the channel dimension into Channel group and for the first Execute in one channel group: ; ; in, Indicates the first Spatial attention response of each channel group; This represents a 1×1 pointwise convolution; Indicates the kernel size as void ratio And maintains the same spatial dimensions with open-hole depth convolution; Indicates the kernel size as And depthwise convolution that maintains the same spatial dimensions; Indicates the first Attention output of each channel group; This represents element-wise multiplication; Indicates the number of channel groups; where at least two channel groups correspond to or The differences result in different effective receptive fields.

[0029] In one specific implementation, Four channel groups Take respectively , , and The effective receptive fields increase sequentially.

[0030] The outputs of each channel group are concatenated and mapped to obtain multi-scale contextual features: ; in, This indicates splicing along the channel dimension. express Convolution output mapping, This represents the multi-scale contextual features.

[0031] Shared window attention path: Perform query-key-value window attention on a local window and reuse the shared attention weight matrix to generate aggregated features. ,through Linear projection get .

[0032] Two routes merged: ; in, For the first The final output of the hybrid high-efficiency attention module is then added to the residuals as described in step 2.

[0033] Shared attention weight matrix Spatial attention response They have different meanings: Generated by large kernel depthwise convolution, dilated depthwise convolution and pointwise convolution, it is used to spatially modulate the features of the corresponding channel group. It is calculated from query features and key features through scaling dot product attention, and is used to guide the aggregation of value features of each module within the shared sequence; the two belong to different calculation paths and are not shared with each other.

[0034] Step S4: For a shared sequence consisting of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates the shared attention weight matrix based on query features and key features. Subsequent hybrid efficient attention modules generate features based on their respective input values ​​and reuse the shared attention weight matrix for feature aggregation. When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated.

[0035] See Figure 4 , in the In a shared sequence consisting of consecutive, highly efficient attention modules, the first... Each module generates complete query features. Key features Sum value characteristics , No. The first to the second Each module generates its own value characteristics. ( ), and reuse the first The shared attention weight matrix is ​​calculated by each module.

[0036] For the Each module flattens its input features into a sequence of local windows. ,Will , and Rearranged as ,in And calculate: ; ; in, Indicates the number of window sequences in the batch. This indicates the number of markers within each window. Indicates channel dimension, Indicates the number of attention heads. This represents the feature dimension of each attention head. This indicates the number of hybrid efficient attention modules contained in the shared sequence (in one specific implementation). ), This represents the relative position offset (obtained by querying the learnable position offset parameter table, constructed in step 5). same), Indicates the window attention mask. Represents the shared attention weight matrix. Indicates the first The result of shared attention aggregation of several hybrid high-efficiency attention modules. Take at locations where attentional connections can be established. Take at the location where blocking is needed .

[0037] When entering the next shared sequence, or when any of the spatial dimensions, channel dimensions, or window sizes of the current feature changes, the shared attention weight matrix is ​​recalculated by the first module of the new shared sequence.

[0038] Existing shared attention mechanisms unconditionally reuse the attention weight matrix calculated by the first module in all subsequent deep modules, implicitly assuming that the input features of each module within the shared sequence maintain a high degree of consistency at the global structural level. However, when processing natural images that simultaneously contain large smooth regions and local fine textures, as features are continuously refined in residual stacking, the input features of deep modules gradually become enriched with high-frequency detail information. If the low-frequency global topological relationships characterized by the shallow attention weight matrix are still used, this bias will systematically suppress high-frequency components in the value feature aggregation stage, resulting in texture blurring and loss of detail in the reconstructed image.

[0039] This invention provides an image super-resolution reconstruction method based on adaptive aggregation perception. When applied to the reconstruction of natural images containing smooth backgrounds and complex high-frequency textures, the method performs the following steps for a shared sequence of consecutive, hybrid, efficient attention modules: Using the input features of the first hybrid efficient attention module constituting this shared sequence as the starting point of this sequence, a basic texture reference basis for characterizing the smooth structure of the image is established through low-pass filtering or spatial pooling. Based on the extracted query features and key features, a shared attention weight matrix for global topological association is generated. In this embodiment, low-pass filtering and spatial pooling are two optional paths for implementing the basic texture reference substrate. Both aim to suppress high-frequency components and preserve the smooth structure of low frequencies. One of them can be selected according to the deployment environment: When using the low-pass filtering path, a two-dimensional Gaussian kernel (in one specific implementation, the kernel size is 5×5 and the standard deviation is 1) is applied independently to each channel of the input feature of the first hybrid efficient attention module. Convolutional filtering is performed in the spatial domain to suppress local edges and texture oscillations, while maintaining the same spatial size and number of channels as the input feature. When using the spatial pooling path, global average pooling is performed on the input feature to compress the spatial dimension to a single scalar. Then, bilinear interpolation is used to restore it to the same spatial size as the input feature, eliminating all local high-frequency disturbances in a uniform and smooth manner. The output of both paths serves as the basic texture reference substrate. Its number of channels is the same as the input feature of the first hybrid efficient attention module and remains fixed throughout the entire processing of this shared sequence, without changing with the feature updates of subsequent modules. The input features of each hybrid efficient attention module located in the deep layers of the shared sequence are used as the processing objects to generate corresponding value features. Local spatial gradient calculation is performed on the input features of each hybrid efficient attention module to extract high-frequency texture fluctuation response used to characterize the intensity of detail depiction of the current input feature. In this embodiment, the local spatial gradient calculation uses a 3×3 Sobel operator to perform first-order differences in the horizontal and vertical directions on each channel of the input feature of the current hybrid efficient attention module. For each channel, the absolute values ​​of the horizontal gradient component and the vertical gradient component are taken element by element and then added to obtain the local gradient magnitude map of that channel. Then, the channel mean of the gradient magnitude maps of all channels is taken to obtain a single-channel response map with the same spatial size as the input feature, i.e., the high-frequency texture fluctuation response. The higher the value of the spatial position in the response map, the stronger the high-frequency detail intensity carried by the current feature at that position, i.e., the more complex the local texture and the clearer the edge. The reason for choosing to perform gradient calculation on the input features of each subsequent module rather than the output features is that the input features more directly reflect the amount of high-frequency information received by the current layer, eliminating the interference of the current layer's attention aggregation itself on the feature distribution. The system receives the base texture reference substrate transmitted by the first hybrid high-efficiency attention module and the high-frequency texture fluctuation response of the corresponding subsequent hybrid high-efficiency attention module as comparative inputs. By calculating the distance between the two, the system measures the degree of deviation of the current deep layer relative to the shallow global structure and outputs the quantized texture divergence gradient value. In this embodiment, the base texture reference substrate is a multi-channel feature map, and the high-frequency texture fluctuation response is a single-channel gradient mean map. Before performing distance measurement, the two need to be aligned in the channel dimension: the mean of the base texture reference substrate along the channel dimension is taken and compressed into a single-channel feature map with the same number of channels as the high-frequency texture fluctuation response, so that both are single-channel matrices with the same spatial size; then, the element-wise difference is performed on the two, the L2 norm of all elements of the difference map is taken, and normalized by dividing by the spatial size (the product of height and width), and the resulting scalar is the texture divergence gradient value; the channel mean operation compresses the multi-channel smoothing information of the base texture reference substrate into a global average low-frequency intensity distribution, which is semantically equivalent to the single-channel high-frequency texture fluctuation response that also reflects the global average high-frequency intensity, thereby ensuring that the physical meaning of the distance measurement is valid: the larger the texture divergence gradient value, the more significant the accumulation of high-frequency components carried by the current deep features relative to the low-frequency smooth substrate at the beginning of the sequence, and the greater the deviation in frequency distribution between the two; The system receives the texture divergence gradient value and the spatial size, channel dimension, and window partitioning parameters of the corresponding hybrid high-efficiency attention module as joint verification inputs. When any of the spatial size, channel dimension, or window partitioning parameters changes, or when the texture divergence gradient value exceeds the preset high-frequency detail protection threshold, the system triggers and outputs an attention recalculation instruction. If none of these conditions are met, the system outputs an attention reuse instruction. In this embodiment, changes in spatial size, channel dimension, or window partitioning parameters constitute hard triggering conditions. Once any of the above structural parameters change, the shared attention weight matrix generated by the first module becomes incompatible with the value features of the current module in terms of sequence dimension or number of tags. Even if there is no high-frequency deviation at the content level, forced reuse will lead to calculation errors due to dimensional misalignment. Therefore, regardless of the size of the texture divergence gradient value, recalculation must be forcibly triggered. The texture divergence gradient value exceeding the high-frequency detail protection threshold constitutes soft triggering conditions. That is, under the premise that the structural parameters have not changed, the degree of high-frequency evolution of the feature content determines whether to recalculate. The logic is that consistent structural parameters only guarantee the mathematical validity of shared computation, but do not guarantee the consistency between the attention weight matrix and the feature semantics of the current module. When the high-frequency accumulation exceeds the threshold, using shallow attention bias will systematically suppress high-frequency details. The two types of conditions are combined logically or relationally to form the criteria for attention recalculation instructions. In this embodiment, the high-frequency detail preservation threshold is a pre-set scalar threshold, which is 0.15 in one specific implementation. The value is determined as follows: image samples containing a mixture of large-area smooth background and local fine texture are randomly selected from the training set DIV2K (DIVerse 2K resolution images dataset). The distribution of texture divergence gradient values ​​is statistically analyzed on each shared sequence of the pre-trained network. The lower bound of the texture divergence gradient value distribution interval that causes a significant decrease in the reconstructed PSNR (a decrease of more than 0.1dB) is taken as the threshold. Under the above statistical settings, this value is approximately 0.15. This threshold matches the numerical scale of the aforementioned normalization calculation method (L2 norm divided by spatial size) and can be fine-tuned at different resolution magnifications or different data domains. The fine-tuning range is usually between 0.10 and 0.20. The received instructions guide the attention aggregation of the corresponding hybrid high-efficiency attention module: if the instruction is attention reuse, the shared attention weight matrix generated by the first hybrid high-efficiency attention module is directly extracted and applied to the value feature for feature modulation; if the instruction is attention recalculation, the cross-layer shared link is broken, a new shared attention weight matrix is ​​recalculated based on the input feature of the corresponding hybrid high-efficiency attention module, and applied to the value feature, and the modulated and fused result is output as the feature aggregation output of the module.

[0040] In this embodiment, breaking the cross-layer shared link means that the current hybrid efficient attention module stops reading cached query features and key features from the first module, and instead uses its own input features as the source to independently perform linear projection operations to regenerate query features and key features, and calculate a new shared attention weight matrix; sharing means that the matrix is ​​structurally within the same computational framework as the shared attention weight matrix generated by the first module (i.e., it also serves the current module's own query-key-value attention aggregation), rather than that the recalculated matrix will be passed to subsequent hybrid efficient attention modules in this shared sequence; the recalculated matrix only serves the current module that triggered the recalculation. When subsequent modules perform texture divergence evaluation, they still use the basic texture reference base established by the first hybrid efficient attention module as an unchanging reference benchmark. They independently calculate their own texture divergence gradient values ​​and perform joint verification again, thereby ensuring that the evaluation logic of the entire shared sequence can still be completely closed after recalculation without causing drift of the reference benchmark. In the specific implementation, attention reuse instructions and attention recalculation instructions are transmitted to the attention aggregation unit in the form of single-bit control signals, corresponding to two mutually exclusive execution paths: directly reading the cache matrix and independently recalculating. The two paths are dynamically switched based on the texture content during the inference phase, and the switching operation itself does not introduce additional learnable parameters.

[0041] This scheme anchors a low-frequency structural benchmark at the beginning of the sequence using low-pass filtering or spatial pooling. It then uses a 3×3 Sobel gradient operator to quantify the high-frequency accumulation intensity of deep features in real time. After averaging the channels of both, it performs dimensional alignment and calculates the L2 distance, transforming the cross-layer high-low frequency deviation into comparable texture divergence gradient values. Combining hard verification of structural parameter consistency with soft threshold judgment of texture divergence gradient values, a dual joint triggering mechanism is established. This transforms the reuse and recalculation decisions of the attention weight matrix from static rules to dynamic perception based on feature content. After recalculation, the new matrix only serves the current module and does not change the texture evaluation reference benchmark for subsequent modules, thus ensuring the complete closure of the entire sequence evaluation logic. This mechanism, while preserving computational efficiency gains, specifically protects high-frequency texture details from being diluted by shallow attention bias, effectively solving the problems of over-smoothing and detail loss caused by the mixed use of high- and low-frequency features in existing shared attention mechanisms.

[0042] Step S5: Perform in-window self-attention calculation alternately between the regular local window and the shifted local window using a low-resolution self-attention module, and apply an attention mask to the shifted local window to establish information interaction between adjacent windows.

[0043] Between low-resolution self-attention modules with different residual groups, regular local windows and shifted local windows are used alternately. The shifted local window is achieved by performing a cyclic shift operation on the input feature map: shifting the feature map cyclically along the height and width directions. Each pixel position is then divided in the same way as a regular local window. The window is shifted so that pixels that were originally located near the boundaries of adjacent windows fall into the same window after the shift and participate in the attention calculation; after the attention calculation is completed, the output features are subjected to reverse cyclic shift to restore the original spatial alignment.

[0044] The input features are divided into Local windows, each containing One marker; the cyclic shift of the regular local window is... The cyclic shift of the local window along both the height and width directions is... Generate for each local window , and And calculate: ; ; in, This represents the feature dimension of each attention head. The relative position offset of a local window is obtained by: for each pair of positions within the window... Calculate relative coordinate difference ( After mapping the two-dimensional relative coordinates to a one-dimensional index, the learnable position offset parameter table is queried to obtain the corresponding scalar offset value; This represents the attention mask for the shifted window. This represents the local attention weight matrix. This represents the output of local self-attention. Take at the location where connection is allowed Take between non-homologous position pairs introduced by the cyclic shift ; in a regular local window Take all In one specific implementation, , .

[0045] Step S6: Divide the expanded feature channels into unprocessed channels and bypass channels through a partial convolutional gated feedforward module. Perform deep convolutional gated processing on the unprocessed channels, fuse the gated processing results with the bypass channels, and add the residuals with the module input to obtain deep features.

[0046] See Figure 5A partial convolutional gated feedforward module is used to perform spatial gating transformations on a portion of the channels while retaining bypass channels. The processing of the partial convolutional gated feedforward module satisfies: ; ; ; in, This represents the input features of a partial convolutional gated feedforward module. Indicates the use of channel expansion Convolution mapping, Indicates division along the channel dimension. Indicates the channel to be processed. Indicates a bypass route. Indicates will Linear mappings for feature branches and gated branches projection, Indicates feature branches, Indicates a gated branch. Represents the Gaussian error linear unit activation function ( , (the standard normal cumulative function), used to introduce a nonlinear transformation into the characteristic branch. This represents the Sigmoid activation function ( Compress the output value to (The interval is used as the multiplicative gating weight). express Depthwise convolution, Indicates the gating process result. Indicates the channel dimension used for recovery Convolution mapping, This represents the output characteristics of a partially convolutionally gated feedforward module. The channel expansion factor is... , and The ratio of the number of channels is or When the ratio of the number of channels is At that time, the number of channels to be processed accounts for the total number of channels after expansion. When the ratio of the number of channels is At that time, the number of channels to be processed accounts for the total number of channels after expansion. .

[0047] Step S7: Refine the deep features and perform long-range residual fusion with the shallow features. Improve the spatial resolution through the upsampling reconstruction module and perform global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.

[0048] After deep feature extraction is completed, feature refinement convolution pairs are used. Perform mapping and compare with shallow features Long-range residual fusion is performed; then, the reconstruction residual is generated through the upsampling reconstruction module and globally fused with the bicubic interpolation upsampling result of the low-resolution image: ; ; in, Indicates will Mapped to Feature-refined convolution with the same spatial size and channel dimension Indicates the characteristics of long-range residual fusion. This represents an upsampling reconstruction module consisting of convolutional mapping and subpixel rearrangement. The subpixel rearrangement will reconstruct the shape as follows: The feature maps are rearranged as High-resolution feature maps, This indicates bicubic interpolation upsampling. This represents a high-resolution image. The magnification factor of the upsampling reconstruction module. for , or ;when or At that time, the target spatial resolution is obtained after one subpixel rearrangement; when Two levels are used at the time Subpixel rearrangement and progressive magnification.

[0049] The image super-resolution reconstruction network constructed according to the above method will be referred to as ADAPT. This network adopts a three-stage structure: shallow feature extraction, deep residual group processing, and upsampling reconstruction; the deep feature extraction backbone includes... Each cascaded residual group includes a reconstruction ratio. times, Double times.

[0050] In the experimental implementation, the training set used the DIV2K dataset, and the test sets included the Set5 image super-resolution benchmark dataset, the Set14 image super-resolution benchmark dataset, B100 (also known as BSD100, a Berkeley segmentation dataset with 100 test images), the Urban100 city image super-resolution benchmark dataset, and the Manga109 Japanese manga image dataset. Random horizontal flipping and rotation enhancement were used during training, and the input image patch size was [missing information]. Batch size is The total number of iterations is ,Every The model is saved in the next iteration, with the optimizer being Adam (a stochastic optimization algorithm based on first-order gradients), and the parameters... , The initial learning rate is Cosine annealing scheduling is employed, and the loss function is adopted. Loss (i.e., the sum of the absolute values ​​of the differences between the predicted pixel value and the actual pixel value).

[0051] In the 4x amplification experiment, the PSNR (Peak Signal-to-Noise Ratio) / SSIM (Structural Similarity Index Measure) of ADAPT on Set5, Set14, B100, Urban100, and Manga109 were 32.57 / 0.8999, 28.91 / 0.7882, 27.71 / 0.7420, 26.85 / 0.8079, and 31.11 / 0.9178, respectively; in the 4x amplification efficiency test, the number of parameters was approximately 662. The computational complexity is approximately 47.2. The average inference time per image is approximately 0.028 seconds. PSNR is 32.38 The SSIM was 0.898. In the ablation experiments, the complete ADAPT network achieved higher PSNR / SSIM compared to the network variants in the ablation experiments, indicating that multi-scale large kernel convolutional attention, shared attention weight matrix reuse, window shifting interaction, and partial convolutional gating feedforward modules jointly improved reconstruction quality and computational efficiency.

[0052] This invention provides an image super-resolution reconstruction system based on adaptive aggregation sensing, comprising: The shallow feature extraction module is used to acquire the low-resolution image to be reconstructed, and to perform convolution mapping on the low-resolution image to obtain shallow features. The deep feature extraction module is used to input shallow features into the deep feature extraction backbone, which is composed of multiple concatenated residual groups. Each residual group processes its input features in the order of a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module, and outputs its output features through residual connections. In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features. For a shared sequence consisting of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates the shared attention weight matrix based on query features and key features. Subsequent hybrid efficient attention modules perform feature aggregation based on their respective input generated value features and reuse the shared attention weight matrix. When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated. Intra-window self-attention computation is performed alternately between regular local windows and shifted local windows using a low-resolution self-attention module, and an attention mask is applied to the shifted local window to establish information interaction between adjacent windows. The extended feature channels are divided into unprocessed channels and bypass channels by a partially convolutional gated feedforward module. Deep convolutional gate processing is performed on the unprocessed channels, the gate processing results are fused with the bypass channels, and the residuals are added to the module input to obtain deep features. The residual fusion module is used to refine deep features and perform long-range residual fusion with shallow features. It improves spatial resolution through the upsampling reconstruction module and performs global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.

[0053] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An image super-resolution reconstruction method based on adaptive convergent sensing, characterized in that, include: Step S1: Obtain the low-resolution image to be reconstructed, and perform convolution mapping on the low-resolution image through the shallow feature extraction module to obtain shallow features; Step S2: Input the shallow features into the deep feature extraction backbone composed of multiple concatenated residual groups; each residual group processes the input features of its own residual group in the order of hybrid efficient attention module, low-resolution self-attention module and part of convolutional gated feedforward module, and outputs the output features of its own residual group through residual connection; Step S3: In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features. Step S4: For the shared sequence of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates the shared attention weight matrix based on the query features and key features, and subsequent hybrid efficient attention modules perform feature aggregation based on their respective input value features and reuse the shared attention weight matrix; When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated. Step S5: Perform in-window self-attention calculation alternately between the regular local window and the shifted local window using a low-resolution self-attention module, and apply an attention mask to the shifted local window to establish information interaction between adjacent windows; Step S6: Divide the expanded feature channels into channels to be processed and bypass channels through a partial convolutional gated feedforward module. Perform deep convolutional gated processing on the channels to be processed, fuse the gated processing results with the bypass channels, and add the residuals with the module input to obtain deep features. Step S7: Refine the deep features and perform long-range residual fusion with the shallow features. Improve the spatial resolution through the upsampling reconstruction module and perform global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.

2. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, The shallow feature extraction in step S1 satisfies: ; in, Indicates a low-resolution image. This indicates a shallow feature extraction module that uses convolutional mapping. This indicates shallow features.

3. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, The deep feature extraction backbone includes The cascaded residual sets; for the ... One set of residuals, Its processing satisfies: ; ; ; in, and They represent the first Input and output features of each residual set; and For the first Intermediate features of each residual set; Representation layer normalization; , and They represent the first The residual sets consist of a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module.

4. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, Step S3 includes: mixing the input features of the efficient attention module Divided into Channel group and for the first Execute in one channel group: ; ; in, Indicates the first Spatial attention response of each channel group; This represents a 1×1 pointwise convolution; Indicates the kernel size as void ratio And maintains the same spatial dimensions with hole depth convolution; Indicates the kernel size as And depthwise convolution that maintains the same spatial dimensions; Indicates the first Attention output of each channel group; This represents element-wise multiplication; Indicates the number of channel groups; where at least two channel groups correspond to or different.

5. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 4, characterized in that, The attention outputs of each channel group are concatenated and output mapped to obtain multi-scale contextual features: ; in, Represents multi-scale contextual features; This represents a 1×1 convolution output mapping; This indicates splicing along the channel dimension.

6. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, The shared attention weight matrix in step S4 is generated and reused in the following manner: The first in the shared sequence The input features of the hybrid efficient attention module are flattened into a window sequence. Query features are generated by the first hybrid efficient attention module. Bond features Each hybrid high-efficiency attention module generates value features. ;Will , and Rearranged as ,in And calculate: ; ; in, Indicates the number of window sequences in the batch; This indicates the number of tags in each window sequence; Indicates the channel dimension; Indicates the number of attention heads; Represents the feature dimensions of each attention head; Represents the shared attention weight matrix; Indicates relative position offset; Indicates the window attention mask; Indicates the first Shared attention aggregation results of several hybrid high-efficiency attention modules; This indicates the number of hybrid, efficient attention modules contained in the shared sequence; Shared sequence by The system consists of three consecutively configured hybrid efficient attention modules with identical spatial dimensions, channel dimensions, and window partitioning parameters for the input features; the first hybrid efficient attention module calculates a shared attention weight matrix, and the second to third modules... Each hybrid efficient attention module reuses a shared attention weight matrix; the shared attention weight matrix is ​​calculated on a local window sequence or a downsampled low-resolution sequence, and is recalculated when entering the next shared sequence or when any of the spatial size, channel dimension, or window partitioning parameters changes.

7. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, Step S5 includes: dividing the input features of the low-resolution self-attention module into... Local windows, each containing One marker; the shift of a regular local window is 0, while the shift of a shifted local window is 0 in both the height and width directions. Generate query features for each local window. Key features Sum value characteristics And calculate: ; in, Represents the local attention weight matrix; Represents the feature dimensions of each attention head; Indicates the relative position offset of a local window; This represents the attention mask for the shift window, which is set to 0 at positions where connections are allowed and to negative infinity at positions where connections are blocked. This represents the local self-attention output; it performs window restoration on the local self-attention output and performs reverse shifting when using a shifted local window.

8. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, The partial convolutional gated feedforward module in step S6 satisfies: ; ; ; in, This represents the input features of a partial convolutional gated feedforward module; express Intermediate features obtained after layer normalization and linear mapping; This represents a 1×1 convolutional mapping used for channel expansion. Indicates division along the channel dimension; Indicates the channel to be processed; Indicates a bypass route; Indicates a feature branch; Indicates a gated branch; Represents gated branch convolution mapping; Indicates the gating processing result; It is a feature branch Nonlinear activation; This represents a 3×3 depthwise convolution; It is the Sigmoid activation function; This represents a 1×1 convolutional mapping used to recover the channel dimension. This represents the output characteristics of a partially convolutionally gated feedforward module; the channel expansion factor is 2. and The ratio of the number of channels is 1:1 or 1:

3.

9. The image super-resolution reconstruction method based on adaptive aggregation sensing as described in claim 1, characterized in that, Step S7 satisfies: ; in, Indicates the characteristics of long-range residual fusion; Indicates will Mapping to Refined convolutions with the same spatial dimensions and channel dimensions; These are shallow features; For deep features; Represents a high-resolution image; This represents an upsampling reconstruction module consisting of convolutional mapping and subpixel rearrangement; This indicates bicubic interpolation upsampling; The input image is low resolution; the upsampling reconstruction module has a magnification factor of 2, 3 or 4, and when the magnification factor is 4, two-level 2x subpixel rearrangement is used.

10. An image super-resolution reconstruction system based on adaptive convergent sensing, characterized in that, include: The shallow feature extraction module is used to acquire the low-resolution image to be reconstructed, and to perform convolution mapping on the low-resolution image to obtain shallow features. The deep feature extraction module is used to input shallow features into the deep feature extraction backbone, which is composed of multiple concatenated residual groups. Each residual group processes its input features in the order of a hybrid efficient attention module, a low-resolution self-attention module, and a partially convolutional gated feedforward module, and outputs its output features through residual connections. In the hybrid efficient attention module, the input features are divided into multiple channel groups along the channel dimension. For each channel group, spatial attention responses are generated by depthwise convolution, dilated depthwise convolution and pointwise convolution with different kernel sizes or dilation rates. The spatial attention responses are then used to modulate the features of the corresponding channel groups to obtain multi-scale contextual features. For a shared sequence of consecutive hybrid efficient attention modules, the first hybrid efficient attention module calculates a shared attention weight matrix based on query features and key features, and subsequent hybrid efficient attention modules perform feature aggregation based on their respective input value features and reuse the shared attention weight matrix. When the spatial size, channel dimension, or window partitioning parameters of the shared sequence change, the shared attention weight matrix is ​​recalculated. Intra-window self-attention computation is performed alternately between regular local windows and shifted local windows using a low-resolution self-attention module, and an attention mask is applied to the shifted local window to establish information interaction between adjacent windows. The extended feature channels are divided into unprocessed channels and bypass channels by a partially convolutional gated feedforward module. Deep convolutional gate processing is performed on the unprocessed channels, the gate processing results are fused with the bypass channels, and the residuals are added to the module input to obtain deep features. The residual fusion module is used to refine deep features and perform long-range residual fusion with shallow features. It improves spatial resolution through the upsampling reconstruction module and performs global residual fusion with the upsampling results of the low-resolution image to obtain a high-resolution image.