Image super-resolution reconstruction method based on multi-scale mixed attention residual network

Abstract

The invention discloses an image super-resolution reconstruction method based on a multi-scale mixed attention residual network, and belongs to the technical field of computer vision and image processing. The invention provides an end-to-end reconstruction network (MHARN) aiming at the problems of insufficient detail recovery capability, limited cross-dimension feature interaction and poor adaptability to different image contents in the prior art. The method comprises the following steps: firstly, dynamically aggregating multi-scale features through parallel convolution branches with different voidage by utilizing a progressive convolution group (PCG), and breaking through the limitation of a single receptive field; the features are input into a cascaded enhanced residual attention block (ERAB), a multi-head hybrid attention module (MHAM) in the ERAB is utilized to capture window texture, geometric structure and global channel information in parallel, and a dynamic feature enhancement module (DFEM) is combined to adaptively generate convolution kernel parameters according to local image content; and finally, a high-resolution image is obtained through global feature fusion and up-sampling reconstruction. The texture detail recovery capability can be effectively enhanced, the visual artifacts are remarkably reduced, and the image reconstruction quality is improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a single-image super-resolution (SISR) reconstruction method based on deep learning, and particularly to a technique for texture restoration using multi-scale feature aggregation and dynamic attention mechanisms.

[0002] Single-image super-resolution (SISR) aims to recover high-resolution (HR) images from low-resolution (LR) images and has wide applications in medical imaging, security monitoring, and consumer electronics. While current deep learning methods (such as CNNs and Transformers) have made significant progress, they still have limitations: First, most networks (such as RCAN and MARNet) tend to process spatial and channel features independently or statically, making it difficult to capture the complex cross-dimensional dependencies required to recover fine textures; Second, the use of static convolutional kernels or heavy nested blocks limits the network's ability to adapt to different image content, easily leading to over-smoothing in high-frequency regions (such as edges and textures); Third, traditional convolutions with fixed receptive fields cannot fully utilize the multi-scale self-similarity features in images. Therefore, to address these problems, this invention proposes an improved network architecture that solves the aforementioned technical bottlenecks through multi-scale dilated convolutions, hybrid attention mechanisms, and dynamic feature enhancement. Summary of the Invention

[0003] The purpose of this invention is to overcome the problems of insufficient detail recovery capability, limited cross-dimensional feature interaction, and poor adaptability to different image content in the prior art, and to provide an image super-resolution reconstruction method based on multi-scale hybrid attention residual network (MHARN).

[0004] To achieve the above objectives, the technical solution adopted in this invention is: an image super-resolution reconstruction method based on a multi-scale hybrid attention residual network. This method constructs an end-to-end network that includes shallow feature extraction, deep feature extraction, and image reconstruction. In the shallow feature extraction stage, a progressive convolution group (PCG) is designed, which uses parallel convolution branches with different dilation rates to dynamically aggregate multi-scale features and break through the limitation of a single receptive field. In the deep feature extraction stage, an enhanced residual attention block (ERAB) is proposed, which contains two innovative modules: (1) a multi-head hybrid attention module (MHAM), which integrates window attention, coordinate attention, and channel-space attention mechanisms in parallel, while capturing local high-frequency details, geometric structure information, and long-distance dependencies, and fusing cross-dimensional features through a gating mechanism; (2) a dynamic feature enhancement module (DFEM), which adaptively generates convolution kernel parameters according to the local image content, and adopts different processing strategies for flat areas and complex texture areas, thereby reducing artifacts and sharpening details.

[0005] The beneficial effects of this invention are as follows: 1. The PCG module effectively extracts feature information at different scales in the image through multi-scale dilated convolution, enhancing its adaptability to object structures; 2. The MHAM module solves the information bias problem caused by the traditional serial attention mechanism and achieves more comprehensive feature interaction through a hybrid parallel design; 3. The DFEM module significantly improves the model's performance when processing complex textures through content-aware dynamic convolution, avoiding excessive image smoothing and achieving high-fidelity image reconstruction. Attached Figure Description

[0006] Figure 1 is a flowchart of the overall network architecture of the method of the present invention (MHARN). Figure 2 is a schematic diagram of the progressive convolution group (PCG) in this invention; Figure 3 is a schematic diagram of the structure of the Enhanced Residual Attention Block (ERAB) in this invention; Figure 4 is a schematic diagram of the structure of the Multi-Head Hybrid Attention Module (MHAM) in this invention; Figure 5 is a schematic diagram of the structure of the Dynamic Feature Enhancement Module (DFEM) in this invention; Figure 6 is a visual comparison diagram on the benchmark dataset in this invention. Detailed Implementation

[0007] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0008] As shown in Figure 1, this invention provides an image super-resolution reconstruction method based on a multi-scale hybrid attention residual network (MHARN). This embodiment uses 4x super-resolution reconstruction (Scale factor = 4) as an example for illustration. This network is an end-to-end mapping model, and the specific processing flow is as follows: First, the low-resolution image to be reconstructed is acquired. I LR (The input channel is 3, i.e., RGB image), which is input into the shallow feature extraction module; this module consists of a 3 × 3 convolutional layer that maps the image to the feature space, and the number of output channels is set to C = 64. The network weights are initialized using the Kaiming normal distribution to accelerate model convergence; then the features are fed into the progressive convolutional group (PCG) module.

[0009] As shown in Figure 2, the specific implementation process of the PCG module is as follows: a parallel topology structure is adopted, which includes 4 parallel convolutional branches. The kernel size of each branch is 3 × 3, but different dilation rates are set. d k The receptive fields are set to 1, 2, 4, and 8 respectively to cover different scales from local details to global context. The output features of each branch are processed by a convolutional attention (CA) mechanism to enhance salient features by dynamically adjusting the channel weights. Finally, the Softmax function is used to generate probability distribution weights, and the weighted features of the four branches are adaptively fused to output multi-scale shallow features. F 0. This design effectively expands the receptive field by utilizing structural symmetry.

[0010] Next, shallow features F 0. The input is fed into the cascaded deep feature extraction module. In this embodiment, the number of cascaded Enhanced Residual Attention Blocks (ERABs) is set to 0. N = 8, as shown in Figure 3. Each ERAB employs a dense connection structure and contains two core sub-modules. The Multi-Head Hybrid Attention Module (MHAM) is as follows: Figure 4 As shown, to address the asymmetry of traditional serial attention, three parallel branches and one global recalibration branch are designed: the window attention branch divides the feature map into non-overlapping windows (e.g., 8 × 8) and computes local spatial self-attention to recover high-frequency texture details; the output is denoted as... F win The coordinate attention branch aggregates features along the horizontal and vertical directions to generate position-sensitive features to maintain geometric continuity; the output is denoted as... F coord The global recalibration branch utilizes the CBAM module in conjunction with global statistical information to perform feature recalibration in both channel and spatial dimensions, with the output denoted as... F cbam Finally, a spatial probability map is generated using 1×1 convolution and softmax. W The parallel branches are weighted and merged, and the output formula is: F out = Conv(ReLU( W 1 ⊙ F win + W 2 ⊙ F coord + W 3 ⊙ F GN + Fcbam ,in F GN This is a residual identity branching characteristic.

[0011] Then, the Dynamic Feature Enhancement Module (DFEM) is connected, such as... Figure 5 As shown, the specific implementation steps of this module are as follows: First, the parameter generation network dynamically generates convolutional kernel weight parameters based on the input features. F r Next, the Unfold operation is used to extract local neighborhood features, which are then combined with the generated dynamic weights. F r By performing element-wise multiplication and summation, the dynamic characteristics are obtained. F s Then, the gated graph is generated using the Sigmoid function. G g To balance dynamic features with original features; finally, the overall ERAB output uses local residual connections, as shown in the formula. F next = F in + λ · F DFEM The scaling factor λ is set to 0.2 to ensure the training stability of deep networks.

[0012] After deep feature extraction, the process moves to the global fusion and image reconstruction stage: First, the hierarchical global feature fusion (HGFF) module is used to aggregate the output features of all ERAB to obtain the global fused features. F GF Next, global residual learning is performed to incorporate shallow features. F -1 , F 0 and global fusion features F GF Add them together to get F DF Finally, the upsampling module employs sub-pixel convolution (PixelShuffle), first expanding the number of channels through 3 × 3 convolution, then converting the channel dimension into spatial dimension through pixel rearrangement, and finally outputting a high-resolution reconstructed image through the convolutional layer. I SR .

[0013] To verify the effectiveness of this embodiment, the network training configuration is as follows: The DIV2K and Flickr2K datasets are used as the training set; during training, high-resolution images are randomly cropped into 192 × 192 patches, corresponding to a low-resolution input size of 48 × 48; the L1 loss function is used to minimize the pixel difference between the reconstructed image and the real image; the Adam optimizer is used, with parameters set to... β 1 = 0.9, β 2 = 0.99; initial learning rate set to 1 × 10 -4 The learning rate was gradually reduced during the 800 epochs of training using a cosine annealing strategy; the model was trained on an NVIDIA RTX 4080 GPU.

[0014] As shown in Table 1, experiments conducted by this invention on benchmark datasets such as Set5 and Urban100 with 4x magnification demonstrate that MHARN achieves significant improvements in both PSNR and SSIM metrics compared to existing technologies such as EDSR, RCAN, and MARNet (e.g., PSNR reaches 27.46 dB on Urban100 × 4).

[0015] Method Scale Set5 Set14 BSD100 Urban100 Manga109 Bicubic ×4 28.42 / 0.8104 26.00 / 0.7027 25.96 / 0.6675 23.14 / 0.6577 24.89 / 0.7866 EDSR ×4 32.46 / 0.8968 28.80 / 0.7876 27.71 / 0.7420 26.64 / 0.8033 31.02 / 0.9148 RCAN ×4 32.62 / 0.9001 28.86 / 0.7888 27.76 / 0.7435 26.82 / 0.8087 31.21 / 0.9172 RDN ×4 32.47 / 0.8990 28.81 / 0.7871 27.72 / 0.7419 26.61 / 0.8028 31.00 / 0.9151 HAN ×4 32.64 / 0.9002 28.90 / 0.7890 27.80 / 0.7442 26.85 / 0.8094 31.42 / 0.9177 ELAN ×4 32.59 / 0.9000 28.87 / 0.7891 27.78 / 0.7444 26.96 / 0.8109 31.27 / 0.9184 DHTCUN ×4 32.83 / 0.9029 29.01 / 0.7929 27.98 / 0.7474 27.34 / 0.8191 31.78 / 0.9230 Ours ×4 32.83 / 0.9017 29.18 / 0.7956 27.91 / 0.7406 27.46 / 0.8196 31.81 / 0.9246 Furthermore, as shown in Figure 6, in terms of visual contrast, thanks to the enhanced adaptability to object scale of the PCG module and the more comprehensive cross-dimensional feature interaction of the MHAM module, especially the content-aware dynamic convolution of the DFEM module, this invention significantly reduces visual artifacts when reconstructing images, restoring clearer and sharper texture details compared to contrast algorithms. The method described in this invention can be widely applied in fields such as medical image analysis (e.g., lesion detail restoration), security monitoring (e.g., face or license plate sharpening), and consumer electronics image enhancement.

[0016] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image super-resolution reconstruction method based on a multi-scale hybrid attention residual network, characterized in that, The process includes the following steps: Acquiring a low-resolution image to be reconstructed; Inputting the low-resolution image into a pre-constructed progressive convolutional group (PCG) module for shallow feature extraction to obtain a shallow feature map; Inputting the shallow feature map into a cascaded set of enhanced residual attention blocks (ERABs) for deep feature extraction to obtain a deep feature map; Each ERAB contains a multi-head hybrid attention module (MHAM) and a dynamic feature enhancement module (DFEM); Fusing the deep feature map through a global feature fusion module and performing a global residual connection with the shallow feature map to obtain a fused feature map; Inputting the fused feature map into an upsampling module for resolution upsampling to output the reconstructed high-resolution image.

2. The method according to claim 1, characterized in that, The Progressive Convolutional Group (PCG) module adopts a parallel topology structure, including multiple parallel convolutional branches; each branch uses convolutional kernels with different dilation rates to process the input features in order to extract multi-scale receptive field features. The outputs of each branch are weighted by the channel attention mechanism and then fused by the dynamic gating mechanism.

3. The method according to claim 1, characterized in that, The Multi-Head Hybrid Attention Module (MHAM) includes parallel window attention, coordinate attention, and channel-spatial attention branches: the window attention branch captures short-range feature interaction information within a local window to restore high-frequency texture details; the coordinate attention branch captures position-sensitive information in the horizontal and vertical directions to preserve the geometric structure of the image; and the channel-spatial attention branch recalibrates feature channels based on global statistical information. The outputs of the three branches are adaptively weighted and fused using normalized fusion weights generated by a spatial gating mechanism.

4. The method according to claim 1, characterized in that, The Dynamic Feature Enhancement (DFEM) module is used to adaptively generate convolutional kernel parameters based on the content of the input features. Specific steps include: generating a dynamic parameter map from the input features using a parameter generation network; extracting local neighborhood features of the input features at each spatial location using an unfold operation; performing a dot product operation between the local neighborhood features and the dynamic parameter map to generate dynamic convolutional features; and weightedly fusing the dynamic convolutional features with the original input features using a gating mechanism to obtain the enhanced features.

Images