Efficient multi-attention feature fusion super-resolution reconstruction model, method, device and storage medium
By using a super-resolution reconstruction model that integrates efficient multi-attention features, the problems of information loss and large parameter count caused by excessively deep network layers are solved, achieving efficient restoration of image edge contours and texture details, and improving image reconstruction quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV OF SCI & TECH
- Filing Date
- 2022-10-20
- Publication Date
- 2026-07-07
AI Technical Summary
Existing deep learning-based single-image super-resolution reconstruction methods suffer from poor reconstruction results, mainly due to excessively deep network layers leading to information loss and a large number of parameters, making it difficult to achieve lightweight design while maintaining performance.
A super-resolution reconstruction model employing efficient multi-attention feature fusion is adopted. By combining shallow and deep feature extraction modules with progressive feature fusion blocks and efficient multi-attention blocks, image feature information is extracted step by step. Feature enhancement and fusion are performed through multi-scale receptive field blocks to reduce information loss and improve reconstruction performance.
More high-frequency information and texture details were restored, and the reconstructed image is closer to the original image, improving the observation effect while keeping the model lightweight.
Smart Images
Figure CN115660955B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image reconstruction technology and relates to an efficient super-resolution reconstruction model, method, device and storage medium based on multi-attention feature fusion. Background Technology
[0002] Single image super-resolution (SISR) technology refers to the process of reconstructing a given low-resolution (LR) image into a corresponding high-resolution (HR) image using a specific algorithm. It aims to overcome or compensate for problems such as low image quality and indistinct regions of interest caused by limitations in the image acquisition system or environment. Traditional image super-resolution reconstruction algorithms mainly rely on basic digital image processing techniques, which are computationally complex and cannot effectively recover the original image information.
[0003] With the widespread application of deep learning in image super-resolution reconstruction and the gradual achievement of good results, it has attracted increasing attention from researchers. Dong et al. were the first to apply convolutional neural networks to super-resolution reconstruction technology. Their proposed SRCNN uses three convolutional layers to generate reconstructed images through pixel mapping. See "C. Dong, CCLoy, K. He, X. Tang, Image super-resolution using deep convolutional networks, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2014, pp. 184-199"; Dong et al., based on SRCNN, proposed FSRCNN, which uses smaller convolutional layers internally and enlarges the size at the end of the network through deconvolutional layers. See "C. Dong, CCLoy, X. Tang, Accelerating the super-resolution convolutional neural network, Computer Vision-ECCV Workshops, 2016, pp. 391-407"; Shi et al. proposed the efficient sub-pixel convolutional neural network ESPCN, which uses rearranged sub-pixel layers in the reconstruction module to achieve upsampling. See "W. Shi, J. Caballero, F. Huszar, et al." al., Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, in Proc. IEEE Conf Comput Vis. Pattern Recognit (CVPR), 2016, pp. 1874-1883. The above super-resolution reconstruction methods are relatively lightweight, but the reconstruction results are not as expected.
[0004] Therefore, people began to improve model performance by increasing network depth. Kim et al. proposed a 20-layer deep network, VDSR, which uses the idea of residual learning to accelerate the convergence speed of network training. For details, see "J. Kim, KJ Lee, MK Lee, Accurate Image Super-Resolution Using Very Deep Convolutional Networks, in Proc. IEEE Conf Comput. Vis. Pattern Recognit (CVPR), 2016, pp. 1646-1654"; Ahn et al. proposed an architecture CARN that implements a cascading mechanism on a residual network (ResNet). Its middle part is based on ResNet, and the global and local parts of the network use a cascading mechanism to better integrate the features of each layer. For details, see "N. Ahn, B. Kang, AK Sohn, Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network, Computer Vision-ECCV" Workshops, 2018, pp. 252-268; Zhao et al. proposed the Pixel Attention Super-Resolution Reconstruction Network PAN, which improves the reconstruction performance of the network by introducing a lightweight pixel attention mechanism, see "H. Zhao, X. Kong, J. He, et al., Efficient image super-resolution using pixel attention, Computer Vision-ECCV Workshops, 2020, pp. 56-72"; Tian et al. proposed CFSRCNN, which uses a feature extraction module to learn long and short path features and fuses the learned features by extending the information from the shallow layers of the network to the deep layers, see "C. Tian, Y. Xu, W. Zuo, et al., Coarse-to-fine CNN for image super-resolution, IEEE Transactions on Multimedia, vol. 23, 2021, pp. 1489-1502". The above methods improve the model performance by increasing the number of network layers, but the resulting surge in parameters leads to a significant increase in training difficulty, and the reconstruction performance of some models will significantly decrease after parameter reduction. It is evident that it is difficult to achieve a good balance between parameters and performance.
[0005] The inventors have found that existing deep learning-based single-image super-resolution reconstruction methods have poor reconstruction results, mainly due to the following problems: (1) Some super-resolution reconstruction networks based on CNNs are relatively lightweight, but their performance does not meet expectations. (2) More and more models are increasing network depth to pursue better reconstruction results. Although the performance has improved, the large number of parameters increases the time complexity of training. At the same time, the loss of feature information during the transmission of deep network also affects the reconstruction effect. How to make full use of the feature information of the image, enable the limited features to be better transmitted and reused, efficiently restore the edge contours and texture details of the image, and make the model as lightweight as possible while ensuring performance has become an urgent problem to be solved. Summary of the Invention
[0006] To address the aforementioned issues, this invention provides an efficient multi-attention feature fusion image super-resolution reconstruction model that can improve image quality, enhance observation results, and solve the problems of information loss and large parameter count caused by excessively deep network layers in existing super-resolution reconstruction algorithms.
[0007] The second objective of this invention is to provide an efficient image super-resolution reconstruction method using multi-attention feature fusion.
[0008] A third objective of this invention is to provide an electronic device.
[0009] A fourth objective of this invention is to provide a computer storage medium.
[0010] The technical solution adopted in this invention is an efficient multi-attention feature fusion super-resolution reconstruction model, including a feature extraction module and a reconstruction module;
[0011] The feature extraction module is divided into a shallow feature extraction module and a deep feature extraction module.
[0012] The shallow feature extraction module is a 3*3 convolutional layer that performs initial feature extraction on the input low-resolution LR image in a low-dimensional space, effectively reducing its computational cost.
[0013] The deep feature extraction module contains eight progressive feature fusion blocks (PFFB). PFFB uses a progressive fusion connection method to gradually extract deep feature information of the image to enhance feature transfer. At the same time, it combines its internal efficient multi-attention block (EMAB) to weight the extracted feature information so that the network pays more attention to high-frequency information.
[0014] The reconstruction module consists of a multi-scale receptive field block RFB_x, a 3*3 convolutional layer, and a sub-pixel convolutional layer. RFB_x utilizes a multi-branch structure to further enhance the features extracted from the PFFB block and fuses multi-scale feature information to improve the model's reconstruction performance.
[0015] The bicubic upsampling result of the low-resolution LR image is then superimposed with the upsampling result of the subpixel convolutional layer to obtain the reconstructed image.
[0016] Furthermore, the deep feature extraction module includes eight progressive feature fusion blocks (PFFB);
[0017] The PFFB uses four efficient multi-attention blocks (EMABs) to progressively extract deep information from the image.
[0018] The PFFB achieves "information exchange" of convolutional layer results in the EMAB block through multiple channel random mixing (C shuffle). It regroups the output channels and then mixes the information of different channels to solve the problem of poor information flow between convolutional layers, and fully integrates the channels without increasing the amount of computation.
[0019] The PFFB performs a C-shuffle on the features extracted from each EMAB block, then connects two adjacent C-shuffled features and performs another C-shuffle to improve the network's generalization ability. A 1x1 convolution is used to remove redundant information, and the result is fused with the information from the next C-shuffled operation. This operation is repeated between EMAB blocks within the PFFB to progressively collect local information and perform feature fusion, strengthening feature transfer and helping to improve the accuracy of the reconstructed image. Finally, residual learning is used to transfer the input features x... i The output feature x of the PFFB block is obtained by superimposing the fused features. i+1 This maximizes the use of LR image information to mitigate feature loss during transmission;
[0020] The PFFB strengthens feature extraction and fuses the extracted multi-layer information through a progressive feature fusion connection method, making it easier for each layer to make full use of all the features learned from the previous layers, and enabling better transfer and reuse of limited features.
[0021] Furthermore, the PFFB employs four efficient multi-attention blocks (EMABs) to extract features layer by layer;
[0022] The EMAB fully utilizes the feature information of channels and space to gradually denoise the shallow features of the image, allowing the network to focus on high-frequency details in the image, which helps to enhance the texture detail information of the reconstructed image.
[0023] The EMAB uses a 1*1 convolutional layer after two 3*3 convolutional kernels to reduce the channel size, expands the receptive field through a stride convolution with a stride of 2, and further reduces the spatial dimension of the network by combining a 2*2 max pooling layer; then it uses a dilated convolutional layer to further aggregate the contextual information of the receptive field, reducing memory usage while improving network performance, upsampling the obtained features to restore the spatial dimension, and restores the channel dimension through a 1*1 convolution.
[0024] The EMAB uses the Frelu activation function after three convolutional layers to accelerate convergence and prevent gradient explosion.
[0025] The EMAB uses an efficient channel attention block to avoid the problems caused by dimensionality reduction. The channel attention is generated by fast one-dimensional convolution, and the size of the internal convolution kernel is adaptively determined through non-linear mapping of the channel dimension.
[0026] The one-dimensional convolution can efficiently realize local cross-channel interaction. By capturing local cross-channel information, it completes mutual communication between them and learns effective channel attention.
[0027] Furthermore, the multi-scale receptive field block RFB_x in the reconstruction module is composed of 1*1, 3*3, 1*3 and 3*1 convolutional kernels;
[0028] The RFB_x is located after the 8 sequentially connected PFFBs and is responsible for enhancing the extracted deep features, fusing features at multiple scales and reconstructing them, preserving rich features and restoring image details.
[0029] Specifically, the output feature x8 of the 8th PFFB block is used as the input of the RFB-x block. Multi-scale feature extraction is performed using multi-branch convolutional layers of different sizes. At the same time, dilated convolutions with different dilation rates are introduced. The larger the dilation rate of the dilated convolution, the farther the sampling point is from the center point, and the larger the receptive field. This helps to capture information in a larger area to generate better feature maps without increasing the number of parameters.
[0030] Finally, the outputs of multiple branches are connected to fuse different features at multiple scales, resulting in the feature x extracted by RFB_x. e .
[0031] An efficient super-resolution reconstruction method based on multi-attention feature fusion is proposed, comprising the following steps:
[0032] S1. Inputting low-resolution LR images into a super-resolution reconstruction model that integrates efficient multi-attention features;
[0033] S2, the feature extraction module of the super-resolution reconstruction model, after extracting the shallow features of the LR image, performs deep feature extraction through 8 progressive feature fusion blocks (PFFB) and sends it to the reconstruction module.
[0034] S3. The reconstruction module uses RFB_x to enhance the extracted deep features and fuses the features at multiple scales to obtain the fused multidimensional feature x. e ;
[0035] S4. The feature x output by RFB_x e A 3x3 convolution is performed and amplified through a subpixel convolutional layer. Simultaneously, the input LR features are bicubic upsampled, and the bicubic upsampled result of the LR image is superimposed with the upsampled result of the subpixel convolutional layer to obtain the reconstructed super-resolution image.
[0036] Furthermore, the feature extraction module of S2 performs deep feature extraction as follows:
[0037] x0 = f IFE (I LR (1)
[0038]
[0039] Among them, I LR It is the input LR image, f IFE This is a 3x3 convolution operation, where x0 represents the extracted initial features. Let x be the mapping function for the i-th (i = 0, 1, ..., 7) progressive feature fusion block PFFB. i+1 This represents the deep features extracted by the i-th PFFB in the feature extraction module.
[0040] Furthermore, the process by which the reconstruction module of S3 enhances the extracted deep features using RFB_x is as follows:
[0041] x e =f RFB_x (x8) (3)
[0042] Among them, f RFB_x This is a function that enhances the deep features x8 extracted from 8 PFFB blocks using RFB_x. e To enhance the results, specifically the multidimensional features output by RFB_x.
[0043] Furthermore, S4 completes the reconstruction according to the following formula:
[0044] I SR =f P (x e )+f up (ILR (4)
[0045] Among them, f P The enhancement result x e Perform 3x3 convolution and subpixel convolution operations, f up This involves performing a bicubic upsampling operation on the input low-resolution LR image. SR This is the final super-resolution SR image.
[0046] An electronic device that uses the above-described method to reconstruct an image.
[0047] A computer storage medium storing at least one program instruction, which is loaded and executed by a processor to implement the image reconstruction method described above.
[0048] The beneficial effects of this invention are:
[0049] To improve image quality and enhance observation results, this invention proposes an efficient multi-attention feature fusion image super-resolution reconstruction algorithm (EMAFFN) to address the problems of information loss and large parameter count caused by excessively deep network layers in existing super-resolution reconstruction algorithms. This algorithm progressively extracts image features through a progressive feature fusion block (PFFB), reducing feature loss during deep network transmission. Simultaneously, it leverages the efficient multi-attention block (EMAB) within the PFFB, which operates in both channel and spatial branches, adaptively weighting the extracted features to focus more on high-frequency information. Finally, a multi-scale receptive field block (RFB_x) enhances the extracted features and performs multi-scale feature fusion to improve the performance of the reconstruction module. The reconstructed image obtained by this invention recovers more high-frequency information, has richer texture details, and is closer to the original image. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 This is a schematic diagram of the reconstruction model in an embodiment of the present invention.
[0052] Figure 2 This is a schematic diagram of the progressive feature fusion block (PFFB) in the reconstruction model of this invention.
[0053] Figure 3This is a schematic diagram of the structure of the efficient multi-attention block EMAB in the reconstruction model of this invention.
[0054] Figure 4 This is a schematic diagram of the structure of the multi-scale receptive field block RFB_x in the reconstruction model of this invention embodiment.
[0055] Figure 5 This is a comparison chart of the reconstruction effect of the reconstruction method of this invention and other algorithms when the magnification factor of low-resolution image is ×4. Detailed Implementation
[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0057] Example 1,
[0058] An efficient multi-attention feature fusion super-resolution reconstruction model, the structure of which is as follows: Figure 1 As shown, it includes a feature extraction module and a reconstruction module;
[0059] The feature extraction module is divided into a shallow feature extraction module and a deep feature extraction module.
[0060] The shallow feature extraction module is a 3*3 convolutional layer that performs initial feature extraction on the input low-resolution LR image in a low-dimensional space, effectively reducing its computational cost.
[0061] The deep feature extraction module contains eight progressive feature fusion blocks (PFFB).
[0062] like Figure 2PFFB employs a progressive fusion connection method to gradually extract deep-level features from the image to enhance feature transfer. Simultaneously, it combines this with its internal efficient multi-attention blocks (EMABs) to weight the extracted features, allowing the network to focus more on high-frequency information. PFFB uses four efficient EMABs layer by layer to progressively extract deep-level image information. PFFB achieves "information exchange" between the convolutional layer results in the EMAB blocks through multiple channel random mixing (C shuffle). It regroups the output channels and then mixes information from different channels, resolving information flow issues between convolutional layers and ensuring sufficient channel fusion without increasing computational load. PFFB performs C shuffle on each feature extracted by an EMAB block, then connects two adjacent C shuffled features and performs another C shuffle to improve the network's generalization ability. It uses 1*1 convolutions to remove redundant information, and the resulting convolution is then combined with the next C shuffled feature. Information from the shuffle operation is used for feature fusion; this operation is repeated between EMAB blocks within the PFFB to progressively collect local information and perform feature fusion, enhancing feature transfer and improving the accuracy of the reconstructed image; finally, residual learning is used to integrate the input features x i The output feature x of the PFFB block is obtained by superimposing the fused features. i+1 It maximizes the use of LR image information to alleviate feature loss during the transmission process; PFFB strengthens feature extraction and fuses the extracted multi-layer information through a "progressive" feature fusion connection method, so that each layer can make full use of all the features learned by the previous layers, and achieve better transmission and reuse of limited features.
[0063] like Figure 3EMAB fully utilizes channel and spatial feature information to gradually denoise shallow features of the image, allowing the network to focus on high-frequency details in the image, which helps enhance the texture details of the reconstructed image. After two 3*3 convolutional kernels, EMAB uses a 1*1 convolutional layer to reduce the channel size, expands the receptive field through a stride of 2 convolution, and further reduces the spatial dimension of the network by combining a 2*2 max pooling layer. Then, a dilated convolutional layer is used to further aggregate the contextual information of the receptive field, reducing memory usage while improving network performance. The obtained features are upsampled to restore the spatial dimension, and the channel dimension is restored through a 1*1 convolution. After three convolutional layers, EMAB uses the Frelu activation function to accelerate convergence and prevent gradient explosion. EMAB employs efficient channel attention blocks to avoid the problems caused by dimensionality reduction. Channel attention is generated by fast one-dimensional convolution, and the size of the internal convolutional kernel is adaptively determined through a non-linear mapping of the channel dimension. One-dimensional convolution can efficiently realize local cross-channel interaction, and by capturing local cross-channel information, it completes mutual communication and learns effective channel attention.
[0064] The reconstruction module consists of a multi-scale receptive field (RFB_x), a 3x3 convolutional layer, and a sub-pixel convolutional layer. RFB_x utilizes a multi-branch structure to further enhance the features extracted from the PFFB block and fuses multi-scale feature information to improve the model's reconstruction performance.
[0065] like Figure 4 The multi-scale receptive field block RFB_x in the reconstruction module is composed of 1*1, 3*3, 1*3, and 3*1 convolutional kernels. RFB_x follows eight sequentially connected PFFBs and is responsible for enhancing the extracted deep features, fusing features at multiple scales, and reconstructing the image, preserving rich features and restoring image details. Specifically, the output feature x8 of the 8th PFFB block is used as the input to the RFB-x block. Multi-scale feature extraction is performed using multi-branch convolutional layers of different sizes, while dilated convolutions with different dilation rates are introduced. The larger the dilation rate of the dilated convolution, the farther the sampling point is from the center point, resulting in a larger receptive field. This helps capture information in a larger area to generate better feature maps without increasing the number of parameters. Finally, the outputs of multiple branches are connected to fuse different features at multiple scales, obtaining the feature x extracted by RFB_x. e .
[0066] like Figure 1 Finally, the features x extracted from RFB_x are... e The reconstructed image is obtained by superimposing the bicubic upsampling result of the low-resolution LR image with the upsampling result of the subpixel convolutional layer.
[0067] Example 2,
[0068] An efficient super-resolution reconstruction method based on multi-attention feature fusion is proposed, comprising the following steps:
[0069] S1. Inputting low-resolution LR images into a super-resolution reconstruction model that integrates efficient multi-attention features;
[0070] S2, the feature extraction module of the super-resolution reconstruction model, after extracting the shallow features of the LR image, performs deep feature extraction through 8 progressive feature fusion blocks (PFFB) and sends it to the reconstruction module.
[0071] S3. The reconstruction module uses RFB_x to enhance the extracted deep features and fuses the features at multiple scales to obtain the fused multidimensional feature x. e ;
[0072] S4. The feature x output by RFB_x e A 3x3 convolution is performed and amplified through a subpixel convolutional layer. Simultaneously, the input LR features are bicubic upsampled, and the bicubic upsampled result of the LR image is superimposed with the upsampled result of the subpixel convolutional layer to obtain the reconstructed super-resolution image.
[0073] Furthermore, the feature extraction module of S2 performs deep feature extraction as follows:
[0074] x0 = f IFE (I LR (1)
[0075]
[0076] Among them, I LR It is the input LR image, f IFE This is a 3x3 convolution operation, where x0 represents the extracted initial features. Let x be the mapping function for the i-th (i = 0, 1, ..., 7) progressive feature fusion block PFFB. i+1 This represents the deep features extracted by the i-th PFFB in the feature extraction module.
[0077] Furthermore, the process by which the reconstruction module of S3 enhances the extracted deep features using RFB_x is as follows:
[0078] x e =f RFB_x (x8) (3)
[0079] Among them, f RFB_x This is a function that enhances the deep features x8 extracted from 8 PFFB blocks using RFB_x. e To enhance the results, specifically the multidimensional features output by RFB_x.
[0080] Furthermore, S4 completes the reconstruction according to the following formula:
[0081] I SR =f P (x e )+f up (I LR (4)
[0082] Among them, f P The enhancement result x e Perform 3x3 convolution and subpixel convolution operations, f up This involves performing a bicubic upsampling operation on the input low-resolution LR image. SR This is the final super-resolution SR image.
[0083] To reduce reconstruction error, the L1 loss function is used to optimize the network parameters during training. For a given N LR-HR image pairs in the training set... The optimization goals are as follows:
[0084]
[0085] Where k represents the k-th LR-HR image in the training set, k∈[1,N] and k∈Z, N is 800, θ={w k ,b k} represents the learning parameters of the model, H SR This is the model presented in this paper. By continuously training and optimizing the model parameters, L(θ) is minimized, so that the reconstructed image is as close as possible to the real image.
[0086] To verify the effectiveness of the efficient multi-attention feature fusion image super-resolution reconstruction method of the present invention, four widely used benchmark datasets in super-resolution reconstruction—Set5, Set14, BSD100, and Urban100—were selected as test sets. The following algorithms were used: Keys' algorithm (R. Keys, Cubic convolution interpolation for digital image processing, in IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, December 1981, pp. 1153-1160); Dong's algorithm (C. Dong, CCLoy, K. He, X. Tang, Image super-resolution using deep convolutional networks, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2014, pp. 184-199); and Dong's algorithm (C. Dong, CCLoy, X. Tang, Accelerating the super-resolution convolutional neural network, Computer Vision-ECCV). Workshops, 2016, pp.391-407); Shi’s algorithm (W.Shi, J.Caballero, F.Huszar, et al., Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, inProc.IEEE Conf Comput Vis.Pattern Recognit(CVPR), 2016, pp.1874-1883); Kim’s algorithm (J.Kim, KJLee, MKLee, Accurate Image Super-Resolution Using Very DeepConvolutional Networks, in Proc.IEEE Conf Comput.Vis.Pattern Recognit (CVPR), 2016, pp.1646-1654); Ahn’s algorithm (N.Ahn, B.Kang, AKThe experimental results of this invention are compared and analyzed from both subjective and objective perspectives: Sohn, Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network, Computer Vision-ECCV Workshops, 2018, pp. 252-268; Zhao's algorithm (H. Zhao, X. Kong, J. He, et al., Efficient image super-resolution using pixel attention, Computer Vision-ECCV Workshops, 2020, pp. 56-72); Tian's algorithm (C. Tian, Y. Xu, W. Zuo, et al., Coarse-to-fine CNN for image super-resolution, IEEE Transactions on Multimedia, vol. 23, 2021, pp. 1489-1502).
[0087] like Figure 5 The image above shows a comparison of the image super-resolution reconstruction results of the efficient multi-attention feature fusion method provided in this embodiment of the invention with other algorithms on the Urban100 test set. The larger image on the left is the high-resolution original image, with areas rich in texture details marked and magnified. The eight smaller images on the right, in order from left to right and top to bottom, are the original image, the reconstruction results of Dong's SRCNN method, Dong's FSRCNN method, Kim's VDSR method, Ahn's CARN method, Zhao's PAN method, Tian's CFSRCNN method, and the reconstruction result of the method in this embodiment of the invention. It can be observed that the image reconstructed by this method almost accurately recovers the shape of the stripes, with the clearest edge contours; while the images reconstructed by other methods all exhibit visual blurring and fail to effectively restore the true content of the image. Therefore, the reconstruction effect of the method in this embodiment of the invention is significantly superior, and the reconstructed image recovers more high-frequency information, making it closer to the original image.
[0088] To avoid biases from qualitative analysis, this example uses two objective metrics, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), for objective quantitative analysis. On four test datasets—Set5, Set14, BSD100, and Urban100—the reconstruction performance is compared with that of Keys' Bicubic algorithm, Dong's SRCNN algorithm, Dong's FSRCNN algorithm, Shi's ESPCN algorithm, Kim's VDSR algorithm, Ahn's CARN algorithm, Zhao's PAN algorithm, and Tian's CFSRCNN algorithm at magnifications of 2, 3, and 4 times, as shown in Table 1.
[0089] Table 1. PSNR and SSIM values of different algorithms on the test set.
[0090]
[0091] For PSNR and SSIM, higher values indicate a greater similarity to the real image and higher image quality. Table 1 shows that at a magnification of ×4, the method of this embodiment achieved the optimal SSIM value on all four test sets. Comparisons show that, except for a slight difference in PSNR and SSIM values between the PAN algorithm and this embodiment on some datasets, the method of this embodiment achieves the best values compared to other methods. Specifically, the method of this embodiment achieves the highest average PSNR of 37.93 dB and the optimal SSIM of 0.9609. Therefore, the method of this embodiment significantly improves the peak signal-to-noise ratio and structural similarity of the reconstructed image, resulting in enhanced visual quality and richer detail in the reconstructed image.
[0092] If the image reconstruction method described in this embodiment of the invention is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the image reconstruction method described in this embodiment of the invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, ROM, RAM, magnetic disks, or optical disks.
[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A method for constructing an efficient super-resolution reconstruction model using multi-attention feature fusion, characterized in that, It mainly includes a feature extraction module and a reconstruction module; The feature extraction module is divided into a shallow feature extraction module and a deep feature extraction module. The shallow feature extraction module is a 3 Three convolutional layers are used to extract initial features from the input low-resolution LR image in a low-dimensional space, effectively reducing its computational cost. The deep feature extraction module contains eight progressive feature fusion blocks (PFFB). PFFB uses a progressive fusion connection method to gradually extract deep feature information of the image to enhance feature transfer. At the same time, it combines its internal efficient multi-attention block (EMAB) to weight the extracted feature information so that the network pays more attention to high-frequency information. The PFFB uses four efficient multi-attention blocks (EMABs) to progressively extract deep information from the image. The PFFB achieves "information exchange" of convolutional layer results in the EMAB block by randomly mixing multiple channels. It regroups the output channels and then mixes the information from different channels to solve the problem of poor information flow between convolutional layers, and fully integrates the channels without increasing the amount of computation. The PFFB performs channel random mixing on each feature extracted by the EMAB block, and then connects two adjacent features that have undergone channel random mixing and performs channel random mixing again to improve the network's generalization ability. Using 1 1. Convolution removes redundant information, and the resulting product is fused with the information from the next channel random mixing operation. This operation is repeated between EMAB blocks within the PFFB to progressively collect local information and perform feature fusion, enhancing feature transfer and improving the accuracy of the reconstructed image. Finally, residual learning is used to integrate the input features. The first feature is obtained by superimposing it with the fused features. i ( i Output characteristics of PFFB blocks (0, 1, ..., 7) This maximizes the use of LR image information to mitigate feature loss during transmission; The PFFB strengthens feature extraction and fuses the extracted multi-layer information through a progressive feature fusion connection method, which makes it easier for each layer to make full use of all the features learned from the previous layers, and enables better transfer and reuse of limited features. The EMAB fully utilizes the feature information of channels and space to gradually denoise the shallow features of the image, allowing the network to focus on high-frequency details in the image, which helps to enhance the texture detail information of the reconstructed image. The reconstruction module consists of a multi-scale receptive field block RFB_x and a 3 Composed of three convolutional layers and one sub-pixel convolutional layer, RFB_x utilizes a multi-branch structure to further enhance the features extracted from the PFFB block and fuses multi-scale feature information to improve the model's reconstruction performance. The multi-scale receptive field block RFB_x consists of 1 1, 3 3.1 3 and 3 It is composed of 1 convolution kernel; Finally, the bicubic upsampling result of the low-resolution LR image is superimposed with the upsampling result of the subpixel convolutional layer to obtain the reconstructed high-resolution image.
2. The Progressive Feature Fusion Block (PFFB) according to claim 1, characterized in that, The PFFB uses four efficient multi-attention blocks (EMABs) to extract features layer by layer; The EMAB fully utilizes the feature information of channels and space to gradually denoise the shallow features of the image, allowing the network to focus on high-frequency details in the image, which helps to enhance the texture detail information of the reconstructed image. The EMAP is in two 3 After 3 convolution kernels, use 1 One convolutional layer reduces the channel size, while a stride of 2 is used to expand the receptive field and combine it with 2 The max-pooling layer (2 layers) further reduces the spatial dimensionality of the network; then, dilated convolutional layers are used to further aggregate the contextual information of the receptive field, reducing memory usage while improving network performance. The obtained features are upsampled to restore the spatial dimensionality, and then...
1. Convolution restores channel dimension; The EMAB uses the Frelu activation function after three convolutional layers to accelerate convergence and prevent gradient explosion. The EMAB uses an efficient channel attention block to avoid the problems caused by dimensionality reduction. The channel attention is generated by fast one-dimensional convolution, and the size of the internal convolution kernel is adaptively determined through non-linear mapping of the channel dimension. The one-dimensional convolution can efficiently realize local cross-channel interaction. By capturing local cross-channel information, it completes mutual communication between them and learns effective channel attention.
3. The high-efficiency multi-attention feature fusion super-resolution reconstruction model according to claim 1, characterized in that, The multi-scale receptive field block RFB_x in the reconstruction module consists of 1 1, 3 3.1 3 and 3 It is composed of 1 convolution kernel; The RFB_x is located after the 8 sequentially connected PFFBs and is responsible for enhancing the extracted deep features, fusing features at multiple scales and reconstructing them, preserving rich features and restoring image details. Specifically, the output features of the 8th PFFB block As input to the RFB-x block, multi-branch convolutional layers of different sizes are used for multi-scale feature extraction. At the same time, dilated convolutions with different dilation rates are introduced. The larger the dilation rate of the dilated convolution, the farther the sampling point is from the center point, and the larger the receptive field. This helps to capture information in a larger area to generate better feature maps without increasing the number of parameters. Finally, the outputs of multiple branches are connected to fuse different features at multiple scales, resulting in the features extracted by RFB_x. .
4. A highly efficient multi-attention feature fusion super-resolution reconstruction method applied to the method described in claim 1, characterized in that, Follow these steps: S1. Inputting low-resolution LR images into a super-resolution reconstruction model that integrates efficient multi-attention features; S2, the feature extraction module of the super-resolution reconstruction model, after extracting the shallow features of the LR image, performs deep feature extraction through 8 progressive feature fusion blocks (PFFB) and sends it to the reconstruction module. S3. The reconstruction module uses RFB_x to enhance the extracted deep features and fuses multi-scale features to obtain fused multi-dimensional features. ; S4. Features output by RFB_x Perform 3 The image is convolved and amplified by a subpixel convolutional layer. Simultaneously, the input LR features are bicubic upsampled, and the bicubic upsampled results of the LR image are superimposed with the upsampled results of the subpixel convolutional layer to obtain the reconstructed super-resolution image.
5. The efficient multi-attention feature fusion super-resolution reconstruction method according to claim 4 is applied to the method according to claim 1, characterized in that, The feature extraction module of S2 performs deep feature extraction as follows: （1） （2） in, It is the input LR image. For a 3 3-dimensional convolution operation, For the extracted initial features, For the first i ( i The mapping function for progressive feature fusion blocks PFFB (=0,1,…,7) is given. For the feature extraction module i Deep features extracted from PFFB.
6. The efficient multi-attention feature fusion super-resolution reconstruction method according to claim 4 is applied to the method according to claim 1, characterized in that, The process by which the reconstruction module of S3 enhances the extracted deep features using RFB_x is as follows: （3） in, To use RFB_x to process deep features extracted through 8 PFFB blocks The function to be enhanced. To enhance the results, specifically the multidimensional features output by RFB_x.
7. The efficient multi-attention feature fusion super-resolution reconstruction method according to claim 4 is applied to the method according to claim 1, characterized in that, The reconstruction in S4 is completed according to the following formula. （4） in, Enhanced results Perform 3 3. Convolution and subpixel convolution operations, This involves performing a bicubic upsampling operation on the input low-resolution LR image. This is the final super-resolution SR image.
8. An electronic device, characterized in that, Image reconstruction is achieved using the method described in any one of claims 4 to 7.
9. A computer storage medium, characterized in that, The storage medium stores at least one program instruction, which is loaded and executed by a processor to implement the image reconstruction method as described in any one of claims 4 to 7.