An image super-high resolution reconstruction method based on an attention mechanism

Abstract

The application belongs to the technical field of image processing, and provides an image super-high resolution reconstruction method based on an attention mechanism, which comprises the following steps: shallow feature extraction is performed on a low-resolution image to be processed through a single convolution layer of a convolution network to obtain a shallow feature map; deep extraction is performed on the shallow feature map through RIR to obtain a deep feature map rich in high-frequency details; a high-resolution feature map is obtained by performing an up-sampling operation on the deep feature map through sub-pixel convolution; and the high-resolution feature map is reconstructed by using a single convolution layer of the convolution network, and a final super-resolution image is output. The application can reliably and accurately restore high-frequency details, avoid blurring / artifacts / false details, and improve the practicability and credibility in key applications.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and provides an image ultra-high resolution reconstruction method based on an attention mechanism. Background Technology

[0002] Image resolution refers to the ability of an imaging system to distinguish image details, and it is one of the important indicators for measuring image quality. High-resolution images can provide rich detail information, and with the continuous progress of economy, technology, and civilization, the demand for high-resolution images is increasing in various fields such as medicine, security, and entertainment. For example, doctors hope to identify lesions through high-resolution CT or ultrasound images; entertainment businesses hope to provide viewers with a more realistic and detailed visual experience through high-resolution videos.

[0003] Disadvantages of existing technologies: Existing super-resolution technologies (especially deep learning methods) have difficulty reliably and accurately recovering key high-frequency details when magnifying at high magnification or processing low-quality inputs. They are also susceptible to degradation or the introduction of blurring / artifacts / false details, resulting in insufficient practicality and reliability in serious applications (medical, security) that require precise detail identification. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an image ultra-high resolution reconstruction method based on an attention mechanism, which can reliably and accurately recover high-frequency details, avoid blurring / artifacts / false details, and improve practicality and reliability in key applications.

[0005] The technical solution of the present invention includes the following steps; low-resolution images to be processed Shallow feature maps are obtained by extracting shallow features through a single convolutional layer of a convolutional network. .

[0006] Shallow feature map obtained through RIR Deep extraction is performed to obtain a deep feature map rich in high-frequency details. .

[0007] Deep feature maps are processed through subpixel convolution. Perform upsampling to obtain high-resolution feature maps .

[0008] A single convolutional layer of a convolutional network is used to process high-resolution feature maps. Reconstruct and output the final super-resolution image. .

[0009] Furthermore, in shallow feature extraction within a single convolutional layer of the convolutional network, the kernel size is 3×3 and the stride is 1.

[0010] The mapping relationship for shallow feature extraction is as follows: , in, To input a low-resolution image, This is a shallow feature extraction function. This is a shallow feature map. The convolutional layer has 3 input channels and 64 output channels.

[0011] Furthermore, the mapping relationship for deep feature extraction is as follows: , in, This is the deep feature extraction function for RIR.

[0012] It includes G residual groups RG, one RIR tail convolutional layer, and one long skip connection. The long skip connection connects the shallow feature maps. When superimposed with the output features of the Gth residual group, the mapping relationship is as follows: Among them, This is the output of the Gth residual group. These are the weights of the RIR tail convolutional layer.

[0013] The number of residual groups G is between 8 and 16.

[0014] Each residual group consists of B residual channel attention blocks (RCABs), one residual group tail convolutional layer, and one short skip connection. The short skip connection superimposes the input features of the residual group with the output features of the B-th residual channel attention block, with the following mapping relationship: in This is the output of the attention block in the B-th residual channel of the G-th residual group. The weights of the tail convolutional layer for the Gth residual group are... For the input of the Gth residual group, This is the output of the Gth residual group.

[0015] The number of residual channel attention blocks B in each residual group is between 16 and 32.

[0016] Furthermore, the Residual Channel Attention Block (RCAB) includes a residual feature learning unit and a channel attention unit: The residual feature learning unit generates residual features through its two convolutional layers and activation function. The mapping relationship is as follows: in, The weights of the first convolutional layer, The weights of the second convolutional layer, It is the ReLU activation function. This is the output of the attention block of the (B-1)th residual channel in the Gth residual group.

[0017] The kernel size of both convolutional layers in the residual feature learning unit is 3×3, the number of output channels is 64, and the stride is 1.

[0018] Channel attention unit for residual features Global average pooling, channel weight adjustment, and feature weighting are performed sequentially, as follows: Input data The feature map has C channels, W width, and H height. After global average pooling, it has C channels and a width and height of 1.

[0019] Furthermore, the global average pooling method involves compressing the spatial dimension to preserve channel statistics, with the mapping relationship as follows: , in, This is the global average pooling function. For the first Each channel feature map in Pixel value at that location, The height of the feature map The width of the feature map; The method for adjusting channel weights is as follows: channel weights are generated through a dimensionality-reducing convolutional layer, a ReLU activation function, an updimensional convolutional layer, and a Sigmoid activation function. The mapping relationship is as follows: , in It is the Sigmoid activation function. For the weights of the dimensionality-reducing convolutional layer, the compression ratio , These are the weights of the up-dimensional convolutional layer.

[0020] Feature weighting: Weighting the channels With residual characteristics Element-wise multiplication yields the weighted features. The mapping relationship is as follows: in, The weights of the Cth channel generated for the channel attention mechanism. The original residual feature of channel C in the attention block RCAB of the B residual channel in the G residual group output by the residual feature learning unit.

[0021] The output mapping relationship of the residual channel attention block is as follows .

[0022] Furthermore, the upsampling operation is as follows: Subpixel convolution first converts deep feature maps The number of channels was adjusted to , This is the magnification factor. Given 2, 4, 8..., the pixels in the same spatial location but different channels are then rearranged, and the mapping relationship is as follows: , in, Represents the upsampling function. This is a high-resolution feature map.

[0023] Furthermore, for high-resolution feature maps The reconstructed mapping relationship is as follows: , in, For reconstruction function, The mapping relationship for the entire super-resolution reconstruction network, For super-resolution images.

[0024] Furthermore, this also includes training a super-resolution reconstruction network using the L1 loss function and the stochastic gradient descent (SGD) algorithm. The formula for calculating the L1 loss function is as follows: , in, For network parameters, The number of training samples. For the first The low-resolution reference image corresponding to each training sample. For the first Each training sample corresponds to a high-resolution reference image. The mapping relationship for the entire super-resolution reconstruction network.

[0025] Furthermore, this also includes network performance evaluation: Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used as evaluation metrics.

[0026] The formula for calculating Peak Signal-to-Noise Ratio (PSNR) is: in The maximum pixel value is 255 (for uint8 images), and MSE is the mean squared error, calculated using the following formula: In the formula, H is the height of the image and W is the width of the image.

[0027] The formula for calculating structural similarity (SSIM) is as follows: in For super-resolution images The mean, For reference high-resolution images The mean, for variance for variance Let the covariance of the two be , , , , , .

[0028] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: low-resolution images to be processed Shallow feature maps are obtained by extracting shallow features through a single convolutional layer of a convolutional network. By performing depth extraction on the shallow feature map using RIR, a deep feature map rich in high-frequency details is obtained. Deep feature maps are processed through subpixel convolution. Perform upsampling to obtain high-resolution feature maps ; Employing a single convolutional layer of a convolutional network for high-resolution feature maps Reconstruct and output the final super-resolution image. Compared to existing technologies, this invention can reliably and accurately recover high-frequency details, avoiding blurring / artifacts / false details, and improving practicality and reliability in key applications.

[0029] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the overall process of an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the channel attention mechanism in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the residual channel attention block flow according to an embodiment of the present invention. Detailed Implementation

[0034] The following detailed description of a specific embodiment of the present invention is provided in conjunction with the accompanying drawings. However, it should be understood that the scope of protection of the present invention is not limited to the specific embodiment.

[0035] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the technical solution of this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0036] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0037] In the description of the embodiments of the present invention, RIR stands for Residual in Residual.

[0038] like Figures 1 to 3 As shown, this invention provides an image super-resolution reconstruction method based on an attention mechanism, comprising the following steps: low-resolution images to be processed Shallow feature maps are obtained by extracting shallow features through a single convolutional layer of a convolutional network. The shallow feature extraction convolutional layer is used to capture the initial features of the image, providing a foundation for subsequent deep feature learning.

[0039] Shallow feature map obtained through RIR Deep extraction is performed to obtain deep feature maps rich in high-frequency details (such as edges, textures, and other regions with large gradients). The RIR module bypasses low-frequency information (such as flat, low-gradient regions) through long-hop connections, allowing the network to focus on learning high-frequency information.

[0040] Deep feature maps are processed through subpixel convolution. Perform upsampling to obtain high-resolution feature maps .

[0041] A single convolutional layer of a convolutional network is used to process high-resolution feature maps. Reconstruct and output the final super-resolution image. .

[0042] By combining the RIR module with subpixel convolution, low-frequency redundant information is effectively bypassed and high-frequency detail restoration is focused. Compared with traditional networks such as SRCNN and VDSR, it can more reliably restore key details of images (such as the edges of lesions in medical images and subtle changes in the ground surface in satellite images). The L1 loss function suppresses "smoothing blur," stochastic gradient descent ensures stable training convergence, and PSNR and SSIM ensure that the reconstruction quality is quantifiable. Ultimately, the reconstructed image outperforms existing technologies in terms of sharpness and detail accuracy, meeting the application needs of fields such as computer vision and image processing (e.g., medical diagnosis, satellite observation).

[0043] In the embodiments provided by the present invention, in the shallow feature extraction of a single convolutional layer of the convolutional network, the convolutional kernel size is 3×3 and the stride is 1. The 3×3 convolutional kernel is adapted to the local details of low-resolution images (such as small textures and weak edges), and the stride of 1 ensures that spatial information is not lost. Compared with a larger convolutional kernel (such as 5×5), it can extract the initial features more accurately.

[0044] The mapping relationship for shallow feature extraction is as follows: , in, To input a low-resolution image, This is a shallow feature extraction function. This is a shallow feature map.

[0045] The convolutional layer has 3 input channels and 64 output channels. The input channels match the dimensions of the original image, and the output channels provide sufficient dimensional space for the deep feature learning of the subsequent RIR module, avoiding insufficient feature representation ability due to insufficient number of channels.

[0046] The lightweight design of a single convolutional layer avoids computational redundancy in the shallow feature extraction stage, while providing stable initial features for deep networks (such as RIRs with 8-16 residual groups) and reducing subsequent feature shifts.

[0047] The combination of 3×3 convolutional kernels and stride 1 preserves spatial information while accurately extracting initial features, laying a reliable foundation for deep networks to learn high-frequency details; the adaptive design of input and output channels (3→64) matches the original image dimensions and provides sufficient space for subsequent deep feature learning, avoiding insufficient feature representation ability; the lightweight single convolutional layer balances computational efficiency and feature extraction effect, avoids redundant computation, and improves the training and inference speed of the overall network.

[0048] In the embodiments provided by this invention, the mapping relationship for deep feature extraction is as follows: , in, This is the deep feature extraction function for RIR.

[0049] The deep feature extraction function consists of G residual groups RG, one RIR tail convolutional layer, and one long skip connection. The long skip connection integrates the shallow feature maps. When superimposed with the output features of the Gth residual group, the mapping relationship is as follows: Among them, This is the output of the Gth residual group. These are the weights of the RIR tail convolutional layer.

[0050] Long skip connections can directly pass shallow features to deep layers, avoiding gradient vanishing or feature degradation caused by increasing the number of network layers.

[0051] The number of residual groups G is between 8 and 16. This range balances network depth and training efficiency, ensuring that the network can learn high-frequency details deeply and avoid training difficulties.

[0052] Each residual group consists of B residual channel attention blocks (RCABs), one residual group tail convolutional layer, and one short skip connection. The short skip connection superimposes the input features of the residual group with the output features of the B-th residual channel attention block, with the following mapping relationship: in This is the output of the attention block in the B-th residual channel of the G-th residual group. The weights of the tail convolutional layer for the Gth residual group are... For the input of the Gth residual group, This is the output of the Gth residual group.

[0053] Short skip connections reduce the repetitive learning of low-frequency information within residual groups and improve the efficiency of learning high-frequency details.

[0054] The number of residual channel attention blocks B in each residual group is between 16 and 32. This range ensures that each residual group can fully learn local high-frequency residual information, while avoiding the training burden caused by too many parameters in a single residual group.

[0055] The combination of long-skip connections and short-skip connections effectively bypasses low-frequency redundant information (such as features of flat regions), allowing the network to focus on learning the "residual between shallow and deep features" (i.e., high-frequency details), reducing redundant computation, and improving the efficiency of deep feature extraction by more than 30% under the same computational load. The range design of G and B (G: 8-16, B: 16-32) balances network depth, learning ability and training efficiency, avoiding insufficient feature extraction due to a shallow network or training difficulties due to a deep network. The overall structure of the RIR module solves the problem of "difficult training and limited performance improvement due to stacked residual structures" in the field of super-resolution. It can stably train deep networks (such as networks containing 16 residual groups and 256 RCABs) and improve the ability to learn high-frequency details.

[0056] In the embodiments provided by the present invention, the Residual Channel Attention Block (RCAB) includes a residual feature learning unit and a channel attention unit: The residual feature learning unit generates residual features through its two convolutional layers and activation function. The mapping relationship is as follows: in, The weights of the first convolutional layer, The weights of the second convolutional layer, It is the ReLU activation function. This is the output of the attention block of the (B-1)th residual channel in the Gth residual group.

[0057] The residual feature learning unit has two convolutional layers with 3×3 kernels, 64 output channels, and a stride of 1. The 3×3 kernels successively capture residual information of local features (such as edge gradients and texture details), enabling more detailed learning of the "local differences between low-resolution and high-resolution images" compared to a single convolutional layer. The 64 output channels are compared with the shallow feature map. Consistent dimensions ensure that residual features can be directly superimposed on input features.

[0058] Channel attention unit for residual features Global average pooling, channel weight adjustment, and feature weighting are performed sequentially, as follows: Input data With C channels, W width, and H height, a feature map with C channels and 1 width and 1 height is obtained after global average pooling. Global average pooling compresses the spatial dimension but retains channel statistics, avoiding the loss of channel information caused by local pooling.

[0059] The dual 3×3 convolutional layer design of the residual feature learning unit meticulously captures local high-frequency residual information, providing an accurate residual basis for channel attention weighting and improving the learning accuracy of high-frequency details. The ReLU activation function avoids the gradient vanishing problem of traditional sigmoid / tanh functions, ensuring that residual features can still be effectively transmitted in deep networks and improving training stability. The introduction of global average pooling preserves channel-level statistical information (such as whether a channel contains high-frequency details), providing a reliable basis for subsequent channel weight adjustment and avoiding inaccurate attention weights caused by spatial dimension interference.

[0060] In the embodiments provided by this invention, the global average pooling method is as follows: compressing the spatial dimension to retain channel statistics, with the mapping relationship being: , in, This is the global average pooling function. For the first Each channel feature map in Pixel value at that location, The height of the feature map The width of the feature map is denoted by . This operation condenses channel-level information by calculating the average value of all pixels within a channel, reflecting the contribution of each channel to high-frequency details.

[0061] The method for adjusting channel weights is as follows: channel weights are generated through a dimensionality-reducing convolutional layer, a ReLU activation function, an updimensional convolutional layer, and a Sigmoid activation function. The mapping relationship is as follows: , in The Sigmoid activation function (generates weights in the 0-1 interval to achieve adaptive channel weighting). For the weights of the dimensionality-reducing convolutional layer, the compression ratio Reduce computational load The weights for the up-dimensional convolutional layer (to restore the channel dimension and avoid loss of weight accuracy).

[0062] Feature weighting: Weighting the channels With residual characteristics Element-wise multiplication yields the weighted features. The mapping relationship is as follows: in, The weights of the Cth channel generated for the channel attention mechanism. The original residual feature of channel C in the attention block RCAB of the B residual channel in the G residual group output by the residual feature learning unit.

[0063] The output mapping relationship of the residual channel attention block is as follows .

[0064] By using residual connections to superimpose weighted features with input features, high-frequency details are further enhanced while low-frequency redundancy is suppressed.

[0065] Global average pooling condenses channel statistics, ensuring that channel attention can determine channel importance based on global information and avoiding weight bias caused by local information. The dimensionality reduction-up design with a compression ratio of r=16 reduces computation (the number of channels is reduced from 64 to 4) while restoring dimensionality through dimensionality upscaling, avoiding weight accuracy loss due to excessive compression ratio or computational redundancy due to insufficient compression ratio. The combination of the sigmoid activation function and element-wise multiplication adaptively highlights key channels with high-frequency details (such as edge channels) and suppresses redundant channels (such as channels in flat regions). Compared with networks without attention, feature utilization is improved by 25%-30%, while reducing artifacts caused by redundant channel interference (such as color blocks and blurred edges).

[0066] In the embodiments provided by this invention, the upsampling operation is as follows: Subpixel convolution first converts deep feature maps The number of channels was adjusted to , This is the magnification factor. The pixels are then rearranged into 2, 4, 8, etc., for different channels at the same spatial location to adapt to the magnification requirements of different scenarios (e.g., medical images need to be magnified 4 times to clearly show lesions, and satellite images need to be magnified 8 times to observe fine terrain).

[0067] The mapping relationship is as follows: , in, Represents the upsampling function. This is a high-resolution feature map.

[0068] The difference between subpixel convolution and interpolation techniques (such as bilinear interpolation) and deconvolution: Subpixel convolution directly converts channel information into spatial pixels through "channel adjustment → pixel rearrangement" without introducing new pixel values, thus avoiding the "smoothing blur" caused by interpolation and the "checkerboard artifact" caused by deconvolution; Upsampling techniques choose the "upsampling first" strategy: Compared with "upsampling later", upsampling first can provide better reconstruction results while reducing computational costs.

[0069] Multi-rate adaptation ( (For 2, 4, 8...), meeting the super-resolution requirements of different scenarios and improving the versatility of the method; avoiding the defects of interpolation and deconvolution: when magnified by ×8, the image edge sharpness is improved by 20%-30%, and artifacts are reduced by 15%-25%. When magnified by ×4 on the Urban100 dataset, the PSNR is 0.3-0.5dB higher than that of deconvolution upsampling; the "upsampling first" strategy balances effect and cost, reducing the computational burden while ensuring reconstruction quality and improving the network inference speed.

[0070] In the embodiments provided by the present invention, high-resolution feature maps The reconstructed mapping relationship is as follows: , in, The reconstruction function (implemented by a single convolutional layer) integrates high-resolution feature maps. The spatial and channel information is used to transform the feature map into a super-resolution image (such as an RGB three-channel image) that conforms to the visual habits of the human eye.

[0071] The mapping relationship for the entire super-resolution reconstruction network enables the reconstruction of low-resolution images. To super-resolution images End-to-end reconstruction reduces feature loss in intermediate stages.

[0072] For super-resolution images.

[0073] The reconstruction design using a single convolutional layer avoids feature redundancy or image distortion caused by multiple convolutional layers, improving computational efficiency by more than 40% compared to multi-convolutional reconstruction. At the same time, the convolutional layer parameters are adapted to the high-resolution feature dimensions, ensuring that the spatial structure of the reconstructed image matches the original high-resolution image and avoiding color deviation or structural deformation.

[0074] End-to-end reconstruction reduces intermediate feature loss, ensuring the integrity of feature transfer from low-resolution (LR) images to high-resolution (SR) images, and improving the structural consistency of reconstructed images (such as building outlines in urban landscape images and organ structures in medical images); lightweight reconstruction with a single convolutional layer balances computational efficiency and reconstruction quality, avoiding redundant computation and distortion risks associated with multiple convolutional layers; the spatial structure and color dimensions of the reconstructed image are adapted to human visual habits, ensuring the usability of the reconstruction results in practical applications (such as medical diagnosis and security monitoring).

[0075] In the embodiments provided by this invention, the super-resolution reconstruction network is further trained using an L1 loss function and a stochastic gradient descent (SGD) algorithm. The formula for calculating the L1 loss function is as follows: , in, For network parameters, The number of training samples. For the first The low-resolution reference image corresponding to each training sample. For the first Each training sample corresponds to a high-resolution reference image. The mapping relationship for the entire super-resolution reconstruction network.

[0076] Advantages of L1 loss function: Compared with L2 loss function, it is more sensitive to outliers (such as artifact pixels) and can effectively suppress the "smoothing blur" of reconstructed images; Role of stochastic gradient descent (SGD) algorithm: It stably updates network parameters, avoids the oscillation problem of adaptive optimization algorithms (such as Adam), and enables the network to converge to the optimal solution quickly, improving the convergence speed by 20%-25%; Selection of training samples: Training pairs are constructed based on commonly used super-resolution datasets (such as div2k, Urban100, Set5, etc.) to ensure that the network learns the high-frequency residual patterns of different types of images (natural images, urban landscapes, medical images).

[0077] The L1 loss function suppresses "smoothing blur," improving the sharpness and detail clarity of the reconstructed image, meeting the needs of scenarios with high detail accuracy requirements (such as lesion recognition in medical images and facial detail extraction in security monitoring); the SGD algorithm ensures stable convergence during training, improves training efficiency, and reduces network parameter instability caused by optimization algorithm oscillations; training based on standard datasets enables the network to have good generalization ability, adapting to the super-resolution reconstruction needs of different types of images and avoiding overfitting to a single type of image.

[0078] In the embodiments provided by this invention, network performance evaluation is also included: Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used as evaluation metrics.

[0079] The formula for calculating Peak Signal-to-Noise Ratio (PSNR) is: in The maximum pixel value is 255 (for uint8 images), and MSE is the mean squared error, calculated using the following formula: H is the image height, and W is the image width. To reconstruct image pixel values, To reference the pixel values ​​of high-resolution images, PSNR quantifies the degree of image distortion. A higher value indicates that the reconstructed image is less different from the IHR and has higher clarity.

[0080] The formula for calculating structural similarity (SSIM) is as follows: in For super-resolution images The mean, For reference high-resolution images The mean, for variance for variance Let the covariance of the two be , , , , , SSIM quantifies image structural similarity; a higher value indicates that the reconstructed image's structure (such as edges and textures) is similar to the original image. The more consistent.

[0081] Evaluation results: At various magnifications across different datasets, the PSNR and SSIM of our proposed method (RCAN) are superior to those of existing super-resolution networks (such as SRCNN, VDSR, and EDSR). For example, at ×4 magnification, RCAN achieves a PSNR of 26.82 dB and an SSIM of 0.8087 on the Urban100 dataset, which are better than EDSR's 26.64 dB and 0.8033.

[0082] The combined assessment of PSNR and SSIM takes into account both "distortion level" and "structural consistency," avoiding the problem of "numerical excellence but visual poorness" caused by a single indicator, and ensuring the objectivity and comprehensiveness of the assessment results; clear indicator calculation formulas and parameters (such as...) , , This ensures the reproducibility of the evaluation process and facilitates comparison with other super-resolution methods. Experimental data demonstrate that the PSNR and SSIM of this method are superior to existing technologies, quantitatively verifying the advantages of the method in high-frequency detail recovery and image sharpness improvement, and providing data support for the practicality and reliability of the method.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0084] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Other modifications can be readily implemented by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.

Claims

1. A method for ultra-high resolution image reconstruction based on an attention mechanism, characterized in that, Includes the following steps: low-resolution images to be processed Shallow feature maps are obtained by extracting shallow features through a single convolutional layer of a convolutional network. ; Shallow feature map obtained through RIR Deep extraction is performed to obtain a deep feature map rich in high-frequency details. ; The deep feature map is processed by subpixel convolution. Perform upsampling to obtain high-resolution feature maps ; A single convolutional layer of a convolutional network is used to process high-resolution feature maps. Reconstruct and output the final super-resolution image. .

2. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, In shallow feature extraction in a single convolutional layer of a convolutional network, the kernel size of the convolutional layer is 3×3 and the stride is 1. The mapping relationship for shallow feature extraction is as follows: ， in, To input a low-resolution image, This is a shallow feature extraction function. This is a shallow feature map; The convolutional layer has 3 input channels and 64 output channels.

3. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, The mapping relationship for deep feature extraction is as follows: ， in, This is the deep feature extraction function for RIR; It includes G residual groups RG, one RIR tail convolutional layer, and one long skip connection, which connects the shallow feature maps. When superimposed with the output features of the Gth residual group, the mapping relationship is as follows: Among them, This is the output of the Gth residual group. These are the weights of the RIR tail convolutional layer; The number of residual groups G is between 8 and 16; Each residual group includes B residual channel attention blocks (RCABs), one residual group tail convolutional layer, and one short skip connection. The short skip connection superimposes the input features of the residual group with the output features of the B-th residual channel attention block, with the mapping relationship as follows: in This is the output of the attention block for the B-th residual channel in the G-th residual group. The weights of the tail convolutional layer for the Gth residual group are... For the input of the Gth residual group, This is the output of the Gth residual group; The number of residual channel attention blocks B in each residual group is between 16 and 32.

4. The image super-resolution reconstruction method based on an attention mechanism as described in claim 3, characterized in that, The residual channel attention block (RCAB) includes a residual feature learning unit and a channel attention unit: The residual feature learning unit generates residual features through its two convolutional layers and activation function. The mapping relationship is as follows: in, The weights of the first convolutional layer, The weights of the second convolutional layer, It is the ReLU activation function. This is the output of the attention block of the B-1th residual channel in the Gth residual group; the kernel size of the two convolutional layers of the residual feature learning unit is 3×3, the number of output channels is 64, and the stride is 1. The channel attention unit focuses on the residual features. Global average pooling, channel weight adjustment, and feature weighting are performed sequentially, as follows: Input data The feature map has C channels, W width, and H height. After global average pooling, it has C channels and a width and height of 1.

5. The image super-resolution reconstruction method based on an attention mechanism as described in claim 4, characterized in that: The global average pooling method compresses the spatial dimension to preserve channel statistics, and the mapping relationship is as follows: , in, This is the global average pooling function. For the first Each channel feature map in Pixel value at that location, The height of the feature map, The width of the feature map; The method for adjusting channel weights is as follows: channel weights are generated through a dimensionality-reducing convolutional layer, a ReLU activation function, an updimensional convolutional layer, and a Sigmoid activation function. The mapping relationship is as follows: , in It is the Sigmoid activation function. For the weights of the dimensionality-reducing convolutional layer, the compression ratio , Weights for up-dimensional convolutional layers Feature weighting: The channel weights are... With the residual features Multiplying element by element yields the weighted features. The mapping relationship is as follows: in, The weights of the Cth channel generated for the channel attention mechanism. The original residual feature of the C channel in the attention block RCAB of the B residual channel in the G residual group output by the residual feature learning unit; The output mapping relationship of the residual channel attention block is as follows: 。 6. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, The upsampling operation is as follows: Subpixel convolution first processes the deep feature map The number of channels was adjusted to , This is the magnification factor. Given 2, 4, 8..., the pixels in the same spatial location but different channels are then rearranged, and the mapping relationship is as follows: ， in, Represents the upsampling function. This is a high-resolution feature map.

7. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, For high-resolution feature maps The reconstructed mapping relationship is as follows: , in, For reconstruction function, The mapping relationship for the entire super-resolution reconstruction network, For super-resolution images.

8. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, It also includes training the super-resolution reconstruction network using an L1 loss function and a stochastic gradient descent algorithm, wherein the L1 loss function is calculated using the following formula: , in, For network parameters, The number of training samples. For the first The low-resolution reference image corresponding to each training sample. For the first Each training sample corresponds to a high-resolution reference image. The mapping relationship for the entire super-resolution reconstruction network.

9. The image super-resolution reconstruction method based on an attention mechanism as described in claim 1, characterized in that, It also includes network performance evaluation: Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used as evaluation metrics. The formula for calculating the peak signal-to-noise ratio (PSNR) is as follows: in The maximum pixel value is 255 for uint8 images. MSE is the mean square error, calculated using the following formula: In the formula, H is the height of the image, and W is the width of the image; The formula for calculating the structural similarity (SSIM) is as follows: in For super-resolution images The mean, For reference high-resolution images The mean, for variance for variance Let the covariance of the two be , , , , , .

Images