Pixel-level image super-resolution reconstruction method, system, device and medium

Abstract

The present invention relates to the field of image reconstruction technology, and specifically to a pixel-level image super-resolution reconstruction method, system, device and medium. To solve the technical problem of poor image reconstruction effect in the prior art, the present invention first performs convolution processing and nonlinear processing on a low-resolution image to be processed to obtain a shallow feature map. Then, the shallow feature map is subjected to several global-local feature extraction and fusion processes to obtain a global-local fusion feature map. After the global-local fusion feature map is subjected to convolution processing, it is fused with the shallow feature map to obtain a deep feature map. Finally, the deep feature map is subjected to sub-pixel-based image reconstruction processing to obtain a high-resolution reconstructed image. The present invention is used in the field of image super-resolution reconstruction, and has good image detail recovery and high image reconstruction quality.

Description

Technical Field

[0001] The present invention relates to the field of image reconstruction technology, and in particular to a pixel-level image super-resolution reconstruction method, system, device and medium. Background Art

[0002] High-resolution images have important applications in numerous fields, including medical imaging, remote sensing, security monitoring, and autonomous driving. However, acquiring high-resolution images directly from sensors often faces challenges, constrained by factors such as the sensor's physical size, pixel density, optical aberrations, motion blur, transmission bandwidth, storage limitations, and cost. For example, the smaller size of sensors commonly used in miniaturized devices reduces the pixel's ability to capture light and lowers the signal-to-noise ratio. Increasing pixel density theoretically improves resolution but exacerbates noise issues. Inherent imperfections in optical systems and motion blur in dynamic scenes further limit image quality. Furthermore, high-resolution data places higher demands on transmission and storage. These hardware limitations make improving image resolution through software algorithms an attractive solution.

[0003] Image super-resolution reconstruction technology aims to reconstruct high-resolution images from low-resolution observations, providing a cost-effective approach to addressing these hardware limitations. Currently, image super-resolution reconstruction methods include regularization, convolutional neural networks, and Transformer methods. However, these existing methods lack the ability to model long-range dependencies and suffer from high global computational costs, leading to an imbalance between local and global information. This in turn limits the enhancement of high-frequency components such as textures and edges, and insufficiently restores high-frequency details, resulting in a lack of realism and poor image reconstruction. Summary of the Invention

[0004] The purpose of the present invention is to provide a pixel-level image super-resolution reconstruction method, system, device and medium.

[0005] The technical solutions of the present invention are as follows:

[0006] A pixel-level image super-resolution reconstruction method includes the following operations:

[0007] S1, the low-resolution image to be processed is processed by convolution and nonlinearity to obtain a shallow feature map;

[0008] S2: The shallow feature map undergoes several rounds of global and local feature extraction and fusion processing to obtain a global and local fusion feature map; the global and local fusion feature map is then fused with the shallow feature map to obtain a deep feature map.

[0009] The operation of global-local feature extraction and fusion processing is as follows: the input feature map is processed by convolution, batch normalization and activation function to obtain a nonlinear feature map; the nonlinear feature map is processed by pixel-level non-local self-attention to obtain an attention feature map; the attention feature map is fused with the input feature map to obtain an attention residual feature map; the attention residual feature map is processed by layer normalization to obtain an attention normalized feature map; the attention normalized feature map is processed by multi-branch dual-path fusion to obtain a cross-branch multi-scale fusion feature; the cross-branch multi-scale fusion feature is fused with the attention normalized feature map to obtain an output feature map for performing global-local feature extraction and fusion processing or convolution processing;

[0010] The pixel-level non-local self-attention processing operation is as follows: each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on pixel similarity, and then fused with its own sub-feature map to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is sequentially processed by feature enhancement, feature information interaction and nonlinear feature extraction, and then spliced ​​to obtain the attention feature map;

[0011] S3. The deep feature map is processed based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

[0012] In S2, the operation of multi-head attention processing based on pixel similarity is as follows: the pixel information in the neighborhood range of each pixel point in the sub-feature map forms its own neighborhood pixel block; select a pixel point from the sub-feature map as the reference pixel point, and the corresponding neighborhood pixel block as the reference neighborhood pixel block; obtain the similarity between the reference neighborhood pixel block and each neighborhood pixel block in the sub-feature map under the current attention head, as the similarity between the current head reference pixel point and each pixel point; the similarity between the current head reference pixel point and each pixel point is multiplied with the corresponding pixel point embedding vector, and then summed to obtain the current head attention feature; all head attention features are weighted summed and regularized to obtain the initial attention feature map of the sub-feature map, which is used to perform the operation of fusion with the sub-feature map.

[0013] The similarity between the reference neighborhood pixel block and the neighborhood pixel block is calculated using the following formula:

[0014] ,

[0015] For the i Reference neighborhood pixel blocks under attention heads Hedi p Neighborhood pixel blocks The similarity between 、 They are the benchmark neighborhood pixel blocks Embedding vector, neighborhood pixel block The embedding vector of 、 are the feature dimensions and the total number of attention heads, respectively. is the standardization function.

[0016] The operation of cross-branch multi-scale fusion features in S2 is as follows: the attention normalized feature map is subjected to feature extraction with different expansion rates to obtain several attention branch feature maps; the current attention branch feature map is subjected to average pooling and maximum pooling respectively to obtain the current branch average pooling feature map and the current branch maximum pooling feature map; the current branch average pooling feature map and the current branch maximum pooling feature map are subjected to weight generation processing based on parallel convolution to obtain the current first path weight matrix; the current branch average pooling feature map and the current branch maximum pooling feature map are subjected to weight generation processing based on fused convolution to obtain the current second path weight matrix; the current first path weight matrix and the current second path weight matrix are multiplied element by element to obtain the current comprehensive weight matrix; each attention branch feature map is convolved with its own comprehensive weight matrix after weighted processing to obtain cross-branch multi-scale fusion features.

[0017] The second path weight matrix is ​​calculated by the following formula:

[0018] ,

[0019] For the m The second path weight matrix of the attention branch feature map, 、 Respectively m The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, 、 Respectively n The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, For splicing operations, 、 The first convolution of the second path and the second convolution of the second path, N is the total number of attention branch feature maps.

[0020] The feature enhancement operation in S2 is as follows: after the multi-head attention sub-feature map is layer-normalized, fully connected feedforward network processed and regularized, it is added element-by-element with the multi-head attention sub-feature map to obtain the multi-head attention enhanced sub-feature map, which is used to perform feature information interaction operations.

[0021] The sub-pixel based image reconstruction processing in S3 can be achieved through convolution, sub-pixel convolution, activation function processing and convolution processing.

[0022] A pixel-level image super-resolution reconstruction system, used to implement the above-mentioned pixel-level image super-resolution reconstruction method, comprising:

[0023] Shallow feature map generation module, which is used to obtain shallow feature maps by convolution and nonlinear processing of the low-resolution image to be processed;

[0024] The deep feature map generation module is used to extract and fuse the shallow feature map several times to obtain the global local fusion feature map; the global local fusion feature map is fused with the shallow feature map after convolution processing to obtain the deep feature map; the operation of the global local feature extraction and fusion processing is as follows: the input feature map is processed by convolution, batch normalization and activation function to obtain a nonlinear feature map; the nonlinear feature map is processed by pixel-level non-local self-attention to obtain an attention feature map; the attention feature map is fused with the input feature map to obtain an attention residual feature map; the attention residual feature map is processed by layer normalization to obtain an attention normalized feature map; the attention normalization feature map is processed by layer normalization to obtain an attention residual feature map. The feature map is fused through multi-branch dual-path processing to obtain cross-branch multi-scale fusion features; the cross-branch multi-scale fusion features are fused with the attention normalized feature map to obtain the output feature map, which is used to perform global local feature extraction fusion processing or convolution processing; the operation of pixel-level non-local self-attention processing is as follows: each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on pixel similarity, and then fused with its own sub-feature map to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is sequentially subjected to feature enhancement, feature information interaction and nonlinear feature extraction, and then spliced ​​to obtain the attention feature map;

[0025] The high-resolution reconstructed image generation module is used to process the deep feature map based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

[0026] A pixel-level image super-resolution reconstruction device comprises a processor and a memory, wherein the processor implements the above-mentioned pixel-level image super-resolution reconstruction method when executing a computer program stored in the memory.

[0027] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the above-mentioned pixel-level image super-resolution reconstruction method is implemented.

[0028] The beneficial effects of the present invention are:

[0029] The present invention provides a pixel-level image super-resolution reconstruction method. First, a low-resolution image to be processed is subjected to convolution processing and nonlinear processing to capture low-level spatial information of the low-resolution image to be processed, thereby obtaining a shallow feature map. Then, the shallow feature map is subjected to several global-local feature extraction and fusion processes to fully capture long-range dependent and non-local feature information, enhance the multi-scale feature representation capability, and realize efficient fusion of information of different scales to obtain a global-local fusion feature map. After convolution processing, the global-local fusion feature map is fused with the shallow feature map to obtain a deep feature map. Finally, the deep feature map is subjected to sub-pixel-based image reconstruction processing to refine the reconstructed image, realize reconstruction of the low-resolution image, and obtain a high-resolution reconstructed image. This method is used in the field of image super-resolution reconstruction, and has good image detail recovery, high image resolution, and good image reconstruction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By reading the detailed description of the preferred embodiment below, the solutions and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0031] In the attached figure:

[0032] Figure 1 This is an image reconstruction effect diagram of the method in this embodiment. DETAILED DESCRIPTION

[0033] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0034] This embodiment provides a pixel-level image super-resolution reconstruction method, including the following operations:

[0035] S1, the low-resolution image to be processed is processed by convolution and nonlinearity to obtain a shallow feature map;

[0036] S2: The shallow feature map undergoes several rounds of global and local feature extraction and fusion processing to obtain a global and local fusion feature map; the global and local fusion feature map is then fused with the shallow feature map to obtain a deep feature map.

[0037] S3. The deep feature map is processed based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

[0038] The specific steps are detailed below.

[0039] S1. The low-resolution image to be processed is subjected to convolution and nonlinear processing to obtain a shallow feature map.

[0040] The low-resolution image to be processed is subjected to 9×9 convolution and nonlinear processing (which can be achieved through the PReLU activation function) to capture the low-level spatial information of the low-resolution image to be processed, which is used as the initial representation to enhance the resolution of the input image and obtain a shallow feature map.

[0041] S2. The shallow feature map is subjected to several times of global and local feature extraction and fusion processing to obtain a global and local fusion feature map; the global and local fusion feature map is subjected to convolution processing and fused with the shallow feature map to obtain a deep feature map.

[0042] The shallow feature map undergoes several rounds of global and local feature extraction and fusion processing to fully capture long-range dependent and non-local feature information, enhance the multi-scale feature representation capability, and achieve efficient fusion of information at different scales, which can better reconstruct local high-frequency information in the image and obtain a global-local fusion feature map; after convolution processing, the global-local fusion feature map is fused with the shallow feature map to obtain a deep feature map.

[0043] First, the shallow feature map is subjected to several times of global-local feature extraction and fusion processing to effectively extract and fuse global and local features to obtain a global-local fusion feature map.

[0044] The operation steps of global and local feature extraction and fusion processing are as follows.

[0045] Step 1: The input feature map (shallow feature map or the output of the previous global-local feature extraction fusion process) is processed by convolution, batch normalization and activation function (which can be achieved through the PReLU activation function) to further enhance the feature representation and obtain a nonlinear feature map.

[0046] Step 2: The nonlinear feature map is processed by pixel-level non-local self-attention to capture the long-range dependencies and non-local information in the image to obtain the attention feature map; the attention feature map is fused with the input feature map (which can be achieved by element-by-element addition) to obtain the attention residual feature map.

[0047] The specific operation of pixel-level non-local self-attention processing is as follows: the nonlinear feature map is divided into several non-overlapping sub-feature maps. Each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on the similarity between pixels, and then fused with its own sub-feature map (which can be achieved by element-by-element addition) to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is processed by feature enhancement, feature information interaction and nonlinear feature extraction in turn, and then spliced ​​to obtain an attention feature map.

[0048] For a single sub-feature map, in the process of multi-head attention processing based on pixel similarity, the specific details are as follows: the pixel information in the neighborhood range of each pixel point in the sub-feature map forms its own neighborhood pixel block; a pixel point is selected from the sub-feature map as the reference pixel point, and the corresponding neighborhood pixel block is used as the reference neighborhood pixel block; taking the current head attention as an example, the similarity between the reference neighborhood pixel block and each neighborhood pixel block (including the reference neighborhood pixel block) in the sub-feature map under the current attention head is obtained (the similarity between the reference neighborhood pixel block and itself is 1), which is used as the similarity between the current head reference pixel point and each pixel point; the similarity between the current head reference pixel point and each pixel point is multiplied with the corresponding pixel point embedding vector, and then summed to obtain the current head attention feature; after weighted summation of all head attention features, regularization is performed to obtain the initial attention feature map of the sub-feature map, which is used to perform the fusion operation with the corresponding sub-feature map.

[0049] The similarity between the reference neighborhood pixel block and the neighborhood pixel block is calculated using the following formula:

[0050] ,

[0051] For the i Reference neighborhood pixel blocks under attention heads Hedi p Neighborhood pixel blocks The similarity between 、 They are the benchmark neighborhood pixel blocks Embedding vector, neighborhood pixel block The embedding vector of 、 are the feature dimensions and the total number of attention heads, respectively. is the standardization function.

[0052] The embedding vector of the neighborhood pixel block is obtained by performing depth convolution, point-by-point convolution, activation function processing (which can be achieved through LeakyReLU activation function) and flattening on the pixel information in the neighborhood of the pixel point.

[0053] The pixel embedding vector is obtained by performing point-by-point convolution, activation function processing (which can be achieved through the ReLU activation function) and flattening of the pixel information.

[0054] The specific operation of feature enhancement is as follows: after the multi-head attention sub-feature map is layer-normalized, fully connected feedforward network processed and regularized, it is added element-by-element with the multi-head attention sub-feature map to obtain the multi-head attention enhanced sub-feature map, which is used to perform feature information interaction operations.

[0055] The specific operation of feature information interaction is as follows: after the multi-head attention enhanced sub-feature map is layer-normalized and processed with multi-head self-attention within the window (preferably implemented through the SW-MSA network), it is element-wise added to the multi-head attention enhanced sub-feature map to obtain a sub-feature interaction map for performing nonlinear feature extraction operations.

[0056] The specific operation of nonlinear feature extraction is as follows: after the sub-feature interaction map is processed by layer normalization and a fully connected feedforward network, it is added element by element with the sub-feature interaction map to obtain an enhanced sub-feature map, which is used to perform the splicing operation with other enhanced sub-feature maps.

[0057] Step 3. The attention residual feature map is layer-normalized to obtain the attention normalized feature map; the attention normalized feature map is fused through multi-branch dual-path to capture contextual information at different scales, solving the inherent scale change challenge in complex image processing tasks, and achieving information complementarity through dual paths to obtain cross-branch multi-scale fusion features; the cross-branch multi-scale fusion features are fused with the attention normalized feature map (which can be achieved by element-by-element addition) to obtain the output feature map, which is used to perform the next global-local feature extraction fusion process or convolution process.

[0058] The operation steps of the above cross-branch multi-scale fusion features are as follows.

[0059] Step a: The attention normalized feature map is subjected to feature extraction at different expansion rates (which can be achieved through dilated convolution processing and ReLU activation function processing at different expansion rates) to obtain several attention branch feature maps.

[0060] Step b: Taking the current attention branch feature map among several attention branch feature maps as an example, the current attention branch feature map is subjected to average pooling and maximum pooling, respectively, to obtain the current branch average pooling feature map and the current branch maximum pooling feature map. The current branch average pooling feature map and the current branch maximum pooling feature map are processed by weight generation based on parallel convolution to emphasize the most informative features and suppress irrelevant noise, thereby obtaining the current first path weight matrix; the current branch average pooling feature map and the current branch maximum pooling feature map are processed by weight generation based on fused convolution to control the contribution of each branch in the aggregation process, thereby obtaining the current second path weight matrix; the current first path weight matrix and the current second path weight matrix are multiplied element-by-element to obtain the current comprehensive weight matrix.

[0061] Each attention branch feature graph respectively performs the operations of obtaining the first path weight matrix and the second path weight matrix, and multiplying the first path weight matrix and the second path weight matrix to obtain the comprehensive weight matrix of each attention branch feature graph.

[0062] The first path weight matrix or the weight generation process based on parallel convolution is calculated or implemented by the following formula:

[0063] ,

[0064] For the m The first path weight matrix of the attention branch feature map, 、 Respectively m The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, 、 The first convolution of the first path and the second convolution of the first path, the convolution scale is 1×1, is the Sigmoid activation function.

[0065] The second path weight matrix or the weight generation process based on the fused convolution is calculated or implemented by the following formula:

[0066] ,

[0067] For the m The second path weight matrix of the attention branch feature map, 、 Respectively m The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, 、 Respectively n The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, For splicing operations, 、 is the first convolution and the second convolution, the convolution scale is 1×1, N is the total number of attention branch feature maps.

[0068] Step c: Each attention branch feature map is weighted and convolved with its own comprehensive weight matrix to obtain cross-branch multi-scale fusion features.

[0069] Next, the global-local fusion feature map is convolved and fused with the shallow feature map (which can be achieved by element-by-element addition) to obtain the deep feature map.

[0070] S3. The deep feature map is processed based on sub-pixel image reconstruction to refine the reconstructed image and achieve reconstruction of the low-resolution image to obtain a high-resolution reconstructed image. The result is shown in Figure 1, the image details are well restored and the image reconstruction quality is high. The sub-pixel based image reconstruction processing can be achieved through convolution, sub-pixel convolution, activation function processing (which can be achieved through PReLU activation function) and convolution processing.

[0071] The image reconstruction method of this embodiment can also establish a model corresponding to steps S1, S2 and S3, use a training set formed by several high-resolution image and low-resolution image pairs to train the model, use K rounds of training, use a loss function to iteratively optimize the model, observe the convergence of the model loss function, and when the model converges to the optimal state, obtain an image reconstruction training model; put the low-resolution image to be processed into the image reconstruction training model for processing to obtain a high-resolution reconstructed image.

[0072] In the training set, low-resolution images are obtained by performing bicubic interpolation, random horizontal flipping, 90° rotation, 270° rotation, or random cropping on the corresponding high-resolution images. All high-resolution and low-resolution images in the training set are normalized.

[0073] This embodiment further provides a pixel-level image super-resolution reconstruction system for implementing the above-mentioned pixel-level image super-resolution reconstruction method, including:

[0074] Shallow feature map generation module, which is used to obtain shallow feature maps by convolution and nonlinear processing of the low-resolution image to be processed;

[0075] The deep feature map generation module is used to extract and fuse the shallow feature map several times to obtain the global local fusion feature map; the global local fusion feature map is fused with the shallow feature map after convolution processing to obtain the deep feature map; the operation of the global local feature extraction and fusion processing is as follows: the input feature map is processed by convolution, batch normalization and activation function to obtain a nonlinear feature map; the nonlinear feature map is processed by pixel-level non-local self-attention to obtain an attention feature map; the attention feature map is fused with the input feature map to obtain an attention residual feature map; the attention residual feature map is processed by layer normalization to obtain an attention normalized feature map; the attention normalization feature map is processed by layer normalization to obtain an attention residual feature map. The feature map is fused through multi-branch dual-path processing to obtain cross-branch multi-scale fusion features; the cross-branch multi-scale fusion features are fused with the attention normalized feature map to obtain the output feature map, which is used to perform global local feature extraction fusion processing or convolution processing; the operation of pixel-level non-local self-attention processing is as follows: each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on pixel similarity, and then fused with its own sub-feature map to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is sequentially subjected to feature enhancement, feature information interaction and nonlinear feature extraction, and then spliced ​​to obtain the attention feature map;

[0076] The high-resolution reconstructed image generation module is used to process the deep feature map based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

[0077] This embodiment further provides a pixel-level image super-resolution reconstruction device, comprising a processor and a memory, wherein the processor implements the above-mentioned pixel-level image super-resolution reconstruction method when executing a computer program stored in the memory.

[0078] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned pixel-level image super-resolution reconstruction method when executed by a processor.

[0079] This embodiment provides a pixel-level image super-resolution reconstruction method. First, a low-resolution image to be processed is subjected to convolution processing and nonlinear processing to capture low-level spatial information of the low-resolution image to be processed, thereby obtaining a shallow feature map. Then, the shallow feature map is subjected to several global-local feature extraction and fusion processing to fully capture long-range dependent and non-local feature information, enhance the multi-scale feature representation capability, and realize efficient fusion of information of different scales to obtain a global-local fusion feature map. After convolution processing, the global-local fusion feature map is fused with the shallow feature map to obtain a deep feature map. Finally, the deep feature map is subjected to sub-pixel-based image reconstruction processing to refine the reconstructed image, realize reconstruction of the low-resolution image, and obtain a high-resolution reconstructed image. This method is used in the field of image super-resolution reconstruction, and has good image detail recovery, high image resolution, and good image reconstruction quality.

Claims

1. A pixel-level image super-resolution reconstruction method, characterized in that: The following operations are included: S1, the low-resolution image to be processed is processed by convolution and nonlinearity to obtain a shallow feature map; S2: The shallow feature map undergoes several rounds of global and local feature extraction and fusion processing to obtain a global and local fusion feature map; the global and local fusion feature map is then fused with the shallow feature map to obtain a deep feature map. The operation of global and local feature extraction and fusion processing is as follows: the input feature map is processed by convolution, batch normalization and activation function to obtain a nonlinear feature map; The nonlinear feature map is processed by pixel-level non-local self-attention to obtain the attention feature map; Fuse the attention feature map with the input feature map to obtain the attention residual feature map; The attention residual feature map is normalized by layers to obtain the attention normalized feature map; The attention normalized feature map is processed by multi-branch dual-path fusion to obtain cross-branch multi-scale fusion features; Fusing the cross-branch multi-scale fusion features with the attention normalized feature map to obtain an output feature map for performing global-local feature extraction fusion processing or convolution processing; The pixel-level non-local self-attention processing operation is as follows: each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on pixel similarity, and then fused with its own sub-feature map to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is sequentially processed by feature enhancement, feature information interaction and nonlinear feature extraction, and then spliced ​​to obtain the attention feature map; S3. The deep feature map is processed based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

2. The pixel-level image super-resolution reconstruction method according to claim 1, characterized in that: In S2, the operation of multi-head attention processing based on pixel similarity is: The pixel information within the neighborhood of each pixel in the sub-feature map forms its own neighborhood pixel block; Select a pixel point from the sub-feature map as the reference pixel point, and the corresponding neighborhood pixel block as the reference neighborhood pixel block; Obtain the similarity between the reference neighborhood pixel block and each neighborhood pixel block in the sub-feature map under the current attention head as the similarity between the current head reference pixel point and each pixel point; The similarity between the current head reference pixel and each pixel is multiplied by the corresponding pixel embedding vector and summed to obtain the current head attention feature; After weighted summation of all head attention features, regularization is performed to obtain the initial attention feature map of the sub-feature map, which is used to perform fusion operations with the sub-feature map.

3. The pixel-level image super-resolution reconstruction method according to claim 2, characterized in that: The similarity between the reference neighborhood pixel block and the neighborhood pixel block is calculated using the following formula: , For the i Reference neighborhood pixel blocks under attention heads Hedi p Neighborhood pixel blocks The similarity between 、 They are the benchmark neighborhood pixel blocks Embedding vector, neighborhood pixel block The embedding vector of 、 are the feature dimensions and the total number of attention heads, respectively. is the standardization function.

4. The pixel-level image super-resolution reconstruction method according to claim 1, characterized in that: In S2, the operation of cross-branch multi-scale fusion features is: The attention normalized feature map is subjected to feature extraction at different expansion rates to obtain several attention branch feature maps; The current attention branch feature map is average pooled and max pooled respectively to obtain the current branch average pooling feature map and the current branch max pooling feature map; The current branch average pooling feature map and the current branch maximum pooling feature map are processed by weight generation based on parallel convolution to obtain the current first path weight matrix; the current branch average pooling feature map and the current branch maximum pooling feature map are processed by weight generation based on fusion convolution to obtain the current second path weight matrix; the current first path weight matrix and the current second path weight matrix are multiplied element by element to obtain the current comprehensive weight matrix; Each attention branch feature map is weighted and convolved with its own comprehensive weight matrix to obtain cross-branch multi-scale fusion features.

5. The pixel-level image super-resolution reconstruction method according to claim 4, characterized in that: The second path weight matrix is ​​calculated by the following formula: ， For the m The second path weight matrix of the attention branch feature map, 、 Respectively m The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, 、 Respectively n The branch average pooling feature map and the branch maximum pooling feature map of the attention branch feature map, For splicing operations, 、 The first convolution of the second path and the second convolution of the second path, N is the total number of attention branch feature maps.

6. The pixel-level image super-resolution reconstruction method according to claim 1, characterized in that: In S2, the feature enhancement operations are as follows: After layer normalization, fully connected feedforward network processing and regularization, the multi-head attention sub-feature map is added element-by-element with the multi-head attention sub-feature map to obtain the multi-head attention enhanced sub-feature map, which is used to perform feature information interaction operations.

7. The pixel-level image super-resolution reconstruction method according to claim 1, characterized in that: In S3, sub-pixel based image reconstruction processing can be achieved through convolution, sub-pixel convolution, activation function processing and convolution processing.

8. A pixel-level image super-resolution reconstruction system, used to implement the pixel-level image super-resolution reconstruction method according to claim 1, characterized in that: include: Shallow feature map generation module, which is used to obtain shallow feature maps by convolution and nonlinear processing of the low-resolution image to be processed; The deep feature map generation module is used to extract and fuse the shallow feature map several times to obtain a global-local fusion feature map. After the global-local fusion feature map is processed by convolution, it is fused with the shallow feature map to obtain a deep feature map. The global-local feature extraction and fusion process is as follows: the input feature map is processed by convolution, batch normalization and activation function to obtain a nonlinear feature map. The nonlinear feature map is processed by pixel-level non-local self-attention to obtain the attention feature map; Fuse the attention feature map with the input feature map to obtain the attention residual feature map; The attention residual feature map is normalized by layers to obtain the attention normalized feature map; The attention normalized feature map is processed by multi-branch dual-path fusion to obtain cross-branch multi-scale fusion features; The cross-branch multi-scale fusion features are fused with the attention normalized feature map to obtain an output feature map, which is used to perform global local feature extraction fusion processing or convolution processing; the pixel-level non-local self-attention processing operation is as follows: each non-overlapping sub-feature map of the nonlinear feature map is processed by multi-head attention based on pixel similarity, and then fused with its own sub-feature map to obtain several multi-head attention sub-feature maps; each multi-head attention sub-feature map is sequentially subjected to feature enhancement, feature information interaction and nonlinear feature extraction, and then spliced ​​to obtain an attention feature map; The high-resolution reconstructed image generation module is used to process the deep feature map based on sub-pixel image reconstruction to obtain a high-resolution reconstructed image.

9. A pixel-level image super-resolution reconstruction device, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the pixel-level image super-resolution reconstruction method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, the pixel-level image super-resolution reconstruction method according to any one of claims 1 to 7 is implemented.

Images