Image super-resolution reconstruction method based on feature adaptive attention

Abstract

The application relates to an image super-resolution reconstruction method based on feature adaptive attention, which obtains a reconstructed image based on a constructed image super-resolution reconstruction model, wherein the image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module and a convolution layer, the method effectively improves the quality and precision of low-resolution image reconstruction, and the constructed image super-resolution reconstruction model has the characteristics of being lightweight and extremely fast in inference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer image processing, in particular to an image super-resolution reconstruction method based on feature adaptive attention. BACKGROUND

[0002] Image super-resolution is a cutting-edge technology that uses deep learning algorithms to improve image resolution. It aims to reconstruct high-definition images from low-resolution images, making image details clearer and significantly improving image quality. This technology has broad application prospects in many fields such as surveillance, medical imaging, satellite images, and entertainment industries, showing great practical value.

[0003] Currently, image processing algorithms in the field of computer vision have achieved remarkable achievements and have excellent performance. However, in the sub-field of image super-resolution, the generalization ability of existing models still needs to be further improved, and there are still certain limitations. In order to break through this bottleneck, researchers are constantly exploring innovative algorithms and models, striving to achieve stable and high-quality super-resolution effects in a wider range of application scenarios.

[0004] The current mainstream method in the field of image super-resolution is to increase the depth of the network and introduce self-attention encoding mechanism to enhance the network model's ability to weight and reconstruct the features of low-resolution images. This approach is similar to convolution operation and non-local calculation. Such processing makes it difficult to capture features mapped to the hidden space, and the features are very similar to those of other objects, leading to reconstructed results that are prone to artifacts and do not conform to image logic.

[0005] Traditional interpolation algorithm trained models cannot perform well in severely degraded images and often have unrecognizable feature information and artifacts. Image super-resolution based on GAN faces two major difficulties: first, due to the lack of image pairs with various degradation information, it is difficult to obtain a real dataset for image super-resolution reconstruction; second, the performance of the super-resolution model depends largely on the ability to identify local information and texture features, but increasing the depth of dense layers to extract high-level features cannot be directly applied to image data rich in texture information. The bottleneck of image super-resolution is still the inability to learn the mapping relationship from low-resolution images to high-resolution images, and the proposed ill-posed problem remains a major challenge. SUMMARY

[0006] In order to overcome at least one deficiency in the prior art, the present application provides an image super-resolution reconstruction method based on feature adaptive attention.

[0007] In a first aspect, an image super-resolution reconstruction method based on feature adaptive attention is provided, comprising:

[0008] constructing an image super-resolution reconstruction model; the image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module and a convolution layer, the feature adaptive attention module comprises a plurality of feature adaptive attention units, the fusion module comprises a plurality of fusion units, and the feature adaptive attention units and the fusion units correspond one by one;

[0009] training the image super-resolution reconstruction model based on a training data set to obtain a trained image super-resolution reconstruction model;

[0010] inputting a to-be-reconstructed image into the trained image super-resolution reconstruction model to obtain a reconstructed image;

[0011] the feature extraction module performs feature extraction on the input to-be-reconstructed image to obtain a feature map;

[0012] the feature map is input into the feature adaptive attention module for feature adaptive attention calculation, each feature adaptive attention unit outputs a first calculation result and a second calculation result, the first calculation result is input into the next feature adaptive attention unit, and the second calculation result is input into the corresponding fusion unit; each fusion unit outputs a fusion result, and the fusion result is input into the next fusion unit;

[0013] the feature splicing module is used for splicing the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result;

[0014] the splicing result is input into the convolution layer to obtain the reconstructed image.

[0015] In one embodiment, the feature extraction unit comprises convolution layers, activation function layers, convolution layers and activation function layers connected in sequence, and the feature extraction unit is used for mapping image features to a hidden space and preserving the features of the input image.

[0016] In one embodiment, the feature adaptive attention unit comprises a first branch and a second branch, the first branch comprises a group feature weighted fusion module VFM and a channel space scale attention mechanism module CSA connected in sequence, and the second branch comprises a non-local self-attention module NLSA, a deconvolution layer and an activation function layer; the inputs of the feature adaptive attention unit pass through the first branch and the second branch respectively, the output of the first branch is subtracted from the output of the second branch to obtain a first subtraction result;

[0017] the output after the first subtraction result passes through a convolution layer, an activation function layer and a convolution layer is added to the output of the first branch to obtain a first addition result;

[0018] The output of the first addition result after passing through an activation function layer and a stride convolution layer is subtracted from the input of the feature adaptive attention unit to obtain a second subtraction result;

[0019] The output of the second subtraction result after passing through an inverse convolution layer and an activation function layer is added to the output of the first branch to obtain a first calculation result;

[0020] The first calculation result passes through a stride convolution layer, an activation function layer, a convolution layer, and an activation function layer to obtain a second calculation result.

[0021] In an embodiment, the group feature weighted fusion module VFM includes a first average pooling layer, a second average pooling layer, a convolution layer, a first SoftMax layer, a third average pooling layer, a Sigmoid layer, a GroupNorm unit, a second SoftMax layer, and a fourth average pooling layer.

[0022] The input of the group feature weighted fusion module VFM passes through the first average pooling layer and the second average pooling layer, respectively, and the outputs of the first average pooling layer and the second average pooling layer are spliced and input to the Sigmoid layer. The output of the Sigmoid layer is multiplied by the input of the group feature weighted fusion module VFM to obtain a first multiplication result. The first multiplication result is input to the GroupNorm unit, and the output of the GroupNorm unit is input to the second SoftMax layer and the fourth average pooling layer, respectively.

[0023] The input of the group feature weighted fusion module VFM passes through the convolution layer, and the output of the convolution layer is input to the first SoftMax layer and the third average pooling layer, respectively.

[0024] The output of the first SoftMax layer is multiplied by the output of the fourth average pooling layer to obtain a second multiplication result.

[0025] The output of the third average pooling layer is multiplied by the output of the second SoftMax layer to obtain a third multiplication result.

[0026] The second multiplication result and the third multiplication result are added to obtain the output of the group feature weighted fusion module VFM.

[0027] In an embodiment, the channel-spatial-scale attention mechanism module CSA includes two branches with the same structure and a self-attention mechanism module. Each branch includes a convolution layer, a batch normalization layer, an activation function layer, a convolution layer, and a batch normalization layer.

[0028] The input of the channel-spatial-scale attention mechanism module CSA is input into the two branches with the same structure and the self-attention mechanism module, respectively, and the three obtained outputs are added to obtain the output of the channel-spatial-scale attention mechanism module CSA.

[0029] In one embodiment, the input of the first fusion unit of the plurality of fusion units is the second calculation result output by the first feature adaptive attention unit, the feature map output by the feature extraction module, and the image to be reconstructed.

[0030] In a second aspect, an image super-resolution reconstruction device based on feature adaptive attention is provided, comprising:

[0031] a model construction module, configured to construct an image super-resolution reconstruction model; the image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module, and a convolution layer; the feature adaptive attention module comprises a plurality of feature adaptive attention units; the fusion module comprises a plurality of fusion units; the feature adaptive attention units and the fusion units correspond to each other one by one;

[0032] a model training module, configured to train the image super-resolution reconstruction model based on a training data set, to obtain a trained image super-resolution reconstruction model;

[0033] a reconstruction module, configured to input the image to be reconstructed into the trained image super-resolution reconstruction model, to obtain a reconstructed image;

[0034] The feature extraction module extracts features of the input image to be reconstructed to obtain a feature map.

[0035] The feature map is input into the feature adaptive attention module for feature adaptive attention calculation; each feature adaptive attention unit outputs a first calculation result and a second calculation result; the first calculation result is input into the next feature adaptive attention unit, and the second calculation result is input into the corresponding fusion unit; each fusion unit outputs a fusion result, and the fusion result is input into the next fusion unit.

[0036] The feature splicing module is configured to splice the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result.

[0037] The splicing result is input into the convolution layer to obtain the reconstructed image.

[0038] In a third aspect, a computer readable storage medium is provided, which stores a computer program; when the computer program is executed by a processor, the above-mentioned image super-resolution reconstruction method based on feature adaptive attention is implemented.

[0039] In a fourth aspect, a computer program product is provided, comprising computer programs / instructions; when the computer programs / instructions are executed by a processor, the above-mentioned image super-resolution reconstruction method based on feature adaptive attention is implemented.

[0040] Compared with the prior art, the image super-resolution reconstruction method based on feature adaptive attention has the following beneficial effects: the image super-resolution reconstruction method based on feature adaptive attention of the application obtains a reconstructed image based on a constructed image super-resolution reconstruction model, the image super-resolution reconstruction model includes a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module and a convolution layer, and the method effectively improves the quality and accuracy of low-resolution image reconstruction; and the constructed image super-resolution reconstruction model has the characteristics of lightweight and extremely fast inference. BRIEF DESCRIPTION OF DRAWINGS

[0041] The application can be better understood by referring to the following description in conjunction with the accompanying drawings, which are incorporated in and form a part of the specification, and together with the detailed description, serve to explain the principles of the application. In the drawings:

[0042] Figure 1 A flow block diagram of the image super-resolution reconstruction method based on feature adaptive attention according to an embodiment of the application is shown;

[0043] Figure 2 A structural block diagram of an image super-resolution reconstruction model is shown;

[0044] Figure 3 A structural block diagram of a feature adaptive attention unit is shown;

[0045] Figure 4 A structural block diagram of a group feature weighted fusion module is shown;

[0046] Figure 5 A structural block diagram of a channel space scale attention mechanism module is shown;

[0047] Figure 6 A reconstructed image result obtained according to the method of the application is shown;

[0048] Figure 7 A structural block diagram of an image super-resolution reconstruction device based on feature adaptive attention according to an embodiment of the application is shown. DETAILED DESCRIPTION

[0049] In the following, exemplary embodiments of the application will be described with reference to the accompanying drawings. In the specification, not all the features of the actual embodiments are described for the sake of clarity and conciseness. However, it should be appreciated that many embodiment-specific decisions can be made in the process of developing any such actual embodiment in order to achieve the specific goals of the developer, and these decisions can vary from embodiment to embodiment.

[0050] It should be noted that, in order not to obscure the application with unnecessary details, only the structures of the devices closely related to the scheme according to the application are shown in the drawings, and other details not related to the application are omitted.

[0051] It should be understood that the application is not limited to the described embodiments by virtue of the following description with reference to the drawings. In this context, the embodiments can be combined with each other, features can be replaced or borrowed between different embodiments, and one or more features can be omitted in one embodiment.

[0052] The embodiment of the application provides a feature adaptive attention-based image super-resolution reconstruction method, Figure 1 The feature adaptive attention-based image super-resolution reconstruction method according to the embodiment of the application is shown in a flowchart, see Figure 1 The method comprises the following steps.

[0053] Step S1, constructing an image super-resolution reconstruction model VFSR. Figure 2 The structure block diagram of the image super-resolution reconstruction model is shown, see Figure 2 The image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature concatenation module Concatenate and a convolution layer Convolution. The feature adaptive attention module comprises a plurality of feature adaptive attention units FFR. The fusion module comprises a plurality of fusion units Fusion. The feature adaptive attention units and the fusion units are in one-to-one correspondence.

[0054] Step S2, training the image super-resolution reconstruction model based on a training data set to obtain a trained image super-resolution reconstruction model.

[0055] Here, the DIV2K public data set is used for training. The picture format and resolution of the unified data set are unified, and then the data set is randomly cropped and data enhanced to ensure sufficient data amount. A traditional bicubic down-sampling degradation model is used to process the data images to obtain a LR-HR image pair, i.e., a low-resolution-high-resolution image, to constitute a training data set. The model is trained based on the training data set. The parameters of the network when generating images after each iteration are constantly adjusted by operating the core module. Under the condition of sufficient data amount, the learning rate is constantly reduced, and deep image features are extracted. After the final indicators PSNR (peak signal-to-noise ratio) and SSIM (structural similarity) are stabilized to a certain range, the training is stopped, and the parameter file with better effect is retained, i.e., the trained model is obtained.

[0056] Step S3, inputting the to-be-reconstructed image into the trained image super-resolution reconstruction model to obtain a reconstructed image.

[0057] Here, the feature extraction module performs feature extraction on the input to-be-reconstructed image to obtain a feature map.

[0058] The feature map is input to the feature adaptive attention module for feature adaptive attention calculation, each feature adaptive attention unit outputs a first calculation result and a second calculation result, the first calculation result is input to the next feature adaptive attention unit, and the second calculation result is input to the corresponding fusion unit; each fusion unit outputs a fusion result, and the fusion result is input to the next fusion unit.

[0059] The feature splicing module is used for splicing the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result.

[0060] The splicing result is input to the convolution layer to obtain a reconstructed image.

[0061] Here, each feature adaptive attention unit outputs a feature deepened by the current unit, i.e., the first calculation result, and an up-sampled feature used for hierarchical fusion, i.e., the second calculation result. Each fusion unit receives two inputs, which are the second calculation result output by the feature adaptive attention unit and the output of the previous fusion unit.

[0062] The image super-resolution reconstruction method based on feature adaptive attention of the embodiment obtains a reconstructed image based on the constructed image super-resolution reconstruction model, the image super-resolution reconstruction model includes a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module, and a convolution layer, the method effectively improves the quality and accuracy of low-resolution image reconstruction; and the constructed image super-resolution reconstruction model has the characteristics of lightweight and extremely fast inference.

[0063] In one embodiment, referring to Figure 2 , the feature extraction unit includes convolution layers Convolution, activation function layers PReLU, convolution layers Convolution, and activation function layers PReLU connected in sequence, and the feature extraction unit is used for mapping image features to a hidden space and preserving the features of the input image.

[0064] In one embodiment, Figure 3 The structural block diagram of the feature adaptive attention unit is shown, referring to Figure 3The feature adaptive attention unit comprises a first branch and a second branch, the first branch comprises a group feature weighted fusion module VFM and a channel spatial scale attention mechanism module CSA connected in sequence, and the second branch comprises a non-local self-attention module NLSA, a deconvolution layer Deconvolution and an activation function layer PReLU; inputs of the feature adaptive attention unit pass through the first branch and the second branch respectively, the output of the first branch is subtracted from the output of the second branch to obtain a first subtraction result;

[0065] The output after the first subtraction result passes through a convolution layer, an activation function layer and a convolution layer is added to the output of the first branch to obtain a first addition result;

[0066] The output after the first addition result passes through an activation function layer and a strided convolution layer Strided Convolution is subtracted from the input of the feature adaptive attention unit to obtain a second subtraction result;

[0067] The output after the second subtraction result passes through a deconvolution layer and an activation function layer is added to the output of the first branch to obtain a first calculation result;

[0068] The first calculation result passes through a strided convolution layer, an activation function layer, a convolution layer and an activation function layer to obtain a second calculation result.

[0069] In the field of image processing, the processing of features is usually to directly splice the feature maps output by multiple branches in the channel dimension to form a wider feature map. This method can retain complete feature information from different branches, and is therefore commonly used in scenarios that require the fusion of multiple source heterogeneous features. However, this will cause the number of channels of the feature map to increase significantly, and the computational amount and complexity of the model will also increase geometrically, resulting in a significant reduction in efficiency. Considering that simply splicing features in the channel dimension cannot effectively fuse feature information in multiple dimensions, this will affect the final image reconstruction effect. Therefore, the embodiment performs feature differentiation processing layer by layer before fusing the output features of each branch, and then performs feature fusion.

[0070] In the above process, the NLSA uses sparsity constraints to force the module to focus on relevant and information-rich areas while ignoring irrelevant and noisy features.

[0071] In one embodiment, Figure 4 The structure block diagram of the group feature weighted fusion module is shown, referring to Figure 4 The group feature weighted fusion module VFM comprises a first average pooling layer AugPool, a second average pooling layer AugPool, a convolution layer, a first SoftMax layer, a third average pooling layer AugPool, a Sigmoid layer, a GroupNorm unit, a second SoftMax layer and a fourth average pooling layer AugPool.

[0072] The input of the group feature weighted fusion module VFM is input into a first average pooling layer and a second average pooling layer respectively, the outputs of the first average pooling layer and the second average pooling layer are spliced, and then input into a Sigmoid layer, the output of the Sigmoid layer is multiplied by the input of the group feature weighted fusion module VFM to obtain a first multiplication result; the first multiplication result is input into a GroupNorm unit, and the output of the GroupNorm unit is input into a second SoftMax layer and a fourth average pooling layer respectively;

[0073] The input of the group feature weighted fusion module VFM is input into a convolution layer, and the output of the convolution layer is input into a first SoftMax layer and a third average pooling layer respectively;

[0074] The output of the first SoftMax layer is multiplied by the output of the fourth average pooling layer to obtain a second multiplication result;

[0075] The output of the third average pooling layer is multiplied by the output of the second SoftMax layer to obtain a third multiplication result;

[0076] The second multiplication result and the third multiplication result are added to obtain the output of the group feature weighted fusion module VFM.

[0077] In this embodiment, the data input into the group feature weighted fusion module VFM is divided into two branches, one branch is used as a feature weight adaptive score to weight and reconstruct the overall feature, that is, input into the first average pooling layer and the second average pooling layer, and the other branch is used to fuse the features after feature dimension reduction and channel grouping, that is, input into the convolution layer. The GroupNorm in the first branch is less sensitive to the size of a batch, and the GroupNorm is used for grouped feature normalization. In order to emphasize important features, reduce the weight of unimportant features, and capture global context information, the features output by the GroupNorm are divided into two branches and input into the second SoftMax layer and the fourth average pooling layer respectively.

[0078] In the above embodiment, unlike the traditional method, the grouped features of the image are processed in the first branch, and self-attention weighted fusion is realized. The purpose of the second branch is to extract the spatial features of the image. To this end, the input features are first subjected to convolution operation, and then branch processing is performed.

[0079] In one embodiment, Figure 5 The structural block diagram of the channel spatial scale attention mechanism module is shown, see Figure 5The channel space scale attention mechanism module CSA includes two branches of self-attention mechanism modules with the same structure.

[0080] The input of the channel space scale attention mechanism module CSA is input into two branches of self-attention mechanism modules with the same structure, and three outputs obtained are added to obtain the output of the channel space scale attention mechanism module CSA.

[0081] In an embodiment, the L1 loss is used as an optimization objective in the model training process.

[0082]

[0083] wherein VF * is the optimized image super-resolution reconstruction model, VF is an image super-resolution reconstruction model, M is the number of image pairs in the training process, i is the index of the image pair, I LR and I HR are a low-resolution image and a high-resolution image in the image pair, respectively; VF(I LR ) represents the output of the image super-resolution reconstruction model VF when the input is I LR .

[0084] Figure 6 The reconstructed image results obtained according to the method of the present application are shown. At the same time, the method of the present application is compared with the prior art, and the results are shown in Table 1. The image finally reconstructed by the method of the present application achieves good results, and has made great progress in the symmetry feature of resolution improvement. In addition, the network also has the characteristics of lightweight and extremely fast reasoning.

[0085] Table 1

[0086]

[0087] Based on the same inventive concept as the image super-resolution reconstruction method based on feature adaptive attention, the present embodiment also provides an image super-resolution reconstruction device based on feature adaptive attention, Figure 7 The structural block diagram of the image super-resolution reconstruction device based on feature adaptive attention according to the embodiment of the present application is shown, referring to Figure 7 The device includes:

[0088] The model construction module 71 is configured to construct an image super-resolution reconstruction model. The image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module, and a convolution layer. The feature adaptive attention module comprises a plurality of feature adaptive attention units. The fusion module comprises a plurality of fusion units. The feature adaptive attention units and the fusion units are in one-to-one correspondence.

[0089] The model training module 72 is configured to train the image super-resolution reconstruction model based on a training data set to obtain a trained image super-resolution reconstruction model.

[0090] The reconstruction module 73 is configured to input a to-be-reconstructed image into the trained image super-resolution reconstruction model to obtain a reconstructed image.

[0091] The feature extraction module is configured to perform feature extraction on the input to-be-reconstructed image to obtain a feature map.

[0092] The feature map is input into the feature adaptive attention module for feature adaptive attention calculation. Each feature adaptive attention unit outputs a first calculation result and a second calculation result. The first calculation result is input into a next feature adaptive attention unit, and the second calculation result is input into a corresponding fusion unit. Each fusion unit outputs a fusion result, and the fusion result is input into a next fusion unit.

[0093] The feature splicing module is configured to splice the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result.

[0094] The splicing result is input into the convolution layer to obtain the reconstructed image.

[0095] The image super-resolution reconstruction device based on the feature adaptive attention of the embodiment has the same inventive concept as the image super-resolution reconstruction method based on the feature adaptive attention described above. Therefore, the specific embodiments of the device can be seen in the embodiment part of the image super-resolution reconstruction method based on the feature adaptive attention described above, and the technical effects thereof correspond to the technical effects of the method described above. Here, no longer be repeated.

[0096] The embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the image super-resolution reconstruction method based on the feature adaptive attention is realized.

[0097] The embodiment of the present application provides a computer program product, which comprises computer programs / instructions. When the computer programs / instructions are executed by a processor, the image super-resolution reconstruction method based on the feature adaptive attention is realized.

[0098] The above merely provides the various embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A feature-adaptive attention-based image super-resolution reconstruction method, characterized in that, The method comprises the following steps: constructing an image super-resolution reconstruction model; the image super-resolution reconstruction model comprises a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module and a convolution layer, the feature adaptive attention module comprises a plurality of feature adaptive attention units, and the fusion module comprises a plurality of fusion units, the feature adaptive attention unit and the fusion unit correspond to each other; training the image super-resolution reconstruction model based on a training data set to obtain a trained image super-resolution reconstruction model; inputting a to-be-reconstructed image into the trained image super-resolution reconstruction model to obtain a reconstructed image; the feature extraction module extracts features of the input to-be-reconstructed image to obtain a feature map; the feature map is input into the feature adaptive attention module for feature adaptive attention calculation, each feature adaptive attention unit outputs a first calculation result and a second calculation result, the first calculation result is input into the next feature adaptive attention unit, and the second calculation result is input into the corresponding fusion unit; each fusion unit outputs a fusion result, and the fusion result is input into the next fusion unit; the feature splicing module is used for splicing the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result; the splicing result is input into the convolution layer to obtain a reconstructed image; the feature adaptive attention unit comprises a first branch and a second branch, the first branch comprises a group feature weighted fusion module VFM and a channel space scale attention mechanism module CSA connected in sequence, and the second branch comprises a non-local self-attention module NLSA, a deconvolution layer and an activation function layer; the input of the feature adaptive attention unit is input into the first branch and the second branch respectively, the output of the first branch is subtracted from the output of the second branch to obtain a first subtraction result; the output of the first subtraction result after passing through a convolution layer, an activation function layer and a convolution layer is added to the output of the first branch to obtain a first addition result; the output of the first addition result after passing through an activation function layer and a cross-step convolution layer is subtracted from the input of the feature adaptive attention unit to obtain a second subtraction result; the output of the second subtraction result after passing through a deconvolution layer and an activation function layer is added to the output of the first branch to obtain the first calculation result; the first calculation result passes through a cross-step convolution layer, an activation function layer, a convolution layer and an activation function layer to obtain the second calculation result; the group feature weighted fusion module VFM comprises a first average pooling layer, a second average pooling layer, a convolution layer, a first SoftMax layer, a third average pooling layer, a Sigmoid layer, a GroupNorm unit, a second SoftMax layer and a fourth average pooling layer. The input of the group feature weighted fusion module VFM is input into a first average pooling layer and a second average pooling layer respectively, the outputs of the first average pooling layer and the second average pooling layer are spliced and input into a Sigmoid layer, the output of the Sigmoid layer is multiplied by the input of the group feature weighted fusion module VFM to obtain a first multiplication result; the first multiplication result is input into the GroupNorm unit, and the outputs of the GroupNorm unit are input into the second SoftMax layer and the fourth average pooling layer respectively; The input of the group feature weighted fusion module VFM is input into the convolution layer, and the output of the convolution layer is input into the first SoftMax layer and the third average pooling layer respectively; The output of the first SoftMax layer is multiplied by the output of the fourth average pooling layer to obtain a second multiplication result; The output of the third average pooling layer is multiplied by the output of the second SoftMax layer to obtain a third multiplication result; The second multiplication result and the third multiplication result are added to obtain the output of the group feature weighted fusion module VFM.

2. The method of claim 1, wherein, The feature extraction module includes convolution layers, activation function layers, convolution layers and activation function layers connected in sequence, and is used for mapping image features to an implicit space and preserving the features of an input image.

3. The method of claim 1, wherein, The channel spatial scale attention mechanism module CSA includes two branches with the same structure and a self-attention mechanism module; each branch includes convolution layers, batch normalization layers and activation function layers. The input of the channel spatial scale attention mechanism module CSA is input into two branches with the same structure and a self-attention mechanism module to obtain three outputs, which are added to obtain the output of the channel spatial scale attention mechanism module CSA.

4. The method of claim 1, wherein, The input of a first fusion unit in the plurality of fusion units is the second calculation result output by the first feature adaptive attention unit, the feature map output by the feature extraction module and the image to be reconstructed.

5. An image super-resolution reconstruction device based on feature adaptive attention, characterized in that, The model construction module is configured to construct an image super-resolution reconstruction model; the image super-resolution reconstruction model includes a feature extraction module, a feature adaptive attention module, a fusion module, a feature splicing module and a convolution layer, the feature adaptive attention module includes a plurality of feature adaptive attention units, the fusion module includes a plurality of fusion units, and the feature adaptive attention units and the fusion units correspond one by one; The model training module is configured to train the image super-resolution reconstruction model based on a training data set to obtain a trained image super-resolution reconstruction model; The reconstruction module is configured to input an image to be reconstructed into the trained image super-resolution reconstruction model to obtain a reconstructed image; The feature extraction module extracts features of the input image to be reconstructed to obtain a feature map; The feature extraction module extracts features of the input image to be reconstructed to obtain a feature map; The feature map is input into the feature adaptive attention module for feature adaptive attention calculation, each feature adaptive attention unit outputs a first calculation result and a second calculation result, the first calculation result is input into a next feature adaptive attention unit, and the second calculation result is input into a corresponding fusion unit; Each fusion unit outputs a fusion result, and the fusion result is input into a next fusion unit; The feature splicing module is used for splicing the fusion result output by each fusion unit and the first calculation result output by the last feature adaptive attention module to obtain a splicing result; The splicing result is input into the convolution layer to obtain a reconstructed image; The feature adaptive attention unit includes a first branch and a second branch, the first branch includes a group feature weighted fusion module VFM and a channel space scale attention mechanism module CSA connected in sequence, and the second branch includes a non-local self-attention module NLSA, a deconvolution layer and an activation function layer; The input of the feature adaptive attention unit is input into the first branch and the second branch respectively, the output of the first branch is subtracted from the output of the second branch to obtain a first subtraction result; The output after the first subtraction result passes through a convolution layer, an activation function layer and a convolution layer is added to the output of the first branch to obtain a first addition result; The output after the first addition result passes through an activation function layer and a cross-step convolution layer is subtracted from the input of the feature adaptive attention unit to obtain a second subtraction result; The output after the second subtraction result passes through a deconvolution layer and an activation function layer is added to the output of the first branch to obtain the first calculation result; The first calculation result passes through a cross-step convolution layer, an activation function layer, a convolution layer and an activation function layer to obtain the second calculation result; The group feature weighted fusion module VFM includes a first average pooling layer, a second average pooling layer, a convolution layer, a first SoftMax layer, a third average pooling layer, a Sigmoid layer, a GroupNorm unit, a second SoftMax layer and a fourth average pooling layer; The input of the group feature weighted fusion module VFM passes through the first average pooling layer and the second average pooling layer respectively, the outputs of the first average pooling layer and the second average pooling layer are spliced and input into the Sigmoid layer, the output of the Sigmoid layer is multiplied by the input of the group feature weighted fusion module VFM to obtain a first multiplication result; the first multiplication result is input into the GroupNorm unit, and the output of the GroupNorm unit is input into the second SoftMax layer and the fourth average pooling layer respectively; The input of the group feature weighted fusion module VFM passes through the convolution layer, and the output of the convolution layer is input into the first SoftMax layer and the third average pooling layer respectively; The output of the first SoftMax layer is multiplied by the output of the fourth average pooling layer to obtain a second multiplication result; The output of the third average pooling layer is multiplied by the output of the second SoftMax layer to obtain a third multiplication result; The second multiplication result and the third multiplication result are added to obtain an output of a group feature weighted fusion module VFM.

6. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the image super-resolution reconstruction method based on feature adaptive attention in any one of claims 1-4.

7. A computer program product, characterised in that, The computer program / instructions are executed by the processor to implement the image super-resolution reconstruction method based on feature adaptive attention in any one of claims 1-4.

Images