Improved method for image super-resolution based on self-attention

Abstract

The application discloses an improved image super-resolution method based on self-attention, comprising the following steps: a data preparation step: constructing an image super-resolution database, sampling high-resolution image blocks from high-definition images, and performing degradation processing on the sampled image blocks to obtain low-resolution image blocks, and taking pairs of high-resolution image blocks and low-resolution image blocks as labels and inputs of a training model; an image preprocessing module design step: constructing a multi-scale image preprocessing module, the input of the preprocessing module being a low-resolution image I LR , and the output being a feature map X pre_out ; an enhancement module design step: constructing an enhancement module based on noise level; a residual generation module design step: constructing a multi-scale residual generation module; and a training model step: connecting the image preprocessing module, the enhancement module and the reconstruction module to form a super-resolution model and training the model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image super-resolution improvement method based on self-attention. Background Technology

[0002] Image super-resolution is the process of recovering a high-resolution image from a low-resolution one. For example, it can double the resolution of an image from 960*540 to 1920*1080, making it easier for users to view on large-screen displays. Image super-resolution is one of the fundamental problems in image processing and has a wide range of practical needs and applications.

[0003] In recent years, with the continuous development of deep learning technology, Convolutional Neural Networks (CNNs) have received widespread attention in computer vision and achieved remarkable results. In the field of image super-resolution, most deep learning-based super-resolution methods have also demonstrated excellent performance, but they still have limitations. For convolution, the setting of the convolutional kernel limits the size of the receptive field; each convolutional layer can only acquire local spatial information, leading to the need for multiple layers to focus on the entire feature map. Attention mechanisms are a data processing method in machine learning, and self-attention is one of the most widely used attention mechanisms. It and its variants are widely used in natural language processing, image processing, and speech recognition, achieving very good results. The advantage of self-attention is that it can acquire global spatial information of the feature map; therefore, introducing self-attention into super-resolution models can improve model performance. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention provides an improved image super-resolution method based on self-attention, which introduces a self-attention module to enhance the expressive power of the model and improve the super-resolution effect by filtering global image features.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A self-attention-based image super-resolution improvement method includes the following steps:

[0007] Data preparation steps: Construct an image super-resolution database, sample high-resolution image patches from high-resolution images, and perform degradation processing on the sampled image patches to obtain low-resolution image patches. Use pairs of high-resolution image patches and low-resolution image patches as labels and inputs for training models.

[0008] Image preprocessing module design steps: Construct a multi-scale image preprocessing module, including one 2D convolutional layer, nine DoubleConv2d structures, four attention modules, four downsampling layers, four upsampling layers, and four stitching layers. The input to the preprocessing module is a low-resolution image I. LR The output is the feature map X. pre_out ;

[0009] Enhancement module design steps: Construct a noise-level-based enhancement module, including 3 convolutional layers, 2 upsampling layers, 1 LeakyReLU activation layer, and 1 transform operation. The enhancement module will process the feature map X output by the preprocessing module. pre_out The process involves two branches. One branch convolves the feature map into an image, which is then compared with a low-resolution image I. LR One approach involves fusing the feature maps before upsampling. Another approach is to first upsample the feature maps and then convolve them into an image. The results of these two processes are then fused to obtain a preliminary super-resolution result X. mid_out ;

[0010] The design steps for the residual generation module are as follows: Construct a multi-scale residual generation module, including 9 DoubleConv2d structures, 4 attention modules, 4 downsampling layers, 4 upsampling layers, 4 concatenation layers, and 1 convolutional layer. This module is used to process the feature map X output by the enhancement module. enh_out The process involves operations including convolution, activation, downsampling, self-attention filtering, and upsampling. Then, the feature map is mapped to an image—a residual image containing high-frequency information—through a final convolutional layer. This residual image is then compared with the initial super-resolution result X. mid_out The results are fused to obtain the final super-resolution result G(I). LR );

[0011] Training model steps: Connect the preprocessing module, enhancement module, and reconstruction module to form a super-resolution model, and then train the model.

[0012] In one possible implementation, the construction of the image super-resolution database, high-resolution image patch sampling of high-definition images, and degradation processing of the sampled image patches to obtain low-resolution image patches further includes: the dataset includes Vimeo, RealSR, REDS, DIV2K, and Flickr2K image databases, as well as collected 4K high-definition data, wherein Vimeo, REDS, and the collected 4K high-definition data are video datasets, each scene sequence consists of multiple consecutive frames of images, and only one frame of each scene sequence is taken; image patch sampling is performed on each high-resolution image, the image patch size is 256*256*3, the image patch is used as the label for super-resolution training, and degradation processing including blurring, downsampling, adding noise, and compression is performed on the high-definition 256*256*3 image patch to obtain the corresponding 128*128*3 low-resolution image patch.

[0013] In one possible implementation, the blurring degradation process involves blurring the image using different blur kernels, including anisotropic Gaussian blur kernels, anisotropic Gaussian blur kernels, and motion blur kernels.

[0014] In one possible implementation, the downsampling degradation is achieved by performing a 0.5x bicubic interpolation on the image.

[0015] In one possible implementation, the degradation process of adding noise involves adding Gaussian noise of a random noise level to the image, with the noise level value ranging from 0 to 1.

[0016] In one possible implementation, the image preprocessing module further includes: a first convolutional layer with a kernel size of 3*3, a number of 16 kernels, padding of 1, and a stride of 1; a DoubleConv2d module structure including two 2D convolutional layers and two LeakyReLU activation layers, each convolutional layer having a kernel size of 3*3 and a number of kernels of 16, 32, 64, 128, 256, 128, 64, 32, and 16 respectively, padding of 1, and a stride of 1; the negative slope parameter of the LeakyReLU activation layer is 0.1; and an attention module structure including two 1D convolutional layers, four reshape layers, and one self-attention layer. The module consists of a self-attention module, which includes a Split layer and a Sofmax layer. The Split layer divides the first dimension of the input feature map into three equal parts, resulting in three components: Q, K, and V. The weights are obtained by calculating the similarity between Q and K, and then the Sofmax layer is used to normalize the weights. The weighted sum of the weights and the V component is then used to obtain the attention feature map. Downsampling uses convolutional layers with a stride of 2, and the kernel size is 3*3 with a number of kernels of 32, 64, 128, and 256, respectively, with padding of 1. Upsampling uses bilinear interpolation with a factor of 2. The concatenation layer concatenates the two input feature maps along the channel dimension.

[0017] In one possible implementation, the kernel size of the three convolutional layers of the enhancement module is 3*3, with padding of 1 and stride of 1, and the number of kernels is 3, 3, and 16 respectively; the two upsampling operations both use bilinear interpolation with a sampling factor of 2; the negative slope parameter of the LeakyReLU activation layer is 0.1; the input of the transformation operation is a vector composed of noise level parameters corresponding to the low-resolution image, which is linearly transformed with trainable parameters that follow a normal distribution, with a mean and standard deviation of 0 and 0.01 respectively.

[0018] In one possible implementation, the model training steps specifically include: low-resolution image I LR Input the super-resolution model and obtain the super-resolution result G(I) LRThe super-resolution result is calculated to minimize the loss between the super-resolution result and the labeled high-resolution image. The network parameters are then trained by minimizing the loss using a gradient descent-based optimization method. The training ends when the loss decreases steadily or when the total number of training iterations is reached.

[0019] In one possible implementation, the loss function is optimized using the Adam optimization algorithm during training, with parameters β1 = 0.9, β2 = 0.99, and the initial learning rate set to 10. -4 The number of training iterations is set to 200 epochs. One epoch is when all the data in the training set are used to train the model once. The learning rate is reduced by 1 / 2 every 50 epochs.

[0020] In one possible implementation, the loss function uses the mean squared error function (MSE), as shown below:

[0021]

[0022] in, This represents the pixel value at position (x, y) of a sharp, high-resolution image. Represents low-resolution image I LR The pixel value at position (x,y) of the image after super-resolution by the super-resolution model G, where W and H are the width and height of the image, respectively.

[0023] The present invention offers the following advantages: It introduces a self-attention module, proposing a self-attention-based image super-resolution model. The model's input is a low-resolution image. A preprocessing module extracts features from the image. The self-attention unit in the preprocessing module acquires attention based on the global feature map itself, thereby enhancing useful features and weakening useless features, achieving feature filtering. A preliminary super-resolution image is obtained in the enhancement module, and finally, a residual generation module reconstructs it into high-frequency information, which is then fused with the preliminary super-resolution image to obtain a clearer super-resolution image. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the steps of the self-attention-based image super-resolution improvement method according to an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the image preprocessing module in the self-attention-based image super-resolution improvement method according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the DoubleConv2d module in the self-attention-based image super-resolution improvement method according to an embodiment of the present invention;

[0027] Figure 4This is a schematic diagram of the attention module in the self-attention-based image super-resolution improvement method according to an embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of the Self-Attention unit in the image super-resolution improvement method based on self-attention according to an embodiment of the present invention;

[0029] Figure 6 This is a structural diagram of the enhancement module in the self-attention-based image super-resolution improvement method according to an embodiment of the present invention;

[0030] Figure 7 This is a structural diagram of the residual generation module in the self-attention-based image super-resolution improvement method according to an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] See Figure 1 The diagram shows a flowchart of the self-attention-based image super-resolution improvement method according to an embodiment of the present invention, including the following steps:

[0033] S10, Data preparation steps: Construct an image super-resolution database, sample high-resolution image patches from high-definition images, and perform degradation processing on the sampled image patches to obtain low-resolution image patches. Use pairs of high-resolution image patches and low-resolution image patches as labels and inputs for training models.

[0034] S20, Image Preprocessing Module Design Steps: Construct a multi-scale image preprocessing module, including one 2D convolutional layer, nine DoubleConv2d structures, four attention modules, four downsampling layers, four upsampling layers, and four stitching layers. The input to the preprocessing module is a low-resolution image I. LR The output is the feature map X. pre_out ;

[0035] S30, Enhancement Module Design Steps: Construct a noise-level-based enhancement module, including 3 convolutional layers, 2 upsampling layers, 1 LeakyReLU activation layer, and 1 transform operation. The enhancement module will process the feature map X output by the preprocessing module. pre_out The process involves two branches. One branch convolves the feature maps into an image, which is then compared with a low-resolution image I. LROne approach involves fusing the feature maps before upsampling. Another approach is to first upsample the feature maps and then convolve them into an image. The results of these two processes are then fused to obtain a preliminary super-resolution result X. mid_out ;

[0036] S40, Residual Generation Module Design Steps: Construct a multi-scale residual generation module, including 9 DoubleConv2d structures, 4 attention modules, 4 downsampling layers, 4 upsampling layers, 4 concatenation layers, and 1 convolutional layer. The feature map X output by the enhancement module... enh_out The process involves operations including convolution, activation, downsampling, self-attention filtering, and upsampling. Then, the feature map is mapped to an image—a residual image containing high-frequency information—through a final convolutional layer. This residual image is then compared with the initial super-resolution result X. mid_out The results are fused to obtain the final super-resolution result G(I). LR );

[0037] S50, Model Training Steps: Connect the image preprocessing module, enhancement module, and reconstruction module to form a super-resolution model, and train the model.

[0038] In one embodiment of the present invention, an image super-resolution database is constructed, high-resolution image patch sampling is performed on high-resolution images, and the sampled image patches are degraded to obtain low-resolution image patches. The dataset further includes Vimeo, RealSR, REDS, DIV2K, and Flickr2K image databases, as well as collected 4K high-resolution data. Vimeo, REDS, and the collected 4K high-resolution data are video datasets. Each scene sequence consists of multiple consecutive images, and only one frame is taken from each scene sequence. Image patch sampling is performed on each high-resolution image, with a patch size of 256*256*3. The image patch serves as the label for super-resolution training. Degradation processing, including blurring, downsampling, adding noise, and compression, is performed on the high-resolution 256*256*3 image patch to obtain the corresponding 128*128*3 low-resolution image patch.

[0039] Vimeo, REDS, and the collected 4K data are video datasets, with each scene sequence consisting of multiple consecutive frames. Each database provides high-resolution images, and some databases provide corresponding low-resolution images. This invention uses only high-resolution images, with the low-resolution images generated from the high-resolution images. The content, size, and number of images contained in each database are different. For Vimeo, REDS, and the collected 4K datasets, to avoid image content duplication, only one frame is taken from each scene sequence. Image patch sampling is performed on each high-resolution image, with each patch size being 256*256*3. To increase the data volume, multiple image patches are randomly sampled from large-sized images. For example, four 256*256*3 image patches are randomly sampled from a high-resolution image with a size of 1920*1080*3, and these image patches serve as labels for super-resolution training. The high-resolution 256*256*3 image patches undergo degradation processing, including blurring, downsampling, adding noise, and compression, to obtain corresponding 128*128*3 low-resolution image patches. The process involves several steps: blurring, adding Gaussian noise, and compression. Blur processing uses different blur kernels, including isotropic Gaussian blur kernels, isotropic Gaussian blur kernels, and motion blur kernels. Downsampling involves performing a 0.5x bicubic interpolation on the image. Noise addition adds random Gaussian noise to the image, with noise levels ranging from 0 to 1. This noise level, along with its corresponding low-resolution image, serves as input to the super-resolution model. Compression involves JPEG encoding and decoding the image with random quality values ​​ranging from 50 to 90.

[0040] In one embodiment of the present invention, see Figure 2 As shown, the image preprocessing module specifically includes one 2D convolutional layer, nine DoubleConv2d modules, four Attention Block modules, four downsampling layers, four upsampling layers, and four stitching layers, which process the low-resolution image I... LR Input preprocessing module, output feature map X pre_out The first convolutional layer has a kernel size of 3*3, 16 kernels, padding of 1, and a stride of 1. The DoubleConv2d module structure is as follows: Figure 3 As shown, the network includes two 2D convolutional layers and two LeakyReLU activation layers. The kernel size of each convolutional layer is 3*3, with the following kernel numbers: 16, 32, 64, 128, 256, 128, 64, 32, 16, padding of 1, and stride of 1. The negative slope parameter of the LeakyReLU activation layers is 0.1. The network processes low-resolution images I through convolutional layers. LRFeature extraction and activation layers map these features to a non-linear space to improve the model's expressive power. Downsampling layers are used to obtain feature maps at different scales. These downsampling layers all use convolutional layers with a stride of 2, kernel sizes of 3*3, and kernel numbers of 32, 64, 128, and 256, with padding of 1. Upsampling employs bilinear interpolation with a multiplier of 2, corresponding one-to-one with the downsampling layers to gradually restore the original size. The concatenation layer concatenates the two input feature maps along the channel dimension, achieving cross-layer information fusion. The Attention Block module structure is as follows: Figure 4 As shown, Reshape represents the reshape layer, Conv1d represents a 1D convolutional layer, and Self-Attention represents... Figure 5 The Self-Attention unit structure is shown. The Attention Block module consists of two 1D convolutional layers, four reshape layers, and one Self-Attention unit. The reshape layer reshapes the feature map into the shape required for the next layer.

[0041] Self-Attention unit structure as follows Figure 5 As shown, Split represents a split layer, and Softmax represents a softmax layer. This represents matrix multiplication, consisting of one Split layer and one Sofmax layer. The Split layer divides the first dimension of the input feature map into three equal parts, resulting in three components: Q, K, and V. The global attention weights of the feature map are obtained by calculating the similarity between Q and K. The weights are normalized using a softmax layer, and then the attention weights are summed with V using a weighted average. The output feature map is the feature map that has been filtered by attention.

[0042] In one embodiment of the present invention, the network structure of the enhancement module is as follows: Figure 6 As shown, it includes 3 convolutional layers, 2 upsampling layers, 1 LeakyReLU activation layer, and 1 transform operation. The enhancement module takes the feature map X output by the preprocessing module... pre_out The process involves two branches. One branch convolves the feature maps into an image, which is then compared with a low-resolution image I. LR One approach involves fusing the feature maps before upsampling. Another approach is to first upsample the feature maps and then convolve them into an image. The results of these two processes are then fused to obtain a preliminary super-resolution result X. mid_out The convolutional layers all have a kernel size of 3x3, padding of 1, stride of 1, and a kernel count of 3. The upsampling layers use bilinear interpolation with a sampling factor of 2. Then, the initial super-resolution result X is... mid_out Convolution is performed to remap the image to the feature space, thus reducing the noise level parameter P corresponding to the low-resolution image. noiseA linear transformation is performed to obtain high-frequency information related to noise. This information is then fused with the feature map to obtain an enhanced feature map. Finally, a LeakyReLU activation layer is used to perform a non-linear transformation on the feature map, outputting the feature map X. enh_out The negative slope parameter of the activation layer is 0.1. Figure 6 Middle I LR Represents a low-resolution image, X mid_out P represents the initial super-resolution result output from the enhancement module. noise X represents the noise level parameter corresponding to the low-resolution image, and Transform represents the transformation operation performed on the noise level parameter. enh_out This indicates the output of the enhancement module.

[0043] In one embodiment of the present invention, the network structure of the residual generation module is as follows: Figure 7 As shown, except for the last convolutional layer, the settings are the same as those of the preprocessing module. The residual generation module generates the feature map X output by the enhancement module. enh_out A series of operations, including convolution, activation, downsampling, self-attention filtering, and upsampling, are performed. Then, the feature map is mapped to an image, i.e., a residual image containing high-frequency information, through the final convolutional layer. This residual image is then compared with the initial super-resolution result X. mid_out The results are fused to obtain the final super-resolution result G(I). LR The kernel size of the last convolutional layer is 3*3, the stride is 1, the padding is 1, and the number of padding layers is 3.

[0044] In one embodiment of the present invention, the image preprocessing module, the enhancement module, and the reconstruction module are connected to form a super-resolution model, and the training models are connected to form a super-resolution model, such as... Figure 1 As shown. Low-resolution image I LR Input the super-resolution model and obtain the super-resolution result G(I) LR The network calculates the loss between the super-resolution result and the labeled high-resolution image, and minimizes this loss using a gradient descent-based optimization method to train the network parameters. Training ends when the loss descent trend stabilizes or the set total number of training iterations is reached. During training, the Adam optimization algorithm is used to optimize the loss function, with parameters β1 = 0.9, β2 = 0.99, and an initial learning rate of 10. -4 The training iterations are set to 200 epochs, with each epoch training the model once on all data in the training set. The learning rate is halved every 50 epochs. The loss function used is the mean squared error (MSE), as shown below:

[0045]

[0046] in, This represents the pixel value at position (x, y) of a sharp, high-resolution image. Represents low-resolution image I LR The pixel value at position (x,y) of the image after super-resolution by the super-resolution model G, where W and H are the width and height of the image, respectively.

[0047] This invention proposes a self-attention-based image super-resolution improvement method, which utilizes a deep learning-based image super-resolution model. This model achieves image super-resolution through an image preprocessing module, an enhancement module, and a residual generation module. To address the local feature extraction characteristics of convolution, a self-attention unit is introduced to enhance the model's expressive power and improve super-resolution performance by filtering global image features.

[0048] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. An improved image super-resolution method based on self-attention, characterized in that, Includes the following steps: Data preparation steps: Construct an image super-resolution database, sample high-resolution image patches from high-resolution images, and perform degradation processing on the sampled image patches to obtain low-resolution image patches. Use pairs of high-resolution image patches and low-resolution image patches as labels and inputs for training models. Image preprocessing module design steps: Construct a multi-scale image preprocessing module, including one 2D convolutional layer, nine DoubleConv2d structures, four attention modules, four downsampling layers, four upsampling layers, and four stitching layers. The input to the preprocessing module is a low-resolution image. The output is a feature map. The attention module structure includes two 1D convolutional layers, four reshape layers, and one self-attention module. The self-attention module includes one split layer and one softmax layer. The split layer divides the first dimension of the input feature map into three equal parts, resulting in three components: Q, K, and V. The weights are obtained by calculating the similarity between Q and K. The softmax layer is then used to normalize the weights, and the weighted sum is calculated with the V component to obtain the attention feature map. Enhancement module design steps: Construct a noise-level-based enhancement module, including 3 convolutional layers, 2 upsampling layers, 1 LeakyReLU activation layer, and 1 transform operation. The enhancement module will process the feature map output by the preprocessing module. The process involves two branches. One branch convolves the feature maps into an image, which is then compared with a low-resolution image. One approach involves fusing the feature maps before upsampling. Another approach is to first upsample the feature maps and then convolve them into an image. The results of these two processes are then fused to obtain a preliminary super-resolution result. Then, the preliminary super-resolution results were analyzed. Convolution is performed to remap the image to the feature space, thus removing the noise level parameters corresponding to the low-resolution image. A linear transformation is performed to obtain high-frequency information related to noise. This information is then fused with the feature map to obtain an enhanced feature map. Finally, a LeakyReLU activation layer is used to perform a non-linear transformation on the feature map to output the final feature map. ; The design steps for the residual generation module are as follows: Construct a multi-scale residual generation module, including 9 DoubleConv2d structures, 4 attention modules, 4 downsampling layers, 4 upsampling layers, 4 concatenation layers, and 1 convolutional layer. This process is repeated for each feature map output by the enhancement module. The process involves operations including convolution, activation, downsampling, self-attention filtering, and upsampling. Then, the feature map is mapped to an image—a residual image containing high-frequency information—through a final convolutional layer, which is compared with the initial super-resolution result. The results are then fused to obtain the final super-resolution result. ; Model training steps: Connect the preprocessing module, enhancement module, and reconstruction module to form a super-resolution model, and then train the model.

2. The image super-resolution improvement method based on self-attention as described in claim 1, characterized in that, The construction of the image super-resolution database, which involves high-resolution image patch sampling of high-definition images and degradation processing of the sampled image patches to obtain low-resolution image patches, further includes: the dataset includes Vimeo, RealSR, REDS, DIV2K, and Flickr2K image databases, as well as collected 4K high-definition data, where Vimeo, REDS, and the collected 4K high-definition data are video datasets, each scene sequence consists of multiple consecutive frames of images, and only one frame of each scene sequence is taken; image patch sampling is performed on each high-resolution image, with an image patch size of 256*256*3, and the image patch serves as the label for super-resolution training; degradation processing, including blurring, downsampling, adding noise, and compression, is performed on the high-definition 256*256*3 image patch to obtain the corresponding 128*128*3 low-resolution image patch.

3. The image super-resolution improvement method based on self-attention as described in claim 2, characterized in that, The blurring degradation process involves blurring the image using different blur kernels, including isotropic Gaussian blur kernels, isotropic Gaussian blur kernels, and motion blur kernels.

4. The image super-resolution improvement method based on self-attention as described in claim 2, characterized in that, Downsampling degradation is achieved by performing a 0.5x bicubic interpolation on the image.

5. The image super-resolution improvement method based on self-attention as described in claim 2, characterized in that, Degradation by adding noise involves adding Gaussian noise of random noise levels to the image. Its noise level value ranges from 0 to 1.

6. The image super-resolution improvement method based on self-attention as described in claim 1, characterized in that, The image preprocessing module further includes: the first convolutional layer has a kernel size of 3*3, a kernel size of 16, padding of 1, and a stride of 1; the DoubleConv2d module structure includes two 2D convolutional layers and two LeakyReLU activation layers. The kernel size of each convolutional layer is 3*3, with kernel sizes of 16, 32, 64, 128, 256, 128, 64, 32, and 16 respectively, padding of 1, and a stride of 1; the negative slope parameter of the LeakyReLU activation layer is 0.1; downsampling uses convolutional layers with a stride of 2, kernel sizes of 3*3, with kernel sizes of 32, 64, 128, and 256 respectively, and padding of 1; upsampling uses bilinear interpolation with a multiplier of 2; the concatenation layer concatenates the two input feature maps along the channel dimension.

7. The image super-resolution improvement method based on self-attention as described in claim 1, characterized in that, The enhancement module has three convolutional layers with kernel sizes of 3*3, padding of 1, stride of 1, and kernel numbers of 3, 3, and 16, respectively. The two upsampling operations use bilinear interpolation with a sampling factor of 2. The negative slope parameter of the LeakyReLU activation layer is 0.

1. The input to the transformation operation is a vector of noise level parameters corresponding to the low-resolution image, which is linearly transformed with trainable parameters that follow a normal distribution with a mean and standard deviation of 0 and 0.01, respectively.

8. The image super-resolution improvement method based on self-attention as described in claim 1, characterized in that, The specific steps for training the model include: low-resolution images Input the super-resolution model and obtain the super-resolution result. The super-resolution result and the label high-resolution image are calculated, and the loss is minimized using a gradient descent-based optimization method to train the network parameters. The model training ends when the loss decreases steadily or the total number of training iterations is reached.

9. The image super-resolution improvement method based on self-attention as described in claim 8, characterized in that, During training, the loss function is optimized using the Adam optimization algorithm, where the parameters... The initial learning rate is set to The number of training iterations is set to 200 epochs. One epoch is when all the data in the training set are used to train the model once. The learning rate is reduced by 1 / 2 every 50 epochs.

10. The image super-resolution improvement method based on self-attention as described in claim 9, characterized in that, The loss function uses the mean squared error (MSE) function, as shown below: in, Indicates a clear, high-resolution image at location pixel values, Represents low-resolution images The image after super-resolution by the super-resolution model G is located at... The pixel values ​​are W and H, which are the width and height of the image, respectively.

Images