Transform-based remote sensing image super-resolution reconstruction method

Abstract

The invention relates to a remote sensing image super-resolution reconstruction method based on Transform, and belongs to the field of artificial intelligence surveying and mapping. The method comprises the following steps: acquiring a remote sensing image to construct an MR-super data set, dividing the MR-super data set into a training set and a test set, and performing data enhancement on the image in the training set to obtain an enhanced remote sensing image; constructing a remote sensing image super-resolution reconstruction model based on Transform, wherein the model comprises a feature extraction module, a translation window self-attention module, a dynamic Transform module, a global feature fusion module and a super-resolution reconstruction module; training the model by using images in a training set; performing training optimization on the model through the loss function to obtain a trained model; and inputting an image in a test set into the trained model to obtain a super-resolution remote sensing image. According to the invention, the super-resolution reconstruction effect of the remote sensing image can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence surveying and mapping, and specifically relates to a remote sensing image super-resolution reconstruction method based on Transformer. Background Art

[0002] As an important means of obtaining surface information, remote sensing mapping technology has important application value in the fields of land and resources management, environmental monitoring, disaster assessment, agricultural measurement, etc. However, due to the hardware limitations of imaging equipment, flight altitude, satellite orbit transmission bandwidth and other factors, the resolution of remote sensing mapping images is usually low, which makes it unable to fully meet the task requirements with high detail accuracy. For example, in urban planning, building monitoring or geological disaster identification, low-resolution images may not accurately depict the boundaries and texture details of the target, thus affecting the accuracy of subsequent data analysis. Therefore, how to reconstruct low-resolution remote sensing mapping images into high-resolution images through algorithmic means has become an important topic in current remote sensing technology research.

[0003] Traditional remote sensing image super-resolution methods mainly include algorithms based on interpolation (such as bicubic interpolation) and algorithms based on sparse representation or dictionary learning. Although these methods can improve image resolution to a certain extent, they are insufficient in reconstructing image texture details and complex scene features due to their limited ability to model image features. In recent years, with the rapid development of deep learning technology, super-resolution reconstruction methods based on convolutional neural networks (CNNs) have gradually become mainstream. This type of method learns the mapping relationship from low resolution to high resolution of images through end-to-end training, showing significant performance improvement compared to traditional methods. However, the local receptive field of CNN limits its ability to capture global contextual information, especially for remote sensing images containing complex terrain or large-scale scenes, it is still difficult to fully restore high-quality global details.

[0004] As a deep learning architecture based on the self-attention mechanism, Transformer has made breakthrough progress in natural language processing and computer vision in recent years. Its global modeling capability enables it to capture long-distance dependencies and global feature descriptions in large-scale image processing, and is particularly suitable for processing scenes with complex textures and spatial distributions such as remote sensing mapping. Applying Transformer to super-resolution reconstruction of remote sensing images can effectively make up for the limitations of traditional CNN models, improve the ability to express details of reconstructed images through more accurate global context modeling, and provide more reliable technical support for high-precision applications of remote sensing images. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a remote sensing image super-resolution reconstruction method based on Transformer.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: A remote sensing image super-resolution reconstruction method based on Transformer, comprising the following steps: S1. Acquire remote sensing images to construct an MR-super dataset, and divide the dataset into a training set and a test set. Perform data enhancement on the images in the training set to obtain enhanced remote sensing images. S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, the model includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; the model is trained using the enhanced remote sensing images in the training set; S3. Train and optimize the model through the loss function to obtain a trained model; S4. The images in the test set are super-resolved through the trained model to obtain super-resolution remote sensing images.

[0007] Further, step S1 includes: The MR-super dataset includes the Sentinel-2 open source dataset and UAV remote sensing image data; the UAV remote sensing image data is a variety of landform data collected by UAVs, including urban buildings, farmland, water bodies, mountains and forests; Perform data enhancement on the remote sensing images of the training set in the MR-super dataset to obtain enhanced remote sensing images The data enhancement operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

[0008] Furthermore, in step S2, the enhanced remote sensing image The texture features are obtained through the texture feature extraction module , including the following steps: The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel and a splicing layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer; Enhanced remote sensing image Input the initial convolution decomposition layer and obtain the first convolution feature through the first convolution kernel. , the second convolution feature is obtained through the second convolution kernel , the first convolutional feature and the second convolutional feature After the concatenation layer, the concat method is used to obtain the first concatenation feature ; The first splicing feature In the input feature stacking layer, the global context vector is obtained through the global average pooling layer , the global context vector After compression and activation by the first and second fully connected layers, the activation vector ; The first convolution feature , the second convolution feature and activation vector After the texture modeling layer, the final texture features are obtained , the formula is as follows: Z , in, It means adding the assigned feature maps pixel by pixel.

[0009] Furthermore, in step S2, the texture feature The correlation features are obtained through the correlation feature extraction module , including the following steps: The correlation feature extraction module includes a first convolution layer, a ReLU activation function, an average pooling layer, a second convolution layer, a ReLU activation function, a first maximum pooling layer, a third convolution layer, a ReLU activation function, a second maximum pooling layer, a fourth convolution layer, a ReLU activation function, and a third maximum pooling layer; Texture features Input into the correlation feature extraction module, and obtain the first correlation feature through the first convolution layer, ReLU activation function and average pooling layer ; The first correlation feature After the second convolution layer, ReLU activation function and the first maximum pooling layer, the second correlation feature is obtained. ; The first correlation feature and the second correlation feature After addition, the third convolutional layer, ReLU activation function and the second maximum pooling layer are input to obtain the third correlation feature. ; The second correlation feature and the third correlation feature After addition, the fourth convolution layer, ReLU activation function and the third maximum pooling layer are used to obtain the correlation feature. .

[0010] Furthermore, in step S2, the correlation feature Input into the translation window self-attention module to obtain the window expansion feature , including the following steps: The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer and a window smoothing fusion layer; The correlation characteristics The input is sent to the translation window self-attention module, and the translation window decomposition layer is divided into blocks according to the window size of 8×8 to obtain the block features. , the local window spatial attention calculation layer calculates the spatial attention weight of each window , the formula is as follows: , in, M Indicates the window size, Represents block features The eigenvalue at position (i,j) in Represents block features The first position bias information at position (i, j) in the middle, σ represents the activation function, Represents block features The eigenvalue at position (p,q) in Represents block features The first position bias information at position (p, q) in the middle; the first position bias information is composed of block features The eigenvalue of the corresponding position is obtained by taking the square root; the spatial attention weight Block features applied to the window , weighted summation is performed by element-by-element multiplication to obtain the window feature , the formula is as follows: , The window features After the window smoothing fusion layer, it is restored to the correlation feature The same dimensional space, get the reconstructed feature map , and then reconstruct the feature map Perform convolution operation to obtain window expansion features .

[0011] Furthermore, in step S2, the correlation feature Input into the dynamic Transformer module to obtain multi-scale dynamic features , including the following steps: Correlation characteristics Through projection of convolution kernels of different scales, a multi-scale feature representation query matrix is ​​obtained , key matrix and the value matrix , the formula is as follows: , in, , , The convolution kernels are 3×3, 5×5, and 7×7 in size respectively, ∗ represents the convolution operation; the query matrix , key matrix and the value matrix The query matrix of uniform dimension is obtained by performing convolution operation with 1×1 convolution kernel respectively. , a key matrix of uniform dimension and a matrix of values ​​of uniform dimension , the formula is as follows: in, Represents a 1×1 convolution kernel operation; By correlation feature The global information generates dynamic attention weights , the formula is as follows: , in, Represents the correlation characteristics Perform global average pooling, represents the learnable scale mapping matrix, and uses the Xavier initialization method to initialize a 7×7 random matrix; represents the dynamic attention weight; based on and Calculate the in-scale attention weight , the formula is as follows: , in, (i,j) represents the attention distribution between positions i and j, express The eigenvector at position i, express The eigenvector at position j is, Express The row-by-row elements and After multiplying the column elements, we find the exponential function with the natural constant e as the base, and finally sum the results of each exponential function; Apply to the value matrix , and obtain the reconstructed features , the formula is as follows: , Through dynamic attention weights Reconstruction features Perform weighted fusion, and then use the Leaky ReLU activation function to perform nonlinear mapping on the fused features to obtain multi-scale dynamic features , the formula is as follows: .

[0012] Furthermore, in step S2, the window expansion feature and multi-scale dynamic features Input into the global feature fusion module to obtain the global feature , including the following steps: Window extension features and multi-scale dynamic features Input into the global feature fusion module, first splicing is performed on the channel dimension to obtain the second splicing feature , the second splicing feature After convolution mapping with a 3×3 convolution kernel, the first fusion feature is obtained. , the first fusion feature After global average pooling, we get the global features .

[0013] Furthermore, in step S2, the texture feature and global features Add the input to the super-resolution reconstruction module to obtain a super-resolution remote sensing image , including the following steps: The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the convolution kernel of the first deconvolution layer is 3×3, the step size is 2, and the output channel is 64; the convolution kernel of the second deconvolution layer is 5×5, the step size is 2, and the output channel is 3; The texture feature and global features Directly add pixel by pixel to get the second fusion feature , the second fusion feature The image is input to the super-resolution reconstruction module, and the feature map is upsampled and reconstructed through the first and second deconvolution layers to obtain a super-resolution remote sensing image. .

[0014] Furthermore, step S3 includes: Computing super-resolution remote sensing images through mean square error loss function and remote sensing images The loss is expressed as follows: , in, represents the mean square error loss, Nrepresents the total number of pixels in the image, represents the g-th pixel value of the reconstructed super-resolution image, Represents the g-th pixel value of the real image; use the Adam optimizer to update the parameters of the model.

[0015] The advantages of the present invention are: The dynamic Transformer module in the present invention introduces a learnable scale mapping matrix and a multi-scale feature fusion mechanism. These innovative designs enable the model to adaptively adjust the weights of features of different scales according to the input features. Traditional super-resolution algorithms usually only rely on fixed-scale features or simple convolutional layers, while the present invention can effectively improve the ability to express complex scenes by dynamically adjusting the scale weights. Through multi-scale feature fusion, the model can make full use of the global and local information of low-resolution images, while adaptively enhancing the details of the image. Dynamic adjustment of scale weights enables the model to select the most appropriate scale in different scenes and image content, thereby improving the quality and accuracy of image reconstruction.

[0016] The present invention introduces a translation window self-attention mechanism, which enables the model to capture long-range dependencies between different positions, while avoiding the problem of high computational complexity in the traditional self-attention mechanism. Through the combination of translation window and self-attention, the model can perform local attention modeling in each window, and realize cross-window information transfer through the overlapping parts between windows. This mechanism not only reduces the computational overhead, but also effectively improves the image detail recovery ability while ensuring computational efficiency. It has significant advantages in image detail recovery by introducing local context information and cross-window information sharing, especially in scenes such as remote sensing images that contain complex textures and diverse objects, and can effectively improve the super-resolution effect.

[0017] In the present invention, the global feature fusion module weightedly fuses features from different sources (such as texture features and global features) to ensure that both local texture details and global structural information are retained in the image. This module combines the advantages of translation window self-attention and multi-scale features to ensure the comprehensiveness and diversity of features. The global feature fusion module can more accurately restore image details and avoid possible blur or distortion during reconstruction by comprehensively considering texture and global information. In addition, the design of this module enables the model to take into account both local and global features, improves the model's multi-task learning ability, and thus can show good reconstruction effects on various remote sensing images.

[0018] In summary, the present invention not only improves the effect of super-resolution reconstruction of remote sensing images, and can better adapt to different types of remote sensing image data, but also achieves breakthroughs in efficiency and accuracy through innovative designs such as dynamic Transformer module, translation window self-attention mechanism and global feature fusion module. This method can adaptively extract key information from different scales and local areas, effectively restore image details, provide higher quality image support for remote sensing image analysis, and has significant application value and broad prospects. In practical applications, the algorithm of the present invention can improve image resolution, restore details, and improve the application accuracy of remote sensing images in the fields of environmental monitoring, urban planning, agricultural assessment, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0020] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 This is a comparison diagram of the reconstruction results of the method of the present invention and the bicubic interpolation method. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example 1 In this embodiment, Figure 1 As shown in FIG. 1 , a remote sensing image super-resolution reconstruction method based on Transformer is provided, and the specific steps include: S1. Acquire remote sensing images to construct an MR-super dataset, and divide the dataset into a training set and a test set. Perform data enhancement on the images in the training set to obtain enhanced remote sensing images. Specifically, the MR-super dataset includes the Sentinel-2 open source dataset and UAV remote sensing image data; the UAV remote sensing image data is a variety of land feature data collected by UAVs, including urban buildings, farmland, water bodies, mountains and forests; Perform data enhancement on the remote sensing images of the training set in the MR-super dataset to obtain enhanced remote sensing images The data enhancement operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

[0023] S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, the model includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; the model is trained using the enhanced remote sensing images in the training set; S21. Enhanced remote sensing image The texture features are obtained through the texture feature extraction module , specifically: The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel and a splicing layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer; Enhanced remote sensing image Input the initial convolution decomposition layer and obtain the first convolution feature through the first convolution kernel. , the second convolution feature is obtained through the second convolution kernel , the first convolutional feature and the second convolutional feature After the concatenation layer, the concat method is used to obtain the first concatenation feature ; The first splicing feature In the input feature stacking layer, the global context vector is obtained through the global average pooling layer , the global context vector After compression and activation by the first and second fully connected layers, the activation vector ; The first convolution feature , the second convolution feature and activation vector After the texture modeling layer, the final texture features are obtained , the formula is as follows: Z , in, It means adding the assigned feature maps pixel by pixel.

[0024] S22. Texture features The correlation features are obtained through the correlation feature extraction module , specifically: The correlation feature extraction module includes a first convolution layer, a ReLU activation function, an average pooling layer, a second convolution layer, a ReLU activation function, a first maximum pooling layer, a third convolution layer, a ReLU activation function, a second maximum pooling layer, a fourth convolution layer, a ReLU activation function, and a third maximum pooling layer; Texture features Input into the correlation feature extraction module, and obtain the first correlation feature through the first convolution layer, ReLU activation function and average pooling layer ; The first correlation feature After the second convolution layer, ReLU activation function and the first maximum pooling layer, the second correlation feature is obtained. ; The first correlation feature and the second correlation feature After addition, the third convolutional layer, ReLU activation function and the second maximum pooling layer are input to obtain the third correlation feature. ; The second correlation feature and the third correlation feature After addition, the fourth convolution layer, ReLU activation function and the third maximum pooling layer are used to obtain the correlation feature. .

[0025] S23. Correlation characteristics Input into the translation window self-attention module to obtain the window expansion feature , specifically: The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer and a window smoothing fusion layer; The correlation characteristics The input is sent to the translation window self-attention module, and the translation window decomposition layer is divided into blocks according to the window size of 8×8 to obtain the block features. , the local window spatial attention calculation layer calculates the spatial attention weight of each window , the formula is as follows: , in, M Indicates the window size, Represents block features The eigenvalue at position (i,j) in Represents block features The first position bias information at position (i, j) in the middle, σ represents the activation function, Represents block features The eigenvalue at position (p,q) in Represents block features The first position bias information at position (p, q) in the middle; the first position bias information is composed of block features The eigenvalue of the corresponding position is obtained by taking the square root; the spatial attention weight Block features applied to the window , weighted summation is performed by element-by-element multiplication to obtain the window feature , the formula is as follows: , The window features After the window smoothing fusion layer, it is restored to the correlation feature The same dimensional space, get the reconstructed feature map , and then reconstruct the feature map Perform convolution operation to obtain window expansion features .

[0026] S24. Correlation characteristics Input into the dynamic Transformer module to obtain multi-scale dynamic features , specifically: Correlation characteristics Through projection of convolution kernels of different scales, a multi-scale feature representation query matrix is ​​obtained , key matrix and the value matrix , the formula is as follows: , in, , , The convolution kernels are 3×3, 5×5, and 7×7 in size respectively, ∗ represents the convolution operation; the query matrix , key matrix and the value matrix The query matrix of uniform dimension is obtained by performing convolution operation with 1×1 convolution kernel respectively. , a key matrix of uniform dimension and a matrix of values ​​of uniform dimension , the formula is as follows: in, Represents a 1×1 convolution kernel operation; By correlation feature The global information generates dynamic attention weights , the formula is as follows: , in, Represents the correlation characteristics Perform global average pooling, represents the learnable scale mapping matrix, and uses the Xavier initialization method to initialize a 7×7 random matrix; represents the dynamic attention weight; based on and Calculate the in-scale attention weight , the formula is as follows: , in, (i,j) represents the attention distribution between positions i and j, express The eigenvector at position i, express The eigenvector at position j is, Express The row-by-row elements and After multiplying the column elements, we find the exponential function with the natural constant e as the base, and finally sum the results of each exponential function; Apply to the value matrix , and obtain the reconstructed features , the formula is as follows: , Through dynamic attention weights Reconstruction features Perform weighted fusion, and then use the Leaky ReLU activation function to perform nonlinear mapping on the fused features to obtain multi-scale dynamic features , the formula is as follows: .

[0027] S25. Window extension feature and multi-scale dynamic features Input into the global feature fusion module to obtain the global feature , specifically: Window extension features and multi-scale dynamic features Input into the global feature fusion module, first splicing is performed on the channel dimension to obtain the second splicing feature , the second splicing feature After convolution mapping with a 3×3 convolution kernel, the first fusion feature is obtained. , the first fusion feature After global average pooling, we get the global features .

[0028] S26. Texture features and global features Add the input to the super-resolution reconstruction module to obtain a super-resolution remote sensing image , specifically: The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the convolution kernel of the first deconvolution layer is 3×3, the step size is 2, and the output channel is 64; the convolution kernel of the second deconvolution layer is 5×5, the step size is 2, and the output channel is 3; The texture feature and global features Directly add pixel by pixel to get the second fusion feature , the second fusion feature The image is input to the super-resolution reconstruction module, and the feature map is upsampled and reconstructed through the first and second deconvolution layers to obtain a super-resolution remote sensing image. .

[0029] S3. Train and optimize the model through the loss function to obtain a trained model; Specifically, the super-resolution remote sensing image is calculated by the mean square error loss function and remote sensing images The loss is expressed as follows: , in, represents the mean square error loss, N represents the total number of pixels in the image, represents the g-th pixel value of the reconstructed super-resolution image, Represents the g-th pixel value of the real image; use the Adam optimizer to update the parameters of the model.

[0030] S4. The images in the test set are super-resolved through the trained model to obtain super-resolution remote sensing images.

[0031] Example 2 In order to verify the effectiveness and practicality of the method of the present invention, this paper conducted a large number of experiments and comparative tests to verify the performance of the Transformer-based remote sensing image super-resolution reconstruction algorithm on standard data sets. The following is a detailed description of the experimental data comparison and reconstruction result comparison, which demonstrates the excellent performance of the method of the present invention in super-resolution reconstruction of remote sensing images.

[0032] We tested it on the public Google Earth high-resolution remote sensing image dataset. Each image was downsampled to different low-resolution scales for training and testing. The evaluation indicators used in the experiment include peak signal-to-noise ratio, structural similarity, and natural image quality assessment. We compared the proposed method with several classic remote sensing image super-resolution reconstruction methods, see Table 1: Table 1 Comparison of experimental results between the proposed method and other traditional remote sensing image super-resolution reconstruction methods As can be seen from the table, the method of the present invention is superior to the traditional remote sensing image super-resolution reconstruction method in all evaluation indicators, especially in the peak signal-to-noise ratio and structural similarity indicators, indicating that the method of the present invention has significant advantages in image quality, structural similarity and detail recovery. In addition, the decrease in the natural image quality evaluation index also shows that the generated image is more natural with fewer artifacts and noise.

[0033] In order to intuitively demonstrate the advantages of the method of the present invention, we show the image reconstruction results of the method of the present invention and the bicubic interpolation method on the 2m / 8m Optical Satellites dataset. Figure 2 As shown, the left figure shows the reconstructed image of the bicubic interpolation method, and the right figure shows the reconstructed image of the method of the present invention. Since the bicubic interpolation method only relies on pixel interpolation, the image edges are blurred and details are lost, especially in complex texture areas. Through the method of the present invention, the image details are restored more finely, especially on complex objects, the edges are clearer, and the texture restoration is more accurate.

[0034] By comparing the experimental results and the reconstruction results, it can be concluded that the method of the present invention significantly improves the super-resolution reconstruction quality of remote sensing images. Compared with the existing traditional methods, the present invention shows obvious advantages in peak signal-to-noise ratio, structural similarity and natural image quality evaluation indicators, and has higher accuracy in restoring image details and complex textures. Through the innovative design of multi-scale feature fusion, translation window self-attention mechanism and dynamic Transformer module, the present invention can effectively improve the reconstruction effect of remote sensing images, especially in remote sensing images with complex details, such as urban buildings, vegetation and coastlines, with high restoration accuracy, strong practicality and broad application prospects.

[0035] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote sensing image super-resolution reconstruction method based on Transformer, characterized in that: The following steps are involved: S1. Acquire remote sensing images to construct an MR-super dataset, and divide the dataset into a training set and a test set. Perform data enhancement on the images in the training set to obtain enhanced remote sensing images. S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, the model includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; the model is trained using the enhanced remote sensing images in the training set; S3. Train and optimize the model through the loss function to obtain a trained model; S4. The images in the test set are super-resolved through the trained model to obtain super-resolution remote sensing images.

2. The remote sensing image super-resolution reconstruction method based on Transformer according to claim 1, characterized in that: Step S1 specifically includes: The MR-super dataset includes the Sentinel-2 open source dataset and UAV remote sensing image data; the UAV remote sensing image data is a variety of landform data collected by UAVs, including urban buildings, farmland, water bodies, mountains and forests; Perform data enhancement on the remote sensing images of the training set in the MR-super dataset to obtain enhanced remote sensing images The data enhancement operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

3. The remote sensing image super-resolution reconstruction method based on Transformer according to claim 2 is characterized in that: In step S2, the enhanced remote sensing image The texture features are obtained through the texture feature extraction module , specifically: The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel and a splicing layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer; Enhanced remote sensing image Input the initial convolution decomposition layer, and get the first convolution feature through the first convolution kernel , the second convolution feature is obtained through the second convolution kernel , the first convolutional feature and the second convolutional feature After the concatenation layer, the concat method is used to obtain the first concatenation feature ; The first splicing feature In the input feature stacking layer, the global context vector is obtained through the global average pooling layer , the global context vector After compression and activation by the first and second fully connected layers, the activation vector ; The first convolution feature , the second convolution feature and activation vector After the texture modeling layer, the final texture features are obtained , the formula is as follows: Z ， in, It means adding the assigned feature maps pixel by pixel.

4. The remote sensing image super-resolution reconstruction method based on Transformer according to claim 3 is characterized in that: In step S2, texture features The correlation features are obtained through the correlation feature extraction module , specifically: The correlation feature extraction module includes a first convolution layer, a ReLU activation function, an average pooling layer, a second convolution layer, a ReLU activation function, a first maximum pooling layer, a third convolution layer, a ReLU activation function, a second maximum pooling layer, a fourth convolution layer, a ReLU activation function, and a third maximum pooling layer; Texture features Input into the correlation feature extraction module, and obtain the first correlation feature through the first convolution layer, ReLU activation function and average pooling layer ; First correlation feature After the second convolution layer, ReLU activation function and the first maximum pooling layer, the second correlation feature is obtained. ; The first correlation feature and the second correlation feature After addition, the third convolutional layer, ReLU activation function and the second maximum pooling layer are input to obtain the third correlation feature. ; The second correlation feature and the third correlation feature After addition, the fourth convolution layer, ReLU activation function and the third maximum pooling layer are used to obtain the correlation feature. .

5. The Transformer-based remote sensing image super-resolution reconstruction method according to claim 4, characterized in that: In step S2, the correlation feature Input into the translation window self-attention module to obtain the window expansion feature , specifically: The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer and a window smoothing fusion layer; The correlation characteristics The input is sent to the translation window self-attention module, and the translation window decomposition layer is divided into blocks according to the window size of 8×8 to obtain the block features. , the local window spatial attention calculation layer calculates the spatial attention weight of each window , the formula is as follows: ， in, M Indicates the window size, Represents block features The eigenvalue at position (i,j) in Represents block features The first position bias information at position (i, j) in the middle, σ represents the activation function, Represents block features The eigenvalue at position (p,q) in Represents block features The first position bias information at position (p, q) in the middle; the first position bias information is composed of block features The eigenvalue of the corresponding position is obtained by taking the square root; the spatial attention weight Block features applied to the window , weighted summation is performed by element-by-element multiplication to obtain the window feature , the formula is as follows: ， The window features After the window smoothing fusion layer, it is restored to the correlation feature The same dimensional space, get the reconstructed feature map , and then reconstruct the feature map Perform convolution operation to obtain window expansion features .

6. The Transformer-based remote sensing image super-resolution reconstruction method according to claim 5, characterized in that: In step S2, the correlation feature Input into the dynamic Transformer module to obtain multi-scale dynamic features , specifically: Correlation characteristics Through projection of convolution kernels of different scales, a multi-scale feature representation query matrix is ​​obtained , key matrix and the value matrix , the formula is as follows: ， in, , , The convolution kernels are 3×3, 5×5, and 7×7 in scale respectively, ∗ represents the convolution operation; the query matrix , key matrix and the value matrix The query matrix of uniform dimension is obtained by performing convolution operation with 1×1 convolution kernel respectively. , a key matrix of uniform dimension and a matrix of values ​​of uniform dimension , the formula is as follows: in, Represents a 1×1 convolution kernel operation; By correlation feature The global information generates dynamic attention weights , the formula is as follows: ， in, Represents the correlation characteristics Perform global average pooling, represents the learnable scale mapping matrix, and uses the Xavier initialization method to initialize a 7×7 random matrix; represents the dynamic attention weight; based on and Calculate the in-scale attention weight , the formula is as follows: ， in, (i,j) represents the attention distribution between positions i and j, express The eigenvector at position i, express The eigenvector at position j is, Express The row-by-row elements and After multiplying the column elements, we find the exponential function with the natural constant e as the base, and finally sum the results of each exponential function; Apply to the value matrix , and obtain the reconstructed features , the formula is as follows: ， Through dynamic attention weights Reconstruction features Perform weighted fusion, and then use the Leaky ReLU activation function to perform nonlinear mapping on the fused features to obtain multi-scale dynamic features , the formula is as follows: 。 7. The remote sensing image super-resolution reconstruction method based on Transformer according to claim 6, characterized in that: In step S2, the window expansion feature and multi-scale dynamic features Input into the global feature fusion module to obtain the global feature , specifically: Window extension features and multi-scale dynamic features Input into the global feature fusion module, first splicing is performed on the channel dimension to obtain the second splicing feature , the second splicing feature After convolution mapping with a 3×3 convolution kernel, the first fusion feature is obtained. , the first fusion feature After global average pooling, we get the global features .

8. The Transformer-based remote sensing image super-resolution reconstruction method according to claim 7, characterized in that: In step S2, texture features and global features Add the input to the super-resolution reconstruction module to obtain a super-resolution remote sensing image , specifically: The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the convolution kernel of the first deconvolution layer is 3×3, the step size is 2, and the output channel is 64; the convolution kernel of the second deconvolution layer is 5×5, the step size is 2, and the output channel is 3; The texture feature and global features Directly add pixel by pixel to get the second fusion feature , the second fusion feature The image is input to the super-resolution reconstruction module, and the feature map is upsampled and reconstructed through the first and second deconvolution layers to obtain a super-resolution remote sensing image. .

9. The remote sensing image super-resolution reconstruction method based on Transformer according to claim 8, characterized in that: Step S3 specifically includes: Calculating super-resolution remote sensing images through mean square error loss function and remote sensing images The loss is expressed as follows: ， in, represents the mean square error loss, N represents the total number of pixels in the image, represents the g-th pixel value of the reconstructed super-resolution image, Represents the g-th pixel value of the real image; use the Adam optimizer to update the parameters of the model.

Images