Transformer-based remote sensing image super-resolution reconstruction method

Abstract

The application relates to a kind of remote sensing image super-resolution reconstruction methods based on Transformer, belong to artificial intelligence surveying and mapping field.It includes the following steps: obtaining remote sensing image constructs MR-super dataset, and is divided into training set and test set, the image in training set is carried out data enhancement, obtains the remote sensing image after enhancement;The remote sensing image super-resolution reconstruction model based on Transformer is constructed, the model includes feature extraction module, translation window self-attention module, dynamic Transformer module, global feature fusion module, super-resolution reconstruction module;Image in training set is used to train the model;The model is trained and optimized by loss function, obtains the trained model;Image in test set is input into the trained model, and the super-resolution remote sensing image is obtained.The application can improve the effect of remote sensing image super-resolution reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence surveying, specifically relating to a method for super-resolution reconstruction of remote sensing images based on Transformer. Background Technology

[0002] Remote sensing mapping technology, as an important means of acquiring surface information, has significant application value in fields such as land resource management, environmental monitoring, disaster assessment, and agricultural surveying. However, limited by factors such as the hardware limitations of imaging equipment, flight altitude, and satellite orbit transmission bandwidth, the resolution of remote sensing images is usually low, making it unable to fully meet the needs of tasks requiring high detail accuracy. For example, in urban planning, building monitoring, or geological hazard identification, low-resolution images may not accurately depict the boundaries and texture details of targets, thus affecting the accuracy of subsequent data analysis. Therefore, how to reconstruct high-resolution images from low-resolution remote sensing images using algorithmic methods has become an important research topic in current remote sensing technology.

[0003] Traditional super-resolution methods for remote sensing images mainly include interpolation-based algorithms (such as bicubic interpolation) and algorithms based on sparse representation or dictionary learning. While these methods can improve image resolution to some extent, their limitations in modeling image features result in insufficient performance in reconstructing image texture details and complex scene features. In recent years, with the rapid development of deep learning technology, super-resolution reconstruction methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods learn the mapping relationship between low-resolution and high-resolution images through end-to-end training, showing significant performance improvements compared to traditional methods. However, the local receptive field of CNNs limits their ability to capture global contextual information, especially for remote sensing images containing complex terrain or large-scale scenes, making it difficult to fully recover high-quality global details.

[0004] As a deep learning architecture based on self-attention mechanisms, Transformer has made groundbreaking progress in natural language processing and computer vision in recent years. Its global modeling capabilities enable it to capture long-range dependencies and global feature descriptions in large-scale image processing, making it particularly suitable for handling scenes with complex textures and spatial distributions, such as remote sensing mapping. Applying Transformer to super-resolution reconstruction of remote sensing images can effectively overcome the limitations of traditional CNN models, improving the detail representation of reconstructed images through more accurate global context modeling, and providing more reliable technical support for high-precision applications of remote sensing images. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a Transformer-based method for super-resolution reconstruction of remote sensing images.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A Transformer-based method for super-resolution reconstruction of remote sensing images includes the following steps:

[0008] S1. Acquire remote sensing images to construct the MR-super dataset, and divide the dataset into training and test sets. Perform data augmentation on the images in the training set to obtain augmented remote sensing images.

[0009] S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, which includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; train the model using enhanced remote sensing images from the training set;

[0010] S3. Train and optimize the model using the loss function to obtain a well-trained model;

[0011] S4. Images in the test set are super-resolution reconstructed using a trained model to obtain super-resolution remote sensing images.

[0012] Further, step S1 includes:

[0013] The MR-super dataset includes the Sentinel-2 open-source dataset and UAV remote sensing image data; the UAV remote sensing image data consists of data on various land features such as urban buildings, farmland, water bodies, mountains, and forests collected by UAVs.

[0014] Data augmentation operations are performed on the remote sensing images in the training set of the MR-super dataset to obtain augmented remote sensing images. The data augmentation operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

[0015] Furthermore, in step S2, the enhanced remote sensing image Texture features are obtained after the texture feature extraction module. It includes the following steps:

[0016] The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer, and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel, and a concatenation layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer.

[0017] Enhanced remote sensing images The input is processed through the initial convolutional decomposition layer to obtain the first convolutional feature. The second convolutional features are obtained after passing through the second convolutional kernel. The first convolutional feature Second convolution features The first splicing feature is obtained by using the concat method after the splicing layer. The first splicing feature In the input feature stacking layer, the global context vector is obtained after passing through the global average pooling layer. Global context vector After compression and activation through the first and second fully connected layers, the activation vector is obtained. The first convolutional feature Second convolution features and activation vector After the texture modeling layer, the final texture features are obtained. The formula is expressed as follows:

[0018] ,

[0019] in, This indicates that the assigned feature maps will be added pixel by pixel.

[0020] Furthermore, in step S2, texture features The correlation features are obtained through the correlation feature extraction module. It includes the following steps:

[0021] The correlation feature extraction module includes a first convolutional layer, a ReLU activation function, an average pooling layer, a second convolutional layer, a ReLU activation function, a first max pooling layer, a third convolutional layer, a ReLU activation function, a second max pooling layer, a fourth convolutional layer, a ReLU activation function, and a third max pooling layer.

[0022] Texture features The input is fed into the relevance feature extraction module, where it passes through the first convolutional layer, the ReLU activation function, and the average pooling layer to obtain the first relevance feature. First correlation feature The second correlation feature is obtained after passing through the second convolutional layer, the ReLU activation function, and the first max pooling layer. ; the first correlation feature Second correlation characteristics After summing, the input to the third convolutional layer, ReLU activation function, and second max pooling layer yields the third correlation feature. ; the second correlation feature and the third correlation feature After addition, the data passes through a fourth convolutional layer, a ReLU activation function, and a third max-pooling layer to obtain the correlation features. .

[0023] Furthermore, in step S2, the correlation features The input is fed into the translation window self-attention module to obtain the window expansion feature. It includes the following steps:

[0024] The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer, and a window smoothing and blending layer;

[0025] The correlation feature The input is fed into the translation window self-attention module, and after passing through the translation window decomposition layer, it is divided into blocks according to the window size of 8×8 to obtain block features. The local window spatial attention computation layer calculates the spatial attention weights for each window. The formula is expressed as follows:

[0026] ,

[0027] Where M represents the window size, Representing block features The eigenvalue at position (i,j) in the middle. Representing block features The first position bias information at position (i,j) in the middle, where σ represents the activation function. Representing block features The eigenvalue at position (p, q) in the middle. Representing block features The first position offset information at position (p,q) in the middle; the first position offset information is composed of block features The spatial attention weights are obtained by taking the square root of the feature values ​​at the corresponding positions. Block features applied within the window The window features are obtained by performing weighted summation through element-wise multiplication. The formula is expressed as follows:

[0028] ,

[0029] The window features After a window smoothing and blending layer, the correlation features are restored. In the same dimensional space, the reconstructed feature map is obtained. Then reconstruct the feature map Perform convolution operations to obtain window expansion features. .

[0030] Furthermore, in step S2, the correlation features The input is fed into the dynamic Transformer module to obtain multi-scale dynamic features. It includes the following steps:

[0031] Correlation characteristics By projecting through convolutional kernels of different scales, a multi-scale feature representation query matrix is ​​obtained. Key matrix and value matrix The formula is expressed as follows:

[0032] , , ,

[0033] in, , , The convolutional kernels are 3×3, 5×5, and 7×7 respectively. Indicates convolution operation; query matrix Key matrix and value matrix The query matrix of uniform dimension is obtained by performing convolution operations with 1×1 convolution kernels. A unified dimension key matrix and value matrices of uniform dimension The formula is expressed as follows:

[0034] , , ,

[0035] in, This represents a 1×1 convolution kernel operation;

[0036] Through correlation characteristics Generate dynamic attention weights from global information The formula is expressed as follows:

[0037] ,

[0038] in, Indicates the correlation features Perform global average pooling. The learnable scaling matrix is ​​initialized using the Xavier initialization method on a 7×7 random matrix. Represents dynamic attention weights;

[0039] based on and Calculate intra-scale attention weights The formula is expressed as follows:

[0040] ,

[0041] in, This represents the attention distribution between positions i and j. express The feature vector at position i, express The feature vector at position j, Indicates to Line-by-line elements and After multiplying each column element by the exponential function with the natural constant e as the base, the results of each exponential function are summed; the in-scale attention weights are then applied. Applied to value matrix To obtain reconstructed features The formula is expressed as follows:

[0042] ,

[0043] Through dynamic attention weights Reconstructed features The weighted fusion is performed, and then the fused features are subjected to a Leaky ReLU activation function for nonlinear mapping to obtain multi-scale dynamic features. The formula is expressed as follows:

[0044] .

[0045] Furthermore, in step S2, the window expansion feature... and multi-scale dynamic features The input is fed into the global feature fusion module to obtain global features. It includes the following steps:

[0046] Window expansion features and multi-scale dynamic features The input is fed into the global feature fusion module, where it is first concatenated along the channel dimension to obtain the second concatenated feature. The second splicing feature The first fused feature is obtained by performing convolution mapping using a 3×3 convolution kernel. The first fusion feature Global average pooling is used to obtain global features. .

[0047] Furthermore, in step S2, texture features and global features The summation is input into the super-resolution reconstruction module to obtain the super-resolution remote sensing image. It includes the following steps:

[0048] The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the first deconvolution layer has a 3×3 kernel, a stride of 2, and 64 output channels; the second deconvolution layer has a 5×5 kernel, a stride of 2, and 3 output channels.

[0049] The texture features and global features The second fused feature is obtained by directly adding pixels one by one. The second fusion feature The input is fed into the super-resolution reconstruction module, where it undergoes upsampling and reconstruction of the feature map through the first and second deconvolution layers to obtain the super-resolution remote sensing image. .

[0050] Furthermore, step S3 includes:

[0051] Calculating super-resolution remote sensing images using the mean squared error loss function and remote sensing images The loss is expressed by the following formula:

[0052] ,

[0053] in, This represents the mean squared error loss, where N represents the total number of pixels in the image. This represents the value of the g-th pixel in the reconstructed super-resolution image. This represents the g-th pixel value of the real image; the Adam optimizer is used to update the model's parameters.

[0054] The advantages of this invention are:

[0055] The dynamic Transformer module in this invention introduces a learnable scale mapping matrix and a multi-scale feature fusion mechanism. These innovative designs enable the model to adaptively adjust the weights of features at different scales based on the input features. Traditional super-resolution algorithms typically rely on fixed-scale features or simple convolutional layers, while this invention, by dynamically adjusting scale weights, effectively enhances the expressive power for complex scenes. Through multi-scale feature fusion, the model can fully utilize the global and local information of low-resolution images while adaptively enhancing image detail. Dynamically adjusting scale weights allows the model to select the most suitable scale for different scenes and image content, thereby improving the quality and accuracy of image reconstruction.

[0056] This invention introduces a translation window self-attention mechanism, enabling the model to capture long-range dependencies between different locations while avoiding the excessive computational complexity of traditional self-attention mechanisms. By combining translation windows and self-attention, the model can perform local attention modeling within each window and achieve cross-window information transfer through overlapping areas. This mechanism not only reduces computational overhead but also effectively improves image detail recovery while maintaining computational efficiency. It has significant advantages in image detail recovery by introducing local contextual information and cross-window information sharing, especially in remote sensing images containing complex textures and diverse features, effectively improving super-resolution results.

[0057] In this invention, the global feature fusion module weighted and fused features from different sources (such as texture features and global features) to ensure that both local texture details and global structural information are preserved in the image. This module combines the advantages of translation window self-attention and multi-scale features, ensuring the comprehensiveness and diversity of features. By comprehensively considering texture and global information, the global feature fusion module can more accurately restore image details and avoid possible blurring or distortion during reconstruction. Furthermore, the design of this module enables the model to take into account both local and global features, improving the model's multi-task learning ability, thus demonstrating good reconstruction results on various remote sensing images.

[0058] In summary, this invention, through innovative designs such as a dynamic Transformer module, a translation window self-attention mechanism, and a global feature fusion module, not only improves the effect of super-resolution reconstruction of remote sensing images and better adapts to different types of remote sensing image data, but also achieves breakthroughs in efficiency and accuracy. This method can adaptively extract key information from different scales and local regions, effectively recovering image details and providing higher-quality image support for remote sensing image analysis, demonstrating significant application value and broad prospects. In practical applications, the algorithm of this invention can improve image resolution, recover details, and enhance the accuracy of remote sensing image applications in environmental monitoring, urban planning, agricultural assessment, and other fields. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0060] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0061] Figure 2 This is a comparison chart of the reconstruction results of the method of the present invention and the bicubic interpolation method. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Example 1

[0064] In this embodiment, as Figure 1 As shown, a Transformer-based method for super-resolution reconstruction of remote sensing images is provided, and the specific steps include:

[0065] S1. Acquire remote sensing images to construct the MR-super dataset, and divide the dataset into training and test sets. Perform data augmentation on the images in the training set to obtain augmented remote sensing images.

[0066] Specifically, the MR-super dataset includes the Sentinel-2 open-source dataset and UAV remote sensing image data; the UAV remote sensing image data consists of data on various land features such as urban buildings, farmland, water bodies, mountains, and forests collected by UAVs.

[0067] Data augmentation operations are performed on the remote sensing images in the training set of the MR-super dataset to obtain augmented remote sensing images. The data augmentation operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

[0068] S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, which includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; train the model using enhanced remote sensing images from the training set;

[0069] S21. Enhanced remote sensing image Texture features are obtained after the texture feature extraction module. Specifically:

[0070] The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer, and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel, and a concatenation layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer.

[0071] Enhanced remote sensing images The input is processed through the initial convolutional decomposition layer to obtain the first convolutional feature. The second convolutional features are obtained after passing through the second convolutional kernel. The first convolutional feature Second convolution features The first splicing feature is obtained by using the concat method after the splicing layer. The first splicing feature In the input feature stacking layer, the global context vector is obtained after passing through the global average pooling layer. Global context vector After compression and activation through the first and second fully connected layers, the activation vector is obtained. The first convolutional feature Second convolution features and activation vector After the texture modeling layer, the final texture features are obtained. The formula is expressed as follows:

[0072] ,

[0073] in, This indicates that the assigned feature maps will be added pixel by pixel.

[0074] S22. Texture Features The correlation features are obtained through the correlation feature extraction module. Specifically:

[0075] The correlation feature extraction module includes a first convolutional layer, a ReLU activation function, an average pooling layer, a second convolutional layer, a ReLU activation function, a first max pooling layer, a third convolutional layer, a ReLU activation function, a second max pooling layer, a fourth convolutional layer, a ReLU activation function, and a third max pooling layer.

[0076] Texture features The input is fed into the relevance feature extraction module, where it passes through the first convolutional layer, the ReLU activation function, and the average pooling layer to obtain the first relevance feature. First correlation feature The second correlation feature is obtained after passing through the second convolutional layer, the ReLU activation function, and the first max pooling layer. ; the first correlation feature Second correlation characteristics After summing, the input to the third convolutional layer, ReLU activation function, and second max pooling layer yields the third correlation feature. ; the second correlation feature and the third correlation feature After addition, the data passes through a fourth convolutional layer, a ReLU activation function, and a third max-pooling layer to obtain the correlation features. .

[0077] S23. Correlation Characteristics The input is fed into the translation window self-attention module to obtain the window expansion feature. Specifically:

[0078] The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer, and a window smoothing and blending layer;

[0079] The correlation feature The input is fed into the translation window self-attention module, and after passing through the translation window decomposition layer, it is divided into blocks according to the window size of 8×8 to obtain block features. The local window spatial attention computation layer calculates the spatial attention weights for each window. The formula is expressed as follows:

[0080] ,

[0081] Where M represents the window size, Representing block features The eigenvalue at position (i,j) in the middle. Representing block features The first position bias information at position (i,j) in the middle, where σ represents the activation function. Representing block features The eigenvalue at position (p, q) in the middle. Representing block features The first position offset information at position (p,q) in the middle; the first position offset information is composed of block features The spatial attention weights are obtained by taking the square root of the feature values ​​at the corresponding positions. Block features applied within the window The window features are obtained by performing weighted summation through element-wise multiplication. The formula is expressed as follows:

[0082] ,

[0083] The window features After a window smoothing and blending layer, the correlation features are restored. In the same dimensional space, the reconstructed feature map is obtained. Then reconstruct the feature map Perform convolution operations to obtain window expansion features. .

[0084] S24. Correlation Characteristics The input is fed into the dynamic Transformer module to obtain multi-scale dynamic features. Specifically:

[0085] Correlation characteristics By projecting through convolutional kernels of different scales, a multi-scale feature representation query matrix is ​​obtained. Key matrix and value matrix The formula is expressed as follows:

[0086] , , ,

[0087] in, , , The convolutional kernels are 3×3, 5×5, and 7×7 respectively. Indicates convolution operation; query matrix Key matrix and value matrix The query matrix of uniform dimension is obtained by performing convolution operations with 1×1 convolution kernels. A unified dimension key matrix and value matrices of uniform dimension The formula is expressed as follows:

[0088] , , ,

[0089] in, This represents a 1×1 convolution kernel operation;

[0090] Through correlation characteristics Generate dynamic attention weights from global information The formula is expressed as follows:

[0091] ,

[0092] in, Indicates the correlation features Perform global average pooling. The learnable scaling matrix is ​​initialized using the Xavier initialization method on a 7×7 random matrix. Represents dynamic attention weights;

[0093] based on and Calculate intra-scale attention weights The formula is expressed as follows:

[0094] ,

[0095] in, This represents the attention distribution between positions i and j. express The feature vector at position i, express The feature vector at position j, Indicates to Line-by-line elements and After multiplying each column element by the exponential function with the natural constant e as the base, the results of each exponential function are summed; the in-scale attention weights are then applied. Applied to value matrix To obtain reconstructed features The formula is expressed as follows:

[0096] ,

[0097] Through dynamic attention weights Reconstructed features The weighted fusion is performed, and then the fused features are subjected to a Leaky ReLU activation function for nonlinear mapping to obtain multi-scale dynamic features. The formula is expressed as follows:

[0098] .

[0099] S25. Window Expansion Feature and multi-scale dynamic features The input is fed into the global feature fusion module to obtain global features. Specifically:

[0100] Window expansion features and multi-scale dynamic features The input is fed into the global feature fusion module, where it is first concatenated along the channel dimension to obtain the second concatenated feature. The second splicing feature The first fused feature is obtained by performing convolution mapping using a 3×3 convolution kernel. The first fusion feature Global average pooling is used to obtain global features. .

[0101] S26. Texture Features and global features The summation is input into the super-resolution reconstruction module to obtain the super-resolution remote sensing image. Specifically:

[0102] The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the first deconvolution layer has a 3×3 kernel, a stride of 2, and 64 output channels; the second deconvolution layer has a 5×5 kernel, a stride of 2, and 3 output channels.

[0103] The texture features and global features The second fused feature is obtained by directly adding pixels one by one. The second fusion feature The input is fed into the super-resolution reconstruction module, where it undergoes upsampling and reconstruction of the feature map through the first and second deconvolution layers to obtain the super-resolution remote sensing image. .

[0104] S3. Train and optimize the model using the loss function to obtain a well-trained model;

[0105] Specifically, the super-resolution remote sensing image is calculated using the mean squared error loss function. and remote sensing images The loss is expressed by the following formula:

[0106]

[0107] in, This represents the mean squared error loss, where N represents the total number of pixels in the image. This represents the value of the g-th pixel in the reconstructed super-resolution image. This represents the g-th pixel value of the real image; the Adam optimizer is used to update the model's parameters.

[0108] S4. Images in the test set are super-resolution reconstructed using a trained model to obtain super-resolution remote sensing images.

[0109] Example 2

[0110] To verify the effectiveness and practicality of the method of this invention, extensive experiments and comparative tests were conducted to verify the performance of the Transformer-based remote sensing image super-resolution reconstruction algorithm on standard datasets. The following is a detailed description of the experimental data comparison and reconstruction result comparison, demonstrating the excellent performance of the method of this invention in remote sensing image super-resolution reconstruction.

[0111] We conducted our tests on the publicly available Google Earth high-resolution remote sensing image dataset. Each image was downsampled to different low-resolution scales for both training and testing. Evaluation metrics used in the experiments included peak signal-to-noise ratio, structural similarity, and natural image quality assessment. We compared our method with several classic remote sensing image super-resolution reconstruction methods, as shown in Table 1:

[0112] Table 1. Comparison of experimental results between the method of this invention and other traditional remote sensing image super-resolution reconstruction methods.

[0113]

[0114] As can be seen from the table, the method of this invention outperforms traditional remote sensing image super-resolution reconstruction methods in all evaluation metrics, especially showing significant improvements in peak signal-to-noise ratio and structural similarity, indicating that the method of this invention has significant advantages in image quality, structural similarity, and detail restoration. Furthermore, the decrease in the natural image quality evaluation metric also indicates that the generated image is more natural, with reduced artifacts and noise.

[0115] To visually demonstrate the advantages of the method of this invention, we present image reconstruction results of the method of this invention and the bicubic interpolation method on the 2m / 8m Optical Satellites dataset. For example... Figure 2 As shown, the left image represents the reconstructed image using the bicubic interpolation method, while the right image represents the reconstructed image using the method of this invention. Because the bicubic interpolation method relies solely on pixel interpolation, it results in blurred image edges and loss of detail, especially in complex texture areas. The method of this invention achieves more refined image detail recovery, particularly excelling in complex terrain features, with clearer edges and more accurate texture restoration.

[0116] By comparing the experimental and reconstruction results, it can be concluded that the method of this invention significantly improves the super-resolution reconstruction quality of remote sensing images. Compared with existing traditional methods, this invention shows significant advantages in peak signal-to-noise ratio, structural similarity, and natural image quality evaluation metrics, and also has higher accuracy in restoring image details and complex textures. Through innovative designs such as multi-scale feature fusion, translation window self-attention mechanism, and dynamic Transformer module, this invention can effectively improve the reconstruction effect of remote sensing images, especially in remote sensing images with complex details, such as urban buildings, vegetation, and coastlines, achieving high restoration accuracy and demonstrating strong practicality and broad application prospects.

[0117] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Transformer-based method for super-resolution reconstruction of remote sensing images, characterized in that, Includes the following steps: S1. Acquire remote sensing images to construct the MR-super dataset, and divide the dataset into training and test sets. Perform data augmentation on the images in the training set to obtain augmented remote sensing images. S2. Construct a Transformer-based remote sensing image super-resolution reconstruction model, which includes a feature extraction module, a translation window self-attention module, a dynamic Transformer module, a global feature fusion module, and a super-resolution reconstruction module; the feature extraction module includes a texture feature extraction module and a correlation feature extraction module; train the model using enhanced remote sensing images from the training set; The specific process is as follows: Enhanced remote sensing images Texture features are obtained after the texture feature extraction module. ; Texture features The correlation features are obtained through the correlation feature extraction module. ; Correlation characteristics The input is fed into the translation window self-attention module to obtain the window expansion feature. ; Correlation characteristics The input is fed into the dynamic Transformer module to obtain multi-scale dynamic features. Window expansion feature and multi-scale dynamic features The input is fed into the global feature fusion module to obtain global features. ; Texture features and global features The summation is input into the super-resolution reconstruction module to obtain the super-resolution remote sensing image. ; S3. Train and optimize the model using the loss function to obtain a well-trained model; S4. Images in the test set are super-resolution reconstructed using a trained model to obtain super-resolution remote sensing images.

2. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 1, characterized in that, Step S1 specifically includes: The MR-super dataset includes the Sentinel-2 open-source dataset and UAV remote sensing image data; the UAV remote sensing image data consists of data on various land features such as urban buildings, farmland, water bodies, mountains, and forests collected by UAVs. Data augmentation operations are performed on the remote sensing images in the training set of the MR-super dataset to obtain augmented remote sensing images. The data augmentation operations include random rotation, random brightness adjustment, adding Gaussian noise, and adding Gaussian blur.

3. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 2, characterized in that, In step S2, the enhanced remote sensing image Texture features are obtained after the texture feature extraction module. Specifically: The texture feature extraction module includes a preliminary convolution decomposition layer, a feature stacking layer, and a texture modeling layer; the preliminary convolution decomposition layer includes a first convolution kernel, a second convolution kernel, and a concatenation layer; the feature stacking layer includes a global average pooling layer, a first fully connected layer, and a second fully connected layer. Enhanced remote sensing images The input is processed through the initial convolutional decomposition layer to obtain the first convolutional feature. The second convolutional features are obtained after passing through the second convolutional kernel. The first convolutional feature Second convolution features The first splicing feature is obtained by using the concat method after the splicing layer. The first splicing feature In the input feature stacking layer, the global context vector is obtained after passing through the global average pooling layer. Global context vector After compression and activation through the first and second fully connected layers, the activation vector is obtained. The first convolutional feature Second convolution features and activation vector After the texture modeling layer, the final texture features are obtained. The formula is expressed as follows: ， in, This indicates that the assigned feature maps will be added pixel by pixel.

4. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 3, characterized in that, In step S2, texture features The correlation features are obtained through the correlation feature extraction module. Specifically: The correlation feature extraction module includes a first convolutional layer, a ReLU activation function, an average pooling layer, a second convolutional layer, a ReLU activation function, a first max pooling layer, a third convolutional layer, a ReLU activation function, a second max pooling layer, a fourth convolutional layer, a ReLU activation function, and a third max pooling layer. Texture features The input is fed into the relevance feature extraction module, where it passes through the first convolutional layer, the ReLU activation function, and the average pooling layer to obtain the first relevance feature. First correlation feature The second correlation feature is obtained after passing through the second convolutional layer, the ReLU activation function, and the first max pooling layer. ; the first correlation feature Second correlation feature After summing, the input to the third convolutional layer, ReLU activation function, and second max pooling layer yields the third correlation feature. ; the second correlation feature and the third correlation feature After addition, the data passes through a fourth convolutional layer, a ReLU activation function, and a third max-pooling layer to obtain the correlation features. .

5. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 4, characterized in that, In step S2, correlation features The input is fed into the translation window self-attention module to obtain the window expansion feature. Specifically: The translation window self-attention module includes a translation window decomposition layer, a local window spatial attention calculation layer, and a window smoothing and blending layer; The correlation feature The input is fed into the translation window self-attention module, and after passing through the translation window decomposition layer, it is divided into blocks according to the window size of 8×8 to obtain block features. The local window spatial attention computation layer calculates the spatial attention weights for each window. The formula is expressed as follows: ， Where M represents the window size, Representing block features The eigenvalue at position (i,j) in the middle. Representing block features The first position bias information at position (i,j) in the middle, where σ represents the activation function. Representing block features The eigenvalue at position (p, q) in the middle. Representing block features The first position offset information at position (p,q) in the middle; the first position offset information is composed of block features The spatial attention weights are obtained by taking the square root of the feature values ​​at the corresponding positions. Block features applied within the window The window features are obtained by performing weighted summation through element-wise multiplication. The formula is expressed as follows: ， The window features After a window smoothing and blending layer, the correlation features are restored. The same dimensional space yields the reconstructed feature map. Then reconstruct the feature map Perform convolution operations to obtain window expansion features. .

6. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 5, characterized in that, In step S2, correlation features The input is fed into the dynamic Transformer module to obtain multi-scale dynamic features. Specifically: Correlation characteristics By projecting through convolutional kernels of different scales, a multi-scale feature representation query matrix is ​​obtained. Key matrix and value matrix The formula is expressed as follows: ， ， ， in, , , The convolutional kernels are 3×3, 5×5, and 7×7 respectively. Indicates convolution operation; query matrix Key matrix and value matrix The query matrix of uniform dimension is obtained by performing convolution operations with 1×1 convolution kernels. A unified dimension key matrix and value matrices of uniform dimension The formula is expressed as follows: ， ， ， in, This represents a 1×1 convolution kernel operation; Through correlation characteristics Generate dynamic attention weights from global information The formula is expressed as follows: ， in, Indicates the correlation characteristics Perform global average pooling. The learnable scaling matrix is ​​initialized using the Xavier initialization method on a 7×7 random matrix. Represents dynamic attention weights; based on and Calculate intra-scale attention weights The formula is expressed as follows: ， in, This represents the attention distribution between positions i and j. express The feature vector at position i, express The feature vector at position j, Indicates to Line-by-line elements and After multiplying each column element by the exponential function with the natural constant e as the base, the results of each exponential function are summed; the in-scale attention weights are then applied. Applied to value matrix To obtain reconstructed features The formula is expressed as follows: , Through dynamic attention weights Reconstruction features Weighted fusion is performed, and then the fused features are nonlinearly mapped using the LeakyReLU activation function to obtain multi-scale dynamic features. The formula is expressed as follows: 。 7. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 6, characterized in that, In step S2, the window expansion feature and multi-scale dynamic features The input is fed into the global feature fusion module to obtain global features. Specifically: Window expansion features and multi-scale dynamic features The input is fed into the global feature fusion module, where it is first concatenated along the channel dimension to obtain the second concatenated feature. The second splicing feature The first fused feature is obtained by performing convolution mapping using a 3×3 convolution kernel. The first fusion feature Global average pooling is used to obtain global features. .

8. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 7, characterized in that, In step S2, texture features and global features The summation is input into the super-resolution reconstruction module to obtain the super-resolution remote sensing image. Specifically: The super-resolution reconstruction module includes a first deconvolution layer and a second deconvolution layer; the first deconvolution layer has a 3×3 kernel, a stride of 2, and 64 output channels; the second deconvolution layer has a 5×5 kernel, a stride of 2, and 3 output channels. The texture features and global features The second fused feature is obtained by directly adding pixels one by one. The second fusion feature The input is fed into the super-resolution reconstruction module, where it undergoes upsampling and reconstruction of the feature map through the first and second deconvolution layers to obtain the super-resolution remote sensing image. .

9. The method for super-resolution reconstruction of remote sensing images based on Transformer according to claim 8, characterized in that, Step S3 specifically includes: Calculating super-resolution remote sensing images using the mean squared error loss function and remote sensing images The loss is expressed by the following formula: in, This represents the mean squared error loss, where N represents the total number of pixels in the image. This represents the value of the g-th pixel in the reconstructed super-resolution image. This represents the g-th pixel value of the real image; the Adam optimizer is used to update the model's parameters.

Images