Remote sensing image super-resolution reconstruction method based on Transform hopping network model

Abstract

The invention discloses a remote sensing image super-resolution reconstruction method based on a Transform hopping network model, and relates to the field of image super-resolution reconstruction, and the method comprises the steps: extracting shallow layer features through a shallow layer convolution module in a hopping network model, extracting deep layer features through an encoder, and carrying out the super-resolution reconstruction through a deep layer convolution module; the multi-scale joint connection module enhances correlation among deep features of different scales through an attention mechanism, convolution and normalization to obtain multi-scale coding features, so that the calculation amount is reduced, and data loss is prevented; and performing context reweighting on the multi-scale decoding features by an adaptive weighting strategy in the decoder to obtain decoding features, fusing the decoding features with the up-sampling result of the low-resolution image, obtaining final fusion features through convolution operation, generating a super-resolution image based on the final fusion features, enhancing the expression of deep features, and obtaining the super-resolution image based on the super-resolution image. And high-frequency signals and clear structural features in the reconstructed image are reserved, the texture definition is improved, and the structure recovery capability is enhanced.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution reconstruction technology, and specifically to a method for super-resolution reconstruction of remote sensing images based on the Transformer jump network model. Background Technology

[0002] Remote sensing imagery serves as the data support and foundation for remote sensing technology, providing abundant information for surface monitoring and finding wide application in disaster monitoring, geological surveys, environmental protection, and target detection. Super-resolution (SR) reconstruction is a crucial preprocessing step, capable of recovering high-resolution images from low-quality, low-resolution images for advanced image analysis tasks. Traditional methods for remote sensing image super-resolution can be broadly categorized into interpolation-based and reconstruction-based methods. Interpolation-based methods are simple to implement but can lead to blurred boundaries and loss of detail; reconstruction-based methods are susceptible to noisy information and incur significant data iteration and computational costs.

[0003] In recent years, with the development of deep learning modeling techniques, neural network-based super-resolution (SR) reconstruction technology has been gradually applied to the field of remote sensing images. Generative Adversarial Networks (GANs) in the field of image super-resolution express multi-scale features of remote sensing images through chained training, but they are computationally intensive, prone to data loss, and suffer from blurred image edges in high-resolution reconstructed images. The existing Swin Transformer SR method filters out noisy information in remote sensing images and performs super-resolution reconstruction by overlaying feature information from the remote sensing images; however, it suffers from high-frequency signal loss and structural feature distortion in high-resolution reconstructed images. Summary of the Invention

[0004] I. Technical problems to be solved This invention provides a method for super-resolution reconstruction of remote sensing images based on the Transformer jump network model, which can solve the problems of high-frequency signal loss and structural feature distortion in existing models.

[0005] II. Technical Solution To achieve the above objectives, the present invention provides a method for super-resolution reconstruction of remote sensing images based on a Transformer jump network model, comprising the following steps: A training data pair is formed by acquiring a low-resolution image and a corresponding high-resolution image of the same remote sensing observation area, and multiple training data pairs of the same remote sensing observation area are acquired as a training set. After obtaining the training set, the low-resolution images in each training data pair are input into the shallow convolutional module of the jump network model to extract shallow features, which include the structural information and local texture features of the image. The shallow features are input into the encoder of the jump network model for deep feature extraction, and the extracted deep features at different scales are input into the multi-scale joint connection module. The multi-scale joint connection module enhances the correlation between deep features at different scales through attention mechanism, convolution and normalization operation to obtain fused multi-scale encoded features. The multi-scale encoded features are fed into the decoder of the jump network model, and the multi-scale encoded features are reweighted using the adaptive weighting strategy in the decoder to obtain the scale-consistent fused decoded features. The decoded features output by the decoder are used to generate super-resolution images for training, and compared with the high-resolution images of the corresponding training data pairs. The loss function is calculated, and the training jump network model is optimized through backpropagation. The remote sensing image to be super-resolution reconstructed is input into the trained jump network model to generate the corresponding super-resolution image.

[0006] Furthermore, the shallow convolution module in the jump network model transforms the input low-resolution image into a pixel matrix. The shallow convolution module predefines multiple independent convolution kernels, which slide across the pixel matrix to cover local pixel regions. The pixel elements within each covered region are multiplied and summed. The calculated values ​​of the pixel elements within the covered region correspond to the shallow features extracted by the convolution kernel. Each convolution kernel extracts one shallow feature, thereby extracting shallow features from the low-resolution image.

[0007] Furthermore, the encoder in the jump network model includes a discrete wavelet transform module and a content feature extraction module, used to extract deep features at different scales, specifically: The discrete wavelet transform module is used to extract one low-frequency component and three high-frequency components from the input shallow features by frequency domain decomposition. The low-frequency component represents the structural information of the remote sensing image, and the three high-frequency components represent the texture information of the horizontal texture, vertical texture, and diagonal texture, respectively. The content feature extraction module encodes the low-frequency components and high-frequency components respectively. Based on the deep structural features extracted from the low-frequency components, it encodes the deep features of color distribution, contour shape and edge gradient of the image respectively. Based on the deep texture features extracted from the high-frequency components, it encodes the deep features of horizontal texture, vertical texture and diagonal texture respectively.

[0008] Furthermore, the multi-scale joint connection module receives deep features at different scales extracted by the encoder. The multi-scale joint connection module is used to unify the scale features of deep features at different scales, and then split them into multiple dimensions according to the linear projection of the scale features. The feature data of the same dimension is unified by normalization, and the convolution value of the deep feature pixel elements of each different feature dimension is calculated by convolution to obtain the fused multi-scale encoded features.

[0009] Furthermore, the decoder in the jump network model receives the multi-scale encoded features output by the encoder and sequentially inputs them into the max pooling layer and average pooling layer set by the decoder. The max pooling layer is used to extract local features reflecting the texture of local regions, and the average pooling layer is used to extract global features reflecting the overall statistical distribution, thereby outputting multi-scale decoded features.

[0010] Furthermore, the multi-scale decoding features are assigned different weights to the decoding features at different scales according to an adaptive weighting strategy. The decoder generates context resolution mapping weights for scale adjustment, forming decoding weights for multi-scale deep features. The decoder uses the decoding weights to remap the multi-scale decoding features to obtain scale-consistent fused decoding features.

[0011] Furthermore, during the training phase of the jump network model, the low-resolution image from the training data pair is input into the jump network model. After shallow feature extraction, deep feature extraction, multi-scale encoding and decoding, the scale-consistent fused decoding features output by the decoder are obtained. Then, the input low-resolution image is bilinearly upsampled to make its size consistent with the decoding features. The upsampled low-resolution image and the decoded features are concatenated in the same dimension to form a preliminary fusion feature. Then, the dimension of the preliminary fusion feature is adjusted by convolution operation to obtain a final fusion feature in the same dimension. The final fusion feature is then mapped to the training super-resolution image predicted by the jump network model.

[0012] Furthermore, during the training phase of the jump network model, the predicted super-resolution image for training is compared with the corresponding high-resolution image in the training data pair, a loss function is calculated, and the network parameters of the jump network model are updated by backpropagation based on the loss function; the training process is iterated and repeated so that after the jump network model is trained, it can generate a super-resolution image of the corresponding region when a low-resolution image is input.

[0013] III. Beneficial Effects: Compared with the prior art, this invention has the following beneficial effects: This application proposes a method for super-resolution reconstruction of remote sensing images based on the Transformer jump network model. The method extracts shallow features through the shallow convolution module of the jump network model, extracts deep features through the encoder of the jump network model, and enhances the correlation between deep features at different scales through attention mechanism, normalization and convolution to obtain fused multi-scale encoded features, thereby reducing the amount of computation and preventing data loss.

[0014] The adaptive weighting strategy in the decoder performs context reweighting on the multi-scale decoding features to obtain the decoding features. The decoding features output by the decoder are fused with the upsampling results of the low-resolution image. The final fused features are obtained through convolution operation, and a super-resolution image is generated based on the final fused features. This enhances the expression of deep features, preserves high-frequency signals and clear structural features in the reconstructed image, improves texture clarity, and enhances structural recovery capabilities. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the process of training the Transformer jump network model in the remote sensing image super-resolution reconstruction method based on the Transformer jump network model of the present invention. Figure 2 A schematic diagram of the framework structure of a remote sensing image super-resolution reconstruction method based on the Transformer jump network model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the discrete wavelet transform module in the jump network model of a remote sensing image super-resolution reconstruction method based on the Transformer jump network model provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the enhanced frequency domain sensing Transformer module in the hopping network model of a remote sensing image super-resolution reconstruction method based on the Transformer hopping network model provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of the adaptive weighting module in the Transformer jump network model of a remote sensing image super-resolution reconstruction method based on the Transformer jump network model provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Super-resolution (SR) reconstruction technology essentially infers high-resolution (HR) images from low-resolution (LR) images. Deep learning models are introduced into SR reconstruction technology. These models are primarily used as training sets by inputting a large number of low-resolution (LR) and high-resolution (HR) remote sensing image pairs. The deep learning model automatically learns which details are missing in the low-resolution image and which details are present in the corresponding high-resolution image. Finally, during inference, the deep learning model, based on the learned patterns, "fills in details, textures, and edges" in the low-resolution image to generate a higher-resolution image. In summary, SR reconstruction of remote sensing images is not simply a matter of upscaling. The deep learning model's role in SR reconstruction is not to directly extract features from the remote sensing image, but rather to learn the mapping relationship between low-resolution and high-resolution images, and based on this statistical regularity, to fill in the missing high-frequency textures and edge details in the remote sensing image.

[0018] Regarding the remote sensing image super-resolution reconstruction method based on the Transformer jump network model proposed in this embodiment of the invention: the processing flow of the input low-resolution remote sensing image by the jump network model is as follows: input low-resolution image, use encoder to extract features from low-resolution image, use decoder to reconstruct feature mapping relationship, and finally input the remote sensing image to be super-resolution reconstructed into the trained jump network model, and use the features output by decoder to fuse with the upsampling result of remote sensing image to generate the corresponding super-resolution image.

[0019] like Figure 1 As shown, this application utilizes a constructed jump network model for super-resolution reconstruction of remote sensing images, including the following steps: acquiring a low-resolution image of the same remote sensing observation area and a corresponding high-resolution image to form a training data pair; acquiring multiple training data pairs of the same remote sensing observation area as a training set; after obtaining the training set, inputting the low-resolution image in each training data pair into the shallow convolution module of the jump network model to extract shallow features, which include the structural information and local texture features of the image; inputting the shallow features into the encoder of the jump network model for deep feature extraction, and inputting the extracted deep features at different scales into the multi-scale joint connection module; the multi-scale joint connection module enhances the correlation between deep features at different scales through attention mechanisms, convolution, and normalization operations to obtain fused multi-scale encoded features; Furthermore, the multi-scale encoded features are fed into the decoder of the jump network model, and the multi-scale encoded features are reweighted using an adaptive weighting strategy in the decoder to obtain scale-consistent fused decoded features. The decoded features output by the decoder are used to generate training super-resolution images, which are compared with the high-resolution images of the corresponding training data pairs. The loss function is calculated, and the jump network model is optimized through backpropagation. The remote sensing image to be super-resolution reconstructed is input into the trained jump network model to generate the corresponding super-resolution image.

[0020] like Figure 2 As shown, the remote sensing image super-resolution reconstruction method based on the Transformer jump network model is as follows: A low-resolution image and its corresponding super-resolution image are set. Shallow features are obtained using convolution calculation. These shallow features are then processed by an encoder, which mainly consists of a Discrete Wavelet Transform (DWT) module and a Content-feature Extraction Module (CEM) to enhance the correlation of feature information at different levels. Next, another encoder fuses the scale features of the previous encoder and extracts depth features. Deep features at different scales are processed through a multi-head cross-attention mechanism, convolutional layers, and basic processing layers to output multi-scale encoded features. These multi-scale encoded features are then input into a decoder to obtain local and global features. The decoder includes a Content-feature Extraction Module (CEM) and an Enhanced Frequency-Aware Transformer module. The Transformer (FAT) module encapsulates complex texture details in its local features and preserves overall statistical distribution information in its global features. The decoder's content feature extraction module extracts multi-scale decoding features, and the enhanced frequency domain aware Transformer module generates feature maps for different frequencies based on these multi-scale decoding features. Finally, the extracted decoding features are input into the Self-Adapting Weighting Module (SAWM) to enhance the feature mapping relationship. The bilinear interpolation upsampling results input from the Self-Adapting Weighting Module are then fused to generate a super-resolution image.

[0021] Specifically, this invention acquires a low-resolution image of the same remote sensing observation area and a corresponding high-resolution image to form a training data pair, and acquires multiple training data pairs of the same remote sensing observation area as a training set; for each training data pair, the low-resolution image is used as the input of the model, and the high-resolution image is used as a supervision label to calculate the loss.

[0022] Furthermore, after obtaining the training set, the low-resolution images in each training data pair are input into the shallow convolutional module of the jump network model to extract shallow features. The shallow convolutional module in the jump network model transforms the input low-resolution images into a pixel matrix. Multiple independent convolutional kernels are predefined in the shallow convolutional module. The convolutional kernels slide on the pixel matrix to cover local pixel regions. The pixel elements in each covered region are multiplied and summed. The calculated values ​​of the pixel elements in the covered region correspond to the shallow features extracted by the convolutional kernel. Each convolutional kernel extracts one shallow feature, thereby extracting shallow features from the low-resolution image. The shallow features include the structural information and local texture features of the image.

[0023] In this embodiment, a low-resolution input image is defined as... , The training set is defined as H×W, where H×W represents the spatial size. The channel number is used to obtain shallow features through convolution operations, and the formula is as follows: in, It is a 3×3 convolution operation.

[0024] Furthermore, shallow features are input into the encoder of the jump network model for deep feature extraction, and the extracted deep features at different scales are input into the multi-scale joint connection module. The multi-scale joint connection module enhances the correlation between deep features at different scales through attention mechanism, convolution and normalization operations to obtain fused multi-scale encoded features. The encoder of the jump network model is equipped with a discrete wavelet transform module and a content feature extraction module to extract deep features at different scales. The discrete wavelet transform module is used to extract one low-frequency component and three high-frequency components from the input shallow features by frequency domain decomposition. The low-frequency component represents the structural information of the remote sensing image, and the three high-frequency components represent the texture information of the horizontal, vertical and diagonal textures, respectively. The content feature extraction module encodes the low-frequency and high-frequency components respectively. Based on the deep structural features extracted from the low-frequency components, it encodes the deep features of color distribution, contour shape and edge gradient of the image respectively. Based on the deep texture features extracted from the high-frequency components, it encodes the deep features of horizontal texture, vertical texture and diagonal texture respectively.

[0025] The multi-scale joint connection module receives deep features at different scales extracted by the encoder, unifies the scale features of deep features at different scales, and then splits them into multiple dimensions based on linear projection of the scale features. It unifies the feature data of the same dimension through normalization, and uses convolution to calculate the convolution value of the deep feature pixel elements of each different feature dimension to obtain the fused multi-scale encoded features.

[0026] In this embodiment, shallow features After passing through an encoder, which mainly consists of a content feature extraction module and a discrete wavelet transform module, the encoder reduces the spatial size to [value missing]. Let the shallow layer features be , The inputs are fed into a unified encoder, ensuring that features at different scales are fully computed. The specific computations of convolution and encoder are as follows: ; in, For 1×1 convolution operation, It is the i-th encoder, where i=2,3, if k=1, then i=2; if k=2, then i=3.

[0027] like Figure 3 As shown, the encoder improves the recovery of high-frequency details through a discrete wavelet transform module. It uses the Haar function to decompose the wavelet features, obtaining the high-frequency and low-frequency components of the image, thus avoiding frequency domain aliasing. The discrete wavelet transform is as follows: ; in, The wavelet transform function, These are vertical high-frequency, horizontal high-frequency, diagonal high-frequency, and low-frequency components, respectively. Three high-frequency components Connecting these elements, a convolutional neural network (CNN) unit employing 1×1 convolution, batch normalization (BN), and ReLU activation function enhances deep feature representation. The discrete wavelet module in the encoder recovers high-frequency information, and the output of the discrete wavelet module... for: ; in To standardize batch processing operations, For ReLU operation, For cascading operations, This is a 1×1 convolution operation.

[0028] A multi-head cross-attention mechanism is used to obtain features of the encoder output at three scales: horizontal texture, vertical texture, and diagonal texture. ; Scale characteristics After sequentially passing through 3×3 convolutions and two 1×1 convolutions, the fused multi-scale encoded features of the corresponding outputs are obtained. for: ; in, For multiple crossover attention blocks, To standardize layer operations, For the gate control layer, This is a 1×1 convolution operation.

[0029] Furthermore, the fused multi-scale encoded features are fed into the decoder of the jump network model. The decoder uses an adaptive weighting strategy to reweight the multi-scale encoded features, resulting in scale-consistent fused decoded features. The decoder in the jump network model receives the multi-scale encoded features output by the encoder and sequentially inputs them into the max pooling layer and average pooling layer set in the decoder. The max pooling layer is used to extract local features reflecting the texture of local regions, and the average pooling layer is used to extract global features reflecting the overall statistical distribution, thereby outputting multi-scale decoded features. According to the adaptive weighting strategy, different weights are assigned to the decoded features of different scales. The decoder generates context resolution mapping weights for scale adjustment, forming decoding weights for multi-scale deep features. The decoder uses the decoding weights to remap the multi-scale decoded features, resulting in scale-consistent fused decoded features.

[0030] The decoder comprises a content feature extraction module and an enhanced frequency-domain-aware Transformer module. The content feature extraction module captures the similarity of local textures and multi-scale decoding features that reflect the overall statistical distribution. The enhanced frequency-domain-aware Transformer module generates feature maps of different frequencies for the multi-scale decoding features through Fourier transform, reducing the number of input mapping channels for the multi-scale decoding features. It also transforms different frequency features from the spatial domain to the frequency domain through frequency domain transformation. According to an adaptive weighting strategy, different weights are assigned to the decoding features at different scales to generate context resolution mapping weights for scale adjustment, forming decoding weights for multi-scale deep features. The decoder uses the decoding weights to remap the multi-scale decoding features to obtain scale-consistent fused decoding features, enhancing the mapping of high-frequency texture decoding features.

[0031] In this embodiment, multi-scale coding features , The data is input into the corresponding decoder, which obtains local features and global features obtained through addition. The specific formula is as follows: ; ; ; in, This represents a 2×2 transposed convolutional layer with a stride of 2, used for upsampling. For decoder, These are local features and global features.

[0032] like Figure 4As shown, in this embodiment, the decoder recovers the boundary through an enhanced frequency-domain sensing Transformer module based on Fast Fourier Transform, with the input multi-scale two-dimensional features being... The output decoding features are FFT stands for Fast Fourier Transform, and IFFT stands for Inverse Fourier Transform. IFFT generates frequency domain feature maps through Fast Fourier Transform and uses 1×1 convolutions to reduce the number of channels in the input map beforehand, resulting in two-dimensional deep features. The Fourier transform is: ; in, and This represents the two-dimensional dimension indices of the input drawing. Represents frequency domain coordinates. It is the FFT function.

[0033] The Transformer mechanism is introduced to transform the frequency domain features into three different vectors through a linear transformation: query (Q), key (K), and value (V). ; ; in, , ,、 These are the projected weights of the query (Q), key (K), and value (V) vectors, respectively. It uses the FFT function to perform a dot product of Q and K, resulting in the transposed attention map A, from which the output features are obtained. for: ; ; in, It is the inverse function of FFT. It is a constant. It is the dot product of Q and K.

[0034] like Figure 5 As shown, in this embodiment, the adaptive weighting strategy in the decoder assigns different weights to decoding features of different scales at different decoding stages through the adaptive weighting strategy module. It unifies the scale features of deep features through an attention mechanism, and then splits them into multiple feature dimensions through linear projection. Feature information is trained based on the jump network model, enhancing the feature refinement effect of the jump network model and implementing multi-scale decoding weights. Defined as: ; in, This is a two-dimensional max pooling operation. This is a two-dimensional average pooling operation. For contextual information, To recover high-frequency information, the final adaptive weights are... It can be calculated as: ; in, for function.

[0035] Furthermore, the decoded features output by the decoder are used to generate training super-resolution images, which are then compared with the high-resolution images of the corresponding training data pairs. The loss function is calculated, and the training jump network model is optimized through backpropagation. The remote sensing image to be super-resolution reconstructed is input into the trained jump network model to generate the corresponding super-resolution image.

[0036] Specifically, during the training phase of the jump network model, the low-resolution image from the training data pair is input into the jump network model. After shallow feature extraction, deep feature extraction, multi-scale encoding and decoding, the decoder outputs a scale-consistent fused decoded feature. Then, the input low-resolution image is bilinearly upsampled to make its size consistent with the decoded feature. The upsampled low-resolution image and the decoded feature are concatenated in the same dimension to form a preliminary fused feature. Then, the dimension of the preliminary fused feature is adjusted through convolution operation to obtain the final fused feature of the same dimension. The final fused feature is then mapped to the training super-resolution image predicted by the jump network model.

[0037] During the training phase of the jump network model, the predicted super-resolution image for training is compared with the corresponding high-resolution image in the training data pair. The loss function is calculated, and the network parameters of the jump network model are updated by backpropagation based on the loss function. The calculated loss information is then passed back to the jump network model to optimize the parameters extracted, fused, encoded, and decoded in the jump network model. The training process is repeated iteratively, so that after the jump network model is trained, it can generate a super-resolution image of the corresponding region given a low-resolution image as input.

[0038] In this embodiment, the jump network model is trained by minimizing the L1 loss, assuming the original image is... and its corresponding super-resolution image The loss can be expressed as: ; Where N is the number of training samples, For network parameters, The number of images.

[0039] This invention discloses a remote sensing image super-resolution reconstruction method based on a Transformer jump network model. The method decomposes low-frequency and high-frequency components using a discrete wavelet transform module to recover high-frequency information and avoid frequency domain aliasing. A multi-scale joint connection module enhances the correlation between deep features at different scales through attention mechanisms, convolution, and normalization operations, obtaining fused multi-scale encoded features, reducing computation and preventing data loss. An enhanced frequency-domain aware Transformer module generates feature maps of different frequencies from the multi-scale decoded features. An adaptive weighting module performs context reweighting on the multi-scale decoded features to obtain decoded features, enhancing the expression of deep features, preserving high-frequency signals and clear structural features in the reconstructed image, and thus completing the high-frequency texture and edge details of the remote sensing image in the jump network model, improving texture clarity and enhancing structural recovery capabilities.

[0040] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention should also be included within the scope of protection of the present invention.

Claims

1. A remote sensing image super-resolution reconstruction method based on a Transformer skip network model, characterized in that, The constructed jump network model is used for super-resolution reconstruction of remote sensing images, including the following steps: Obtaining a low-resolution image and a corresponding high-resolution image of the same remote sensing observation area to form a training data pair, and obtaining multiple training data pairs of the same remote sensing observation area as a training set; After obtaining the training set, inputting the low-resolution image in each training data pair into the shallow layer convolution module of the jump network model to extract shallow layer features, the shallow layer features including structural information and local texture features of the image; The shallow layer features are input into the encoder of the jump network model for deep layer feature extraction, and the extracted deep layer features of different scales are input into the multi-scale joint connection module, which enhances the correlation between deep layer features of different scales through attention mechanism, convolution and normalization operation to obtain fused multi-scale coding features; The multi-scale coding features are sent to the decoder of the jump network model, and the adaptive weighting strategy in the decoder is used to reweight the multi-scale coding features to obtain scale-consistent fused decoding features; The decoding features output by the decoder are used to generate a training super-resolution image, which is compared with the high-resolution image of the corresponding training data pair to calculate a loss function, and the jump network model is optimized through back propagation training; The remote sensing image to be super-resolution reconstructed is input into the trained jump network model to generate a corresponding super-resolution image.

2. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 1, characterized in that, The shallow layer convolution module in the jump network model converts the input low-resolution image into a pixel matrix, and a plurality of independent convolution kernels are predefined in the shallow layer convolution module. The convolution kernel slides over the pixel matrix to cover a local pixel area, multiplies the pixel elements in each covered area and sums them up, and the calculation value of the pixel elements in the covered area corresponds to the shallow layer features extracted by the matching convolution kernel. Each convolution kernel extracts a shallow layer feature, and then extracts shallow layer features from the low-resolution image.

3. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 1, characterized in that, The encoder in the jump network model is provided with a discrete wavelet transform module and a content feature extraction module for extracting deep layer features at different scales, specifically: The discrete wavelet transform module is used to extract a low-frequency component and three high-frequency components from the input shallow layer features through frequency domain decomposition, the low-frequency component representing the structural information of the remote sensing image, and the three high-frequency components representing the texture information of the horizontal, vertical and diagonal textures, respectively; The content feature extraction module encodes the low-frequency component and the high-frequency component, respectively, extracts deep layer structural features based on the low-frequency component, and encodes to generate deep layer features of color distribution, contour shape and edge gradient of the image, respectively; Based on the deep layer texture features extracted from the high-frequency component, the horizontal, vertical and diagonal texture deep layer features are encoded, respectively.

4. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 3, characterized in that, The multi-scale joint connection module receives the deep features of different scales extracted by the encoder, and is used for unifying the scale features of the deep features of different scales, and then linearly projecting and splitting the scale features into multiple dimensions, normalizing the feature data of the same dimension, calculating the convolution values of the deep feature pixel elements of each different feature dimension by convolution, and obtaining the fused multi-scale encoding features.

5. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 4, characterized in that, The decoder in the skip network model receives the multi-scale encoding features output by the encoder, and sequentially inputs the maximum pooling layer and the average pooling layer set in the decoder, the maximum pooling layer is used to extract local features reflecting local region texture, and the average pooling layer is used to extract global features reflecting overall statistical distribution, and then outputs multi-scale decoding features.

6. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 5, characterized in that, The multi-scale decoding features are assigned different weights according to the adaptive weighting strategy, the decoder generates context resolution mapping weights for scale adjustment, forms decoding weights of multi-scale deep features, and the decoder remaps the multi-scale decoding features by using the decoding weights to obtain fused decoding features with consistent scales.

7. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 1, characterized in that, In the training phase of the skip network model, the low-resolution image in the training data pair is input into the skip network model, and after shallow feature extraction, deep feature extraction, multi-scale encoding and decoding, the fused decoding features with consistent scales output by the decoder are obtained. Then, the input low-resolution image is bilinearly upsampled to make its size consistent with the decoding features. The upsampled low-resolution image and the decoding features are spliced in the same dimension to form preliminary fusion features, and then the dimensions of the preliminary fusion features are adjusted by convolution operation to obtain final fusion features with the same dimensions, and the final fusion features are mapped to the training super-resolution image predicted by the skip network model.

8. The remote sensing image super-resolution reconstruction method based on the Transformer skip network model according to claim 7, characterized in that, In the training phase of the skip network model, the training super-resolution image generated by prediction is compared with the corresponding high-resolution image in the training data pair, a loss function is calculated, and the network parameters of the skip network model are updated by back propagation based on the loss function. Iterative training process makes the skip network model able to generate super-resolution images of corresponding regions under the condition of input low-resolution images after training is completed.

Images