A remote sensing image super-resolution reconstruction method, device and medium

Abstract

A remote sensing image super-resolution reconstruction method, device and medium, shallow features are extracted from a low-resolution remote sensing image by a shallow feature extraction module; a backbone network composed of a plurality of iterative progressive focusing feature blocks in cascade is used to complete deep feature mapping; the shallow features and the deep features are aggregated; and a high-resolution remote sensing image is output through an upsampling reconstruction module. The iterative progressive focusing feature block includes an iterative progressive focusing attention module, a feature repair and aggregation module, a Gaussian prior enhancement module and a convolution mapping unit, the iterative progressive focusing attention module continuously refines the attention distribution; the feature repair and aggregation module is used to supplement non-local context information and enhance local texture and directional structure expression; and the Gaussian prior enhancement module adaptively enhances the features. The present application can improve the detail recovery capability, structure preservation capability and feature update stability in complex remote sensing scenes while taking into account the computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and computer vision technology, specifically a method, device and medium for super-resolution reconstruction of remote sensing images based on iterative progressive focusing attention. Background Technology

[0002] Super-resolution reconstruction of remote sensing images aims to recover high-resolution remote sensing images with higher spatial resolution and richer texture details from low-resolution remote sensing images. It is of great significance for tasks such as remote sensing scene understanding, target detection, ground feature identification, and change detection. Compared with natural images, remote sensing images typically have a large range of ground feature scales, many weakly textured regions, and complex imaging degradation characteristics. Therefore, it places higher demands on the multi-scale representation ability, detail recovery ability, and feature update stability of super-resolution reconstruction methods.

[0003] Existing super-resolution methods based on convolutional neural networks have achieved certain results in local feature extraction, but due to the limited receptive field of convolution, they are insufficient in modeling long-range dependencies. While Transformer-based methods can establish global associations through self-attention mechanisms, standard self-attention typically has high computational complexity, leading to significant computational and storage overhead in high-resolution remote sensing image scenarios. To reduce complexity, existing methods have introduced sparse attention or progressive filtering strategies, concentrating computation on regions with high information content. However, in remote sensing image scenarios, performing a single sparse filtering operation early in feature updates can easily lead to the accidental deletion of potentially useful features, especially in small-scale targets, elongated structures, and weakly textured regions, further impacting the quality of subsequent reconstructions.

[0004] Furthermore, remote sensing images are often accompanied by noise, background clutter, and complex degradation. Without stable prior constraints, the attention update process is easily disturbed, leading to unstable feature selection. Therefore, this paper proposes a remote sensing image super-resolution reconstruction method that balances global dependency modeling, local detail restoration, and feature update stability, which has significant theoretical and practical application value. Summary of the Invention

[0005] The problem this invention aims to solve is that existing remote sensing image super-resolution methods are prone to problems such as loss of potential effective information, unstable attention distribution, and insufficient local texture restoration during feature selection. This invention provides a remote sensing image super-resolution reconstruction method based on iterative progressive focusing attention to overcome these shortcomings.

[0006] The technical solution of this invention is: a method for super-resolution reconstruction of remote sensing images, which constructs a remote sensing image super-resolution reconstruction network RS-IPFT, including a shallow feature extraction module, a backbone network, and an upsampling reconstruction module, and implements the following steps:

[0007] Acquire low-resolution remote sensing images;

[0008] The shallow feature extraction module performs convolution mapping on low-resolution remote sensing images to obtain shallow features.

[0009] Shallow features are input into the backbone network for deep feature mapping to obtain deep features. The backbone network is composed of several iterative progressive focusing feature blocks cascaded together.

[0010] The deep features are convolutionally aggregated and then globally residual-fused with the shallow features to obtain globally fused features.

[0011] The global fusion features are input into the upsampling reconstruction module for reconstruction, resulting in a high-resolution remote sensing image.

[0012] Furthermore, the backbone network consists of The system is composed of cascaded iterative progressive focusing feature blocks. Each iterative progressive focusing feature block includes, in sequence, an iterative progressive focusing attention module, a feature repair and aggregation module, a Gaussian prior enhancement module, and a convolutional mapping unit. The iterative progressive focusing attention module refines the attention distribution through a combination of multiple rounds of iterative updates within the block and unified filtering at the end of the block. The feature repair and aggregation module is used to supplement non-local context information and enhance the expression of local texture and directional structure. The Gaussian prior enhancement module uses Gaussian filtering to generate a smooth structure prior and adaptively enhances the features.

[0013] No. The output of each iterative progressive focusing feature block is represented as follows:

[0014]

[0015] in, Indicates the first Nonlinear mapping of iterative progressively focusing feature blocks and These represent the input and output features of the feature block, respectively.

[0016] Furthermore, the iterative progressive focusing attention module includes a window attention branch and a shift window attention branch. These two branches are executed alternately and iteratively within a block, and each maintains a historical attention map and a sparse index. The attention update, sparse index generation, and feature aggregation processes are represented as follows:

[0017]

[0018]

[0019]

[0020]

[0021]

[0022] in, Indicates the first In the current attention map calculated, This represents the attention graph from the previous round of inheritance. , , They represent the first The query matrix, key matrix, and value matrix of the round. This represents the sparse index matrix generated in the previous round. This represents element-wise product. This indicates a normalization operation. Represents sparse matrix multiplication. express Sparse selection operator, The sign function is used to generate the sparse index matrix for the current round based on the non-zero attention positions. Indicates the first The wheel's attention converges on the output;

[0023] The intra-block iterative update process of the window attention branch and the shifted window attention branch is represented as follows:

[0024]

[0025]

[0026] in, Indicates the window attention branch, This represents the attention branch of the shift window. Indicates the first Round of iterative input features, Indicates the first The output features are generated through rounds of iteration.

[0027] Furthermore, the feature repair and aggregation module adopts a dual-branch structure, first processing the input features... After convolution mapping, the data is divided into two parts along the channel dimension. The upper branch is used to supplement non-local context information, and the lower branch is used to recover local texture and directional structure.

[0028] The calculation process for the upper branch is as follows:

[0029]

[0030] The calculation process for the lower branch is as follows:

[0031]

[0032] The output after merging the two branches is:

[0033]

[0034] in, This indicates a channel partitioning operation. and These represent downsampling and upsampling operations, respectively. Represents depthwise convolution. This represents the variance information of the input features. This represents the activation function. This indicates a channel attention operation.

[0035] Furthermore, the Gaussian prior enhancement module first smooths the input features using Gaussian filtering to obtain a Gaussian prior response, then fuses the input features with the Gaussian prior response, and generates Gaussian guided attention through a convolutional block; subsequently, it uses the Gaussian guided attention to modulate the input features element-wise, and combines depthwise convolution and channel attention to obtain the enhanced output features; the calculation process is expressed as follows:

[0036]

[0037]

[0038]

[0039]

[0040] in, This represents a Gaussian convolution operation with fixed parameters. Indicates the Gaussian prior response. This represents the convolution mapping operation. This indicates the guiding attention generated by Gaussian priors. This represents the features enhanced by Gaussian prior guidance. Represents depthwise convolution. This indicates a channel attention operation.

[0041] Furthermore, the upsampling reconstruction module adopts The upsampling method outputs high-resolution remote sensing images, and its convolution mapping and reconstruction process is represented as follows:

[0042]

[0043]

[0044] in, Indicates global fusion features, This represents the convolution mapping operation. This represents the features to be reconstructed after convolution mapping. express Upsampling operation, This represents the output high-resolution remote sensing image.

[0045] The present invention also provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to realize the above-described remote sensing image super-resolution reconstruction method.

[0046] The present invention also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein when the at least one instruction or the program is executed, the above-described remote sensing image super-resolution reconstruction method is implemented.

[0047] The advantages of this invention over existing methods are:

[0048] 1) This invention reduces the risk of accidental deletion of potentially valid information caused by premature sparse screening by combining multi-round attention updates within blocks with unified screening at the end of blocks, thereby improving the stability of feature selection in complex remote sensing scenarios.

[0049] 2) This invention supplements non-local context and enhances the expression of local texture and directional structure through feature repair and aggregation modules, which is beneficial to improving the recovery ability of small-scale targets, slender structures and weak texture regions;

[0050] 3) This invention uses a Gaussian prior enhancement module to smooth the input features, which can effectively suppress noise and background clutter interference and improve the stability and robustness of the feature update process;

[0051] 4) While taking into account computational efficiency, this invention improves the detail recovery and structure preservation capabilities of super-resolution reconstruction of remote sensing images. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall network structure of the present invention.

[0053] Figure 2 This is a schematic diagram of the iterative progressive focusing attention module structure of the present invention.

[0054] Figure 3 This is a schematic diagram of the feature repair and aggregation module structure of the present invention.

[0055] Figure 4This is a schematic diagram of the Gaussian prior enhancement module structure of the present invention. Detailed Implementation

[0056] This invention proposes a method for super-resolution reconstruction of remote sensing images based on iterative progressive focusing attention. Given a low-resolution remote sensing image as input... First, shallow features are obtained through the shallow feature extraction module. Then, the shallow features are input into the backbone network to obtain deep features. Then, the deep features are aggregated through terminal convolution and fused with the shallow features using global residual fusion to obtain the globally fused features. Finally, a high-resolution image is output through the upsampling reconstruction module. This process can be represented as:

[0057]

[0058]

[0059]

[0060]

[0061] in, This represents the shallow feature extraction function. This represents the backbone network mapping function. This represents the terminal convolution aggregation function. This represents the upsampling reconstruction function.

[0062] This invention achieves super-resolution reconstruction of remote sensing images by constructing a network called RS-IPFT. This network consists of a shallow feature extraction module, a backbone network, and an upsampling reconstruction module, wherein the backbone network is composed of multiple cascaded iterative progressive focusing feature blocks (IPFBs). The output of each iterative progressive focusing feature block is represented as follows:

[0063]

[0064] Each iterative progressive focusing feature block includes, in sequence, an iterative progressive focusing attention module, a feature repair and aggregation module, a Gaussian prior enhancement module, and a convolutional mapping unit.

[0065] The iterative progressive focusing attention module includes a window attention branch and a shifting window attention branch. These two branches execute alternately and iteratively within a block, maintaining a historical attention graph and a sparse index respectively. The attention update process is as follows:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071] Furthermore, the iterative updates of the window attention branch and the shifted window attention branch within the block are as follows:

[0072]

[0073]

[0074] This structure can continuously refine the attention distribution through multiple iterations within a block without excessively increasing the number of parameters, and then execute it uniformly once at the end of the block. Sparse screening can mitigate the information loss caused by premature sparse screening.

[0075] The feature repair and aggregation module is used to supplement, repair, and aggregate the features after attention update. First, the input features are divided into upper and lower branches along the channel dimension:

[0076]

[0077] The upper branch supplements non-local context information through downsampling, depthwise convolution, variance information modulation, and upsampling. The process is as follows:

[0078]

[0079] The lower branch uses depthwise convolution, convolution, Convolution and Convolution recovers local texture and directional structure; the process is as follows:

[0080]

[0081] The output after merging the two branches is:

[0082]

[0083] The above design can enhance the expression of local details while supplementing non-local information.

[0084] The Gaussian prior enhancement module first smooths the input features using Gaussian filtering to obtain the Gaussian prior response:

[0085]

[0086] Then, the input features are fused with the Gaussian prior response, and Gaussian guided attention is generated through convolutional blocks:

[0087]

[0088] After obtaining Gaussian guided attention, it is used to modulate the input features element-wise and combined with residual compensation to obtain enhanced features:

[0089]

[0090] Furthermore, the enhanced features are output after depthwise convolution and channel attention processing:

[0091]

[0092] The Gaussian prior enhancement module can enhance spatially continuous structural regions while suppressing noise and background clutter, thus providing a smoother and more robust input basis for the current feature representation and subsequent attention updates.

[0093] As an example, the RS-IPFT upsampling reconstruction module of the present invention employs... Upsampling is used to reconstruct high-resolution remote sensing images. Specifically, the features to be reconstructed are first obtained through convolutional mapping:

[0094]

[0095] Then through Output high-resolution remote sensing images:

[0096]

[0097] It can be uniformly represented as:

[0098]

[0099] in, Indicates global fusion features, This represents the convolution mapping operation. This represents the features to be reconstructed after convolution mapping. express Upsampling operation, This represents the output high-resolution remote sensing image. This indicates that the convolution mapping and The reconstruction function is composed of upsampling and other components. The design of the upsampling reconstruction module can effectively control the number of parameters and computational cost while ensuring reconstruction quality.

[0100] During training, this invention employs an L1 loss function to supervise the network, constraining the pixel-level differences between the reconstructed image and the target high-resolution image. Training data can be constructed using remote sensing image datasets such as AID, DIOR, and DOTA. Since the loss function and dataset are existing technologies, they will not be elaborated upon here.

[0101] This invention can be implemented based on a computer program. Based on this, the invention also provides an electronic device comprising a processor and a memory. The memory stores at least one instruction or at least one program segment, which is loaded and executed by the processor to implement the aforementioned remote sensing image super-resolution reconstruction method. A computer-readable storage medium is also provided, storing at least one instruction or at least one program segment. When executed, the at least one instruction or program segment implements the aforementioned remote sensing image super-resolution reconstruction method. The instructions or program code used to implement the method of this invention can be written in any combination of one or more programming languages. The instructions or program code can be executed entirely on a data processor, partially on a processor, partially on a remote device as a standalone software package, or entirely on a remote device or server.

[0102] In summary, by introducing an iterative progressive focusing attention module, a feature repair and aggregation module, and a Gaussian prior enhancement module into the remote sensing image super-resolution reconstruction network, this invention can improve the detail recovery capability, structure preservation capability, and feature update stability in complex remote sensing scenarios while taking into account computational efficiency. It has good practical value and application prospects.

Claims

1. A method for super-resolution reconstruction of remote sensing images, characterized by: Construct a remote sensing image super-resolution reconstruction network RS-IPFT, including a shallow feature extraction module, a backbone network, and an upsampling reconstruction module, and implement the following steps: Acquire low-resolution remote sensing images; The shallow feature extraction module performs convolution mapping on low-resolution remote sensing images to obtain shallow features. Shallow features are input into the backbone network for deep feature mapping to obtain deep features. The backbone network is composed of several iterative progressive focusing feature blocks cascaded together. The deep features are convolutionally aggregated and then globally residual-fused with the shallow features to obtain globally fused features. The global fusion features are input into the upsampling reconstruction module for reconstruction, resulting in a high-resolution remote sensing image.

2. The method for super-resolution reconstruction of remote sensing images according to claim 1, characterized in that: The backbone network is composed of a plurality of iterative progressive focusing feature blocks, each of which sequentially comprises an iterative progressive focusing attention module, a feature repairing and aggregating module, a Gaussian prior enhancing module, and a convolution mapping unit. The iterative progressive focusing attention module refines the attention distribution through a combination of multi-round iterative updating within the block and unified screening at the end of the block. The feature repairing and aggregating module is used to supplement non-local context information and enhance the expression of local texture and directional structure. The Gaussian prior enhancing module generates a smooth structure prior using Gaussian filtering to adaptively enhance the features. No. The output of each iterative progressive focusing feature block is represented as follows: wherein, denotes the nonlinear mapping of the ith iteratively progressively focused feature block, and denote the input and output features of the feature block, respectively.

3. The method of claim 2, wherein the method further comprises: The iterative progressive focusing attention module includes a window attention branch and a shift window attention branch. These two branches are executed alternately and iteratively within the block, and each maintains a historical attention map and a sparse index. The attention update, sparse index generation, and feature aggregation processes are represented as follows: in, Indicates the first In the current attention map calculated, This represents the attention graph from the previous round of inheritance. , , They represent the first The query matrix, key matrix, and value matrix of the round. This represents the sparse index matrix generated in the previous round. This represents element-wise product. This indicates a normalization operation. Represents sparse matrix multiplication. express Sparse selection operator, The sign function is used to generate the sparse index matrix for the current round based on the non-zero attention positions. Indicates the first The wheel's attention converges on the output; The intra-block iterative update process of the window attention branch and the shifted window attention branch is represented as follows: in, Indicates the window attention branch, This represents the attention branch of the shift window. Indicates the first Round of iterative input features, Indicates the first The output features are generated through rounds of iteration.

4. The method for super-resolution reconstruction of remote sensing images according to claim 2, characterized in that the feature repair and aggregation module adopts a dual-branch structure, firstly processing the input features... After convolution mapping, the data is divided into two parts along the channel dimension. The upper branch is used to supplement non-local context information, and the lower branch is used to recover local texture and directional structure. The calculation process for the upper branch is as follows: The calculation process for the lower branch is as follows: The output after merging the two branches is: in, This indicates a channel partitioning operation. and These represent downsampling and upsampling operations, respectively. Represents depthwise convolution. This represents the variance information of the input features. This represents the activation function. This indicates a channel attention operation.

5. The method for super-resolution reconstruction of remote sensing images according to claim 2, characterized in that: The Gaussian prior enhancement module first smooths the input features using Gaussian filtering to obtain a Gaussian prior response, then fuses the input features with the Gaussian prior response, and generates Gaussian guided attention through a convolutional block; subsequently, it uses the Gaussian guided attention to modulate the input features element-wise, and combines depthwise convolution and channel attention to obtain the enhanced output features. The calculation process is expressed as follows: in, This represents a Gaussian convolution operation with fixed parameters. Indicates the Gaussian prior response. This represents the convolution mapping operation. This indicates the guiding attention generated by Gaussian priors. This represents the features enhanced by Gaussian prior guidance. Represents depthwise convolution. This indicates a channel attention operation.

6. The method for super-resolution reconstruction of remote sensing images according to claim 1, characterized in that... The sampling and reconstruction module adopts The upsampling method outputs high-resolution remote sensing images, and its convolution mapping and reconstruction process is represented as follows: in, Indicates global fusion features, This represents the convolution mapping operation. This represents the features to be reconstructed after convolution mapping. express Upsampling operation, This represents the output high-resolution remote sensing image.

7. An electronic device comprising a processor and a memory, characterized in that: The memory stores at least one instruction or at least one program segment, which is loaded and executed by the processor to implement the remote sensing image super-resolution reconstruction method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that... The computer-readable storage medium stores at least one instruction or at least one program, which, when executed, implements the remote sensing image super-resolution reconstruction method according to any one of claims 1 to 6.

Images