Super-resolution reconstruction method, device and medium for complex terrain building remote sensing image

Abstract

This application discloses a method, apparatus, and medium for super-resolution reconstruction of remote sensing images of complex terrain and buildings. The method includes: acquiring historical low-resolution images and current high-resolution images; extracting features from the historical low-resolution images by reconstructing the main branch to obtain main branch features; extracting features from the current high-resolution images by using the guiding branch and performing semantic alignment to obtain guiding branch features; calculating the difference between the main branch features and the guiding branch features to obtain feature difference information, and determining a change weight map based on the feature difference information; adaptively weighting and fusing the main branch features and the guiding branch features according to the change weight map to obtain fused features; performing multi-scale convolutional attention processing on the fused features to obtain enhanced features; and decoding the enhanced features to obtain the target high-resolution reconstructed image. Using this application, high-precision and stable image reconstruction can be achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology and reconstruction of remote sensing images of buildings in complex terrain, and more specifically, to a method, apparatus and medium for super-resolution reconstruction of remote sensing images of buildings in complex terrain in the field of computer technology. Background Technology

[0002] Super-resolution reconstruction of remote sensing imagery is a method that uses algorithms to restore high-resolution details. It is commonly used to improve the spatial resolution of remote sensing images to meet the needs of applications such as land surface monitoring and urban planning. However, in complex terrain scenarios (such as mountainous or hilly areas), existing methods often face performance degradation challenges. This may stem from the influence of factors such as terrain undulations, lighting variations, and shadows, leading to problems such as structural blurring or loss of detail in the reconstruction results. Such issues can affect the accuracy of subsequent analysis tasks, such as feature identification or change detection. Therefore, how to effectively achieve super-resolution reconstruction of buildings in complex terrain remote sensing imagery has become an urgent technical challenge to be solved. Summary of the Invention

[0003] This application provides a method, apparatus, and medium for super-resolution reconstruction of remote sensing images of complex terrain and buildings. The method can utilize complementary information between multiple temporal images, guided by the current high-resolution image, to perform detail completion and temporal consistency constraints on historical low-resolution images, thereby achieving high-precision and stable image reconstruction.

[0004] In a first aspect, embodiments of this application provide a method for super-resolution reconstruction of remote sensing images of complex terrain and buildings, including: Acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographic area acquired at different times. The main branch features are obtained by reconstructing the main branch and extracting features from the historical low-resolution image. The guide branch features are obtained by extracting features from the current high-resolution image and performing semantic alignment. Calculate the difference between the main branch feature and the guiding branch feature to obtain feature difference information, and determine the change weight map based on the feature difference information; Based on the change weight map, the main branch features and the guiding branch features are adaptively weighted and fused to obtain the fused features; The fused features are subjected to multi-scale convolutional attention processing to obtain enhanced features; The enhanced features are decoded to obtain a high-resolution reconstructed image of the target.

[0005] Secondly, embodiments of this application provide a super-resolution reconstruction apparatus for remote sensing images of complex terrain and buildings, comprising: The image acquisition unit is used to acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographical area acquired at different time points. The first feature acquisition unit is used to extract features from the historical low-resolution image by reconstructing the main branch to obtain the main branch features; The second feature acquisition unit is used to extract features from the current high-resolution image and perform semantic alignment through the guide branch to obtain guide branch features; An information acquisition unit is used to calculate the difference between the main branch feature and the guiding branch feature, obtain feature difference information, and determine a change weight map based on the feature difference information. The third feature acquisition unit is used to adaptively weight and fuse the main branch features and the guiding branch features according to the change weight map to obtain the fused features; The fourth feature acquisition unit is used to perform multi-scale convolutional attention processing on the fused features to obtain enhanced features; The feature decoding unit is used to decode the enhanced features to obtain a high-resolution reconstructed image of the target.

[0006] Thirdly, embodiments of this application provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0007] Fourthly, embodiments of this application provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0008] In this embodiment, historical low-resolution images and current high-resolution images are first acquired. Main branch features are extracted from the historical low-resolution images by reconstructing the main branch to construct a structural baseline. Simultaneously, guide branch features are extracted from the current high-resolution images by extracting features and performing semantic alignment to obtain guide branch features, thus acquiring high-frequency priors. Next, feature difference information is obtained by calculating the difference between the main branch features and the guide branch features. Based on this information, a change weight map is determined. Then, the main branch features and guide branch features are adaptively weighted and fused according to this change weight map. This process explicitly models ground feature changes through a gating mechanism, thereby maintaining temporal consistency in invariant areas. First, erroneous texture migration is suppressed in changing regions. Then, multi-scale convolutional attention processing is applied to the fused features. This process uses convolutional kernels of multiple scales to simultaneously capture macroscopic structures and microscopic details, thereby enhancing the ability to restore building edges and road textures under complex terrain. Finally, the enhanced features are decoded to generate a high-resolution reconstructed image of the target. Therefore, through the synergistic effect of the above steps, this scheme overcomes the structural blurring and artifact problems caused by the single use of information, unstable registration and fusion, and lack of change perception in existing technologies. Thus, without relying on explicit terrain data, temporal consistency and high-fidelity reconstruction of remote sensing images of buildings in complex terrain are achieved. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the structure of a temporal prior bibranch reconstruction model for remote sensing images of buildings in complex terrain, provided for an embodiment of this application; Figure 2 A flowchart illustrating a method for super-resolution reconstruction of buildings in complex terrain remote sensing images, provided as an embodiment of this application; Figure 3 A flowchart illustrating a main branch feature acquisition process provided in an embodiment of this application; Figure 4 A flowchart illustrating a process for obtaining guide branch features, provided as an embodiment of this application; Figure 5 A flowchart illustrating a process for obtaining a change weight map, provided in an embodiment of this application; Figure 6 A flowchart illustrating a fusion feature acquisition process provided in an embodiment of this application; Figure 7A flowchart illustrating an enhanced feature acquisition process provided in an embodiment of this application; Figure 8 A schematic diagram of the structure of a super-resolution reconstruction device for remote sensing images of complex terrain and buildings provided in this application embodiment; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] Super-resolution reconstruction of remote sensing images is a key technique for improving the spatial resolution of remote sensing images and meeting the needs of detailed surface monitoring. With the development of deep learning technology, reconstruction methods based on models such as convolutional neural networks and Transformers have achieved significant results under ideal imaging conditions. However, in complex terrain environments such as mountainous and hilly areas, the performance of these reconstruction methods is prone to significant degradation due to the complex and variable imaging conditions. This degradation often leads to problems such as structural blurring and loss of detail in the reconstructed images, severely affecting the accuracy of subsequent feature identification and change analysis. Therefore, how to effectively overcome the adverse effects of complex terrain and achieve high-precision and high-stability reconstruction of building remote sensing images has become a pressing technical challenge in this field.

[0013] For details, please see Figure 1 This application provides a schematic diagram of the temporal prior bi-branch reconstruction model for remote sensing images of complex terrain and buildings, as illustrated in this embodiment. Figure 1 As shown, the reconstruction model may include a reconstruction main branch 100, a guiding branch 200, a variation-gated fusion module 300, a multi-scale convolutional attention module 400, and a decoder 500.

[0014] The reconstruction main branch 100 includes a convolutional kernel and a residual Swin block, while the guiding branch 200 includes a Swin Transformer encoder, a shift prediction subnetwork, and a deformable convolutional layer.

[0015] By constructing the reconstruction model, in practical applications, the reconstruction of the main branch 100 extracts features from historical low-resolution images to obtain the main branch features in order to construct the structural baseline. Specifically, the historical low-resolution images are convolved with a preset number of kernels to extract initial features. Then, the initial features are processed hierarchically by residual Swin blocks to output the main branch features of the reconstruction of the main branch 100.

[0016] Simultaneously, the guide branch extracts features from the current high-resolution image and performs semantic alignment to obtain guide branch features and acquire high-frequency priors. Specifically, the Swin Transformer encoder extracts multi-scale features from the current high-resolution image to obtain the first multi-scale features, which provide a basis for subsequent semantic alignment. The Swin Transformer encoder provides the output first multi-scale features to the deformable convolutional layer and the offset prediction sub-network. The offset prediction sub-network can predict the features of the first multi-scale features using a set of learnable parameters with low location cost. Then, the deformable convolutional layer adaptively samples the first multi-scale features according to the set of learnable parameters through deformable convolution to output the guide branch features of guide branch 200.

[0017] Next, the difference between the main branch features and the guiding branch features is calculated by the change gating fusion module 300 to obtain feature difference information. Based on this information, a change weight map is determined. Then, the main branch features and the guiding branch features are adaptively weighted and fused according to this change weight map. This process explicitly models the changes in land cover through a gating mechanism, thereby maintaining temporal consistency in invariant areas and suppressing erroneous texture migration in change areas.

[0018] Then, the fused features are processed by a multi-scale convolutional attention module 400. This process utilizes convolutional kernels of multiple scales to simultaneously capture macroscopic structures and microscopic details, thereby enhancing the ability to restore building edges and road textures in complex terrain. Finally, the enhanced features are decoded by a decoder 500 to generate a high-resolution reconstructed image of the target.

[0019] Through the synergistic effect of the above steps, the embodiments of this application overcome the structural ambiguity and artifact problems caused by the single use of information, unstable registration and fusion, and lack of change perception in the prior art. Thus, without relying on explicit terrain data, the temporal consistency and high-fidelity reconstruction of remote sensing images of complex terrain buildings are achieved.

[0020] based on Figure 1 The model structure shown below will be combined with... Figures 2-7 This application provides a detailed description of the super-resolution reconstruction method for remote sensing images of complex terrain and buildings provided in the embodiments of this application.

[0021] Please see Figure 2 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 2 As shown, the method in this application embodiment may include the following steps S101-S107.

[0022] S101, acquire historical low-resolution images and current high-resolution images; Specifically, the historical low-resolution images involved in this embodiment can be acquired through satellite sensors or aerial photography; the current high-resolution images can be high-resolution remote sensing images of the same geographical area acquired at the latest time point. Here, low resolution means that the spatial resolution of the image is low and the details are blurry; high resolution means that the spatial resolution of the image is high and the details are clear.

[0023] To achieve cross-temporal image reconstruction, it is necessary to acquire historical low-resolution images and current high-resolution images. Regarding this step, some possible implementations include retrieving historical low-resolution images and current high-resolution images from a remote sensing database and preprocessing them; others involve using image registration techniques to spatially align the historical low-resolution images and current high-resolution images; and still others involve performing radiometric correction on the historical low-resolution images and current high-resolution images to eliminate illumination differences.

[0024] S102, the main branch features are obtained by reconstructing the main branch and extracting features from the historical low-resolution image. S103, the current high-resolution image is subjected to feature extraction and semantic alignment by the guide branch to obtain the guide branch features; Specifically, in this embodiment, feature extraction is performed on the historical low-resolution image through the reconstruction of the main branch to obtain main branch features; feature extraction is performed on the current high-resolution image through the guiding branch and semantic alignment is performed to obtain guiding branch features. Here, the reconstruction of the main branch refers to the neural network branch used to process the historical low-resolution image, aiming to extract basic structural features; the main branch features refer to the feature maps representing the structure of the historical low-resolution image extracted from the reconstruction of the main branch; the guiding branch refers to the neural network branch used to process the current high-resolution image, aiming to provide high-frequency detail guidance; the guiding branch features refer to the feature maps representing the details of the current high-resolution image extracted from the guiding branch; semantic alignment refers to the process of aligning the guiding branch features with the main branch features at the semantic level. In some possible implementations, a convolutional neural network can be used as the basic architecture for the reconstruction of the main branch and the guiding branch; in some possible implementations, an attention mechanism can be used to enhance the feature extraction capabilities of the reconstruction of the main branch and the guiding branch.

[0025] S104, calculate the difference between the main branch feature and the guiding branch feature to obtain feature difference information, and determine the change weight map based on the feature difference information; Specifically, this embodiment calculates the difference between the main branch feature and the guiding branch feature to obtain feature difference information, and determines a change weight map based on the feature difference information. Here, feature difference information refers to a measure representing the difference between the main branch feature and the guiding branch feature; the change weight map refers to a weight map generated based on the feature difference information, used to indicate the degree of change of each pixel or region. In some possible implementations, the absolute difference between the main branch feature and the guiding branch feature can be calculated as feature difference information, and then a change weight map can be generated through a gating network; in some possible implementations, pooling operations can be used to process the feature difference information to generate the change weight map.

[0026] S105, Adaptive weighted fusion of the main branch features and the guiding branch features is performed on the change weight map to obtain fused features; Specifically, in this embodiment, the main branch features and the guiding branch features are adaptively weighted and fused according to the changing weight map to obtain the fused features. The fused features refer to the weighted feature map combining the main branch features and the guiding branch features. In some possible implementations, the changing weight map can be applied to the guiding branch features and the main branch features, and the fused features can be obtained through element-wise multiplication and addition operations; in some possible implementations, a weighted average strategy can be used to fuse the main branch features and the guiding branch features to generate the fused features.

[0027] S106, Perform multi-scale convolutional attention processing on the fused features to obtain enhanced features; Specifically, this embodiment applies multi-scale convolutional attention processing to the fused features to obtain enhanced features. Enhanced features refer to the feature map after multi-scale convolutional attention processing, which possesses enhanced details and contextual information. In some possible implementations, multiple convolutional kernel scales can be used to extract fused features, and the results can be fused through an attention mechanism; in other possible implementations, a channel attention module can be introduced after multi-scale convolution to generate enhanced features.

[0028] S107, Decode the enhanced features to obtain a high-resolution reconstructed image of the target; Specifically, in this embodiment, the enhanced features are decoded to obtain the target high-resolution reconstructed image. Here, the target high-resolution reconstructed image refers to the final output high-resolution image, representing the reconstruction result at a historical time point. In some possible implementations, upsampling and convolutional layers can be used as decoders to convert the enhanced features into pixel-level images; in other possible implementations, deconvolution operations can be used to enhance the edge details of the target high-resolution reconstructed image.

[0029] Regarding this step, in some possible implementations, the high-resolution reconstructed image of the target can be post-processed, including denoising and sharpening, to improve visual quality.

[0030] In this embodiment, historical low-resolution images and current high-resolution images are first acquired. Main branch features are extracted from the historical low-resolution images by reconstructing the main branch to construct a structural baseline. Simultaneously, guide branch features are extracted from the current high-resolution images by extracting features and performing semantic alignment to obtain guide branch features, thus acquiring high-frequency priors. Next, feature difference information is obtained by calculating the difference between the main branch features and the guide branch features. Based on this information, a change weight map is determined. Then, the main branch features and guide branch features are adaptively weighted and fused according to this change weight map. This process explicitly models ground feature changes through a gating mechanism, thereby maintaining temporal consistency in invariant areas. First, erroneous texture migration is suppressed in changing regions. Then, multi-scale convolutional attention processing is applied to the fused features. This process uses convolutional kernels of multiple scales to simultaneously capture macroscopic structures and microscopic details, thereby enhancing the ability to restore building edges and road textures under complex terrain. Finally, the enhanced features are decoded to generate a high-resolution reconstructed image of the target. Therefore, through the synergistic effect of the above steps, this scheme overcomes the structural blurring and artifact problems caused by the single use of information, unstable registration and fusion, and lack of change perception in existing technologies. Thus, without relying on explicit terrain data, temporal consistency and high-fidelity reconstruction of remote sensing images of buildings in complex terrain are achieved.

[0031] Please see Figure 3 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 3 As shown, the method in this application embodiment may include the following steps S201-S202.

[0032] S201, Perform convolutional mapping on the historical low-resolution image to obtain initial features; S202, the initial features are hierarchically processed based on the residual Swin block to obtain the main branch features; Specifically, the historical low-resolution imagery involved in this embodiment refers to low-resolution remote sensing imagery data representing a geographic region acquired from historical time points, used to reflect the spatial information of the Earth's surface in the past. The initial features refer to the initial feature representation generated after performing a convolution operation on the historical low-resolution imagery, serving as the basic input for subsequent feature extraction. To extract low-level visual information from the historical low-resolution imagery, it is necessary to perform convolution mapping on the historical low-resolution imagery to obtain the initial features. The initial features refer to feature data containing the image's basic texture and edge information, serving as the input source for hierarchical processing. Regarding this step, in some possible implementations, a convolution kernel of a preset size can be used to perform convolution operations on the historical low-resolution imagery to generate the initial features; in some possible implementations, an activation function and a normalization step can be combined to process the convolution result to obtain the initial features.

[0033] The formula for calculating the initial features is: F0=Conv 3×3 (L t ); Wherein, F0 represents the initial feature; Conv 3×3 This represents a 3×3 convolution operation used for low-level feature extraction from the input image; L t This refers to the historical low-resolution image.

[0034] Based on the obtained initial features, the residual Swing block involved in this embodiment refers to a neural network module that integrates residual connections and a SwingTransformer structure to enhance feature extraction capabilities. The main branch features refer to the deep semantic feature representation extracted after hierarchical processing of the initial features, used to reconstruct the structural information of the image. To capture multi-level contextual dependencies from the initial features, the initial features need to be hierarchically processed based on the residual Swing block to obtain the main branch features. Here, the main branch features refer to feature maps representing the geometric contours and spatial relationships of historical low-resolution images. Regarding this step, in some possible implementations, the initial features can be input into multiple cascaded residual Swing blocks for processing to extract the main branch features; in some possible implementations, the depth of feature extraction can be controlled by adjusting the stacking layer number of the residual Swing blocks, thereby obtaining the main branch features.

[0035] The formula for calculating the main branch characteristics is: F r =E r (F0); Among them, F r E represents the main branch characteristic; rThis represents the encoder structure composed of residual Swing blocks, used for hierarchical processing of the initial features; F0 represents the initial features.

[0036] In this embodiment, through the aforementioned convolutional mapping and residual Swing block processing, principal features that combine local texture details and macroscopic spatial structure can be effectively extracted from historical low-resolution images. The initial features generated by the convolutional mapping operation retain the basic edge and texture information of the image, providing stable input for subsequent deep feature extraction. The hierarchical processing based on residual Swing blocks, utilizing their residual connections and windowed self-attention mechanism, enhances feature transfer efficiency while capturing a wide range of contextual dependencies, thereby more accurately constructing the geometric shapes of key features such as building outlines and road directions. Therefore, the principal features formed in this process not only fully characterize the static structural baseline of historical moments but also provide a reliable spatial reference for subsequent dynamic alignment and fusion with guiding features, forming the basis for achieving high-fidelity reconstruction across temporal phases.

[0037] Please see Figure 4 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 4 As shown, the method in this application embodiment may include the following steps S301-S303.

[0038] S301, The current high-resolution image is processed using a lightweight Swin Transformer encoder to obtain the first multi-scale feature; Specifically, the lightweight Swin Transformer encoder involved in this embodiment refers to a lightweight feature extraction network based on the Transformer architecture, which aims to efficiently extract multi-level semantic features; the current high-resolution image refers to high-resolution remote sensing image data of the same geographic area acquired at the latest time point.

[0039] To extract rich multi-scale features from the current high-resolution image and provide a foundation for subsequent semantic alignment, a lightweight Swin Transformer encoder is needed to process the image to obtain the first multi-scale features. These first multi-scale features refer to feature representations containing different spatial resolutions and semantic levels, used to capture global context and local texture information.

[0040] In some possible implementations, the input to the lightweight Swing Transformer encoder can be set to the current high-resolution image, and the first multi-scale features can be extracted by stacking multiple Swing Transformer blocks. In some possible implementations, the output of the lightweight Swing Transformer encoder can be downsampled to enhance the scale diversity of the first multi-scale features.

[0041] The formula for calculating the first multi-scale feature is: F g =E g (H T ); Among them, F g E represents the first multi-scale feature. g This indicates a lightweight Swing Transformer encoder, H T This indicates the current high-resolution image.

[0042] S302, predict the learnable offset field based on the first multi-scale feature; Specifically, the learnable offset field involved in this embodiment refers to a set of learnable parameters used to indicate the offset of feature sampling positions.

[0043] To achieve spatial alignment of features across time phases and reduce geometric misalignment caused by terrain undulations and illumination variations, a learnable offset field needs to be predicted based on the first multi-scale features. Regarding this step, in some possible implementations, the first multi-scale features can be input into an offset prediction sub-network to generate a learnable offset field. In other possible implementations, the first multi-scale features can be processed through convolutional layers to obtain the learnable offset field.

[0044] The formula for calculating the learnable offset field is: Δ=P(F g ); Where Δ represents the learnable migration field and P represents the migration prediction subnetwork.

[0045] S303, Based on the learnable offset field, adaptive sampling of the first multi-scale features is performed through deformable convolution to obtain the guiding branch features; Specifically, the guiding branch features involved in this embodiment refer to the feature representation after spatial alignment and detail enhancement, which is used to guide the reconstruction of historical images.

[0046] To obtain features that are semantically consistent with and spatially aligned with the main branch features, and to promote cross-temporal information fusion, it is necessary to adaptively sample the first multi-scale features using deformable convolution based on the learnable offset field to obtain the guiding branch features. Regarding this step, in some possible implementations, the learnable offset field can be used as a sampling control parameter to perform deformable convolution operations on the first multi-scale features to generate the guiding branch features. In some possible implementations, bilinear interpolation combined with the learnable offset field can be used to resample the first multi-scale features to obtain the guiding branch features.

[0047] The formula for calculating the guiding branch features is: F g =DeformConv(F g ,Δ); Among them, F g ′ represents the guiding branch feature, and DeformConv represents the deformable convolution operation.

[0048] In this embodiment, a lightweight Swin Transformer encoder is used to process the current high-resolution image, effectively extracting first multi-scale features containing rich texture and structural information. Subsequently, a learnable offset field predicted based on the first multi-scale features provides accurate geometric correction information for subsequent feature space alignment. Finally, deformable convolution adaptively samples the first multi-scale features according to the learnable offset field, ensuring that the sampled guiding features maintain high-frequency details while achieving semantic alignment with the feature space of historical low-resolution images. This process effectively overcomes the problem of cross-temporal feature misalignment caused by terrain undulations, viewpoint differences, and illumination variations, ensuring the accuracy and reliability of the guiding features as reconstruction priors and laying a high-quality input foundation for subsequent change-gated fusion.

[0049] Please see Figure 5 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 5 As shown, the method in this application embodiment may include the following steps S401-S402.

[0050] S401, input the feature difference information into a preset gating network; Specifically, the feature difference information involved in this embodiment refers to a measure of the degree of difference between the main branch features and the leading branch features. To generate a change weight map to control the feature fusion ratio, the feature difference information needs to be input into a preset gating network. The preset gating network refers to a lightweight neural network structure used to predict weight values ​​based on the input features.

[0051] The formula for calculating feature difference information can be: ΔF=|F r –F g ′|; Where ΔF represents feature difference information, F r F represents the main branch characteristic. g ′ indicates a guiding branch feature.

[0052] S402, the changed weight map is output through the gating network; Specifically, the change weight map is output through the gating network. Here, the change weight map refers to a weight map generated based on feature difference information, used to indicate the degree of change for each pixel or region.

[0053] In some possible implementations, feature difference information can be processed through a gating network, and the output can be constrained to the [0, 1] interval by applying the Sigmoid function. Furthermore, the output of the gating network can be normalized to obtain a changing weight map.

[0054] The prediction formula for the weighted graph can be: M = σ(G(ΔF)); Where M represents the changing weight map, σ represents the activation function, G represents the gating network, and ΔF represents the feature difference information.

[0055] In this embodiment, by calculating the feature difference information between the main branch features and the guiding branch features, and inputting this information into a gating network to generate a change weight map, explicit modeling of cross-temporal feature differences is achieved. This change weight map, acting as a gating signal, adaptively controls the ratio of main branch features to guiding branch features during fusion. For regions with small feature differences, the change weight map assigns higher weights to the guiding branch features to maintain temporal consistency; for regions with significant feature differences, the change weight map reduces the influence of the guiding branch features and relies more on the main branch features, thereby suppressing erroneous texture migration caused by changes in real-world features or differences in illumination. This adaptive fusion mechanism based on feature differences enables the model to distinguish between "real-world changes" and "illumination or angle differences," improving the realism and temporal coherence of the reconstruction results while maintaining the ability to update structures in dynamic scenes.

[0056] Please see Figure 6 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 6 As shown, the method in this application embodiment may include the following steps S501-S503.

[0057] S501, Multiply the change weight map with the guiding branch feature to obtain the first weighted feature; S502, after inverting the change weight map, multiply it by the main branch feature to obtain the second weighted feature; S503, merge the first weighted feature and the second weighted feature to obtain the fused feature; Specifically, to achieve adaptive weighting of the guiding branch features in feature fusion, the changing weight map and the guiding branch features need to be multiplied by a dot to obtain the first weighted feature. The first weighted feature refers to the guiding branch feature weighted by the changing weight map, used to reflect the degree of influence of the current temporal information during the fusion process. In some possible implementations, the changing weight map can be applied to the guiding branch features through element-wise multiplication to generate the first weighted feature.

[0058] The second weighted feature is obtained by inverting the change weight map and multiplying it by the main branch feature. This second weighted feature refers to the main branch feature weighted by the complement of the change weight map, used to reflect the influence of historical temporal information during the fusion process. In some possible implementations, the second weighted feature can be generated by calculating the complement of the change weight map (i.e., 1 minus the change weight map) and performing element-wise multiplication with the main branch feature.

[0059] The first weighted feature and the second weighted feature are merged to obtain the fused feature; wherein, the fused feature refers to the final feature representation that combines the main branch feature and the guiding branch feature, which is used for subsequent multi-scale processing and decoding reconstruction. In some possible implementations, the first weighted feature and the second weighted feature can be merged by addition to generate the fused feature.

[0060] Optionally, the formula for calculating the fused features can be: F f =M⊙F g ′+(1-M)⊙F r ; Among them, F f M represents the fusion feature; F represents the change weight map; g ′ represents the guiding branch feature; F r Indicates the main branch characteristics.

[0061] In this embodiment, by calculating the feature difference information between the main branch features and the guiding branch features, and inputting this information into a gating network to generate a change weight map, explicit modeling of cross-temporal feature differences is achieved. This change weight map, acting as a gating signal, adaptively controls the ratio of main branch features to guiding branch features during fusion. For regions with small feature differences, the change weight map assigns higher weights to the guiding branch features to maintain temporal consistency; for regions with significant feature differences, the change weight map reduces the influence of the guiding branch features and relies more on the main branch features, thereby suppressing erroneous texture migration caused by changes in real-world features or differences in illumination. This adaptive fusion mechanism based on feature differences enables the model to distinguish between "real-world changes" and "illumination or angle differences," improving the realism and temporal coherence of the reconstruction results while maintaining the ability to update structures in dynamic scenes.

[0062] Please see Figure 7 This document provides a flowchart illustrating a method for super-resolution reconstruction of complex terrain and building remote sensing images, as exemplified in this application. Figure 7 As shown, the method in this application embodiment may include the following steps S601-S602.

[0063] S601, the fused features are convolved using convolution kernels of various scales to obtain multiple sets of convolution features of different scales; S602, the enhanced features are obtained by adaptively fusing the multiple sets of convolutional features of different scales through a channel attention mechanism; Specifically, the multiple sets of convolutional kernels of different scales involved in this embodiment refer to a set of convolutional operations with different receptive field sizes, used to capture multi-level contextual information; the fusion feature refers to the feature representation obtained after processing by the variation-gated fusion module, which combines information from historical images and current images. To enhance the detail recovery capability of the features, the fused features need to be convolved using convolution kernels of various scales to obtain multiple sets of convolutional features at different scales. These multiple sets of convolutional features at different scales refer to multiple feature maps extracted from the fused features under different receptive fields, capturing local edges, regional textures, and global structural information respectively.

[0064] In some possible implementations, parallel convolution operations can be performed on the fused features using 3x3, 5x5, and 7x7 convolution kernels to generate multiple sets of convolutional features at different scales, ensuring that the fused features are fully utilized in the processing.

[0065] Based on this, the channel attention mechanism refers to a mechanism for automatically learning the importance weights of different feature channels; To enhance the relevance and effectiveness of feature representation, an adaptive fusion mechanism is needed to fuse multiple sets of convolutional features at different scales to obtain the enhanced features. The enhanced features refer to the feature representation after multi-scale convolutional attention processing, which possesses enhanced details and contextual information. Furthermore, the adaptive fusion process can involve calculating the statistics of each scale feature using global average pooling, and then generating weight values ​​through a fully connected layer. The importance weights can specifically represent the normalized coefficients output by the attention module.

[0066] In this embodiment, three convolutional scales are provided: 3x3, 5x5, and 7x7 convolutional kernels. The output convolutional features can be F1, F2, and F3, corresponding to feature maps of different scales. Wherein, F1 = Conv 3×3 (F f ), F2=Conv 5×5 (F f ), F3=Conv 7×7 (F f ), F f This indicates the fusion feature.

[0067] Importance weights can be calculated, and the specific calculation formula is as follows: α i =Softmax(W i ×GAP(F i )); Where αi represents the importance weight of the i-th scale feature, and i can take values ​​of 1, 2, and 3, W i GAP(F) represents the learnable weight matrix. i ) indicates that global average pooling is performed on the i-th convolutional feature.

[0068] After obtaining the importance weights of convolutions at different scales, the enhanced features can be calculated using the following formula: ; in ， F h Indicates enhanced features ， α i Indicates importance weight ， F i This represents the convolutional feature at the i-th scale.

[0069] After obtaining the enhanced features, the enhanced features can be further decoded to obtain a high-resolution reconstructed image of the target.

[0070] In this embodiment, multi-scale convolutional kernels are used to process the fused features separately, enabling the extraction of structural information at different levels from local to global. This allows the model to simultaneously perceive the edge details and overall layout of buildings. Subsequently, a channel attention mechanism generates weights representing the importance of features at each scale by performing global average pooling on multiple sets of convolutional features at different scales. These weights can adaptively adjust the contribution ratio of features at different scales in the fusion process. Finally, by multiplying the weights by the corresponding convolutional features and summing the results, the enhanced features retain both the fine texture extracted by the 3×3 convolution and the macroscopic contours captured by the 7×7 convolution. Therefore, the multi-scale convolutional attention module in this embodiment functionally achieves dynamic optimization of the fused features, enabling the reconstructed images to maintain edge sharpness and structural continuity in complex structures such as densely built-up areas in mountainous regions and road intersections. This effectively solves the problem of detail loss caused by the fixed receptive field in complex terrain using traditional methods.

[0071] In the embodiments of this application, the reconstruction model formed by the above method can adopt a joint loss function during training. The specific implementation process can be as follows: calculate the reconstruction loss between the target high-resolution reconstructed image and the real image, the perceptual loss based on the feature space, the edge loss based on the gradient, and the gate constraint loss based on the changing weight map, and then sum the loss terms by weighting coefficients.

[0072] The formula for the joint loss function is: L=λ1×L rec +λ2×L perc +λ3×L edge +λ4×L gate ; Where L represents the joint loss function, and λ1, λ2, λ3, and λ4 represent weighting coefficients used to balance the optimization intensity of each loss term. rec Representing reconstruction loss, L refers to the loss function used to constrain pixel-level accuracy. perc Perceptual loss refers to the loss function used to maintain the consistency of texture in the feature space. edge The edge loss function, L, refers to the loss function used to enhance the clarity of building and road boundaries. gate The gated constraint loss refers to the loss function used to guide the learning of the change weight map, which is determined based on the difference between the change weight map and the true change mask. The true change mask can be data used to indicate the true changes in ground features between historical low-resolution imagery and current high-resolution imagery, such as data generated through manual annotation or high-precision change detection algorithms.

[0073] The following will combine Figure 8This application provides a detailed description of the super-resolution reconstruction apparatus for remote sensing images of complex terrain and buildings, as described in the embodiments of this application. It should be noted that... Figure 8 The super-resolution reconstruction apparatus for remote sensing images of complex terrain and buildings is specifically used to perform the functions described in this application. Figures 1-7 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 1-7 The example shown.

[0074] Please see Figure 8 This is a schematic diagram of the structure of a super-resolution reconstruction device for remote sensing images of complex terrain and buildings, provided in an embodiment of this application. Figure 8 As shown, the device 1 may include an image acquisition unit 11, a first feature acquisition unit 12, a second feature acquisition unit 13, an information acquisition unit 14, a third feature acquisition unit 15, a fourth feature acquisition unit 16, and a feature decoding unit 17.

[0075] The image acquisition unit 11 is used to acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographical area acquired at different time points. The first feature acquisition unit 12 is used to extract features from the historical low-resolution image by reconstructing the main branch, and obtain the main branch features. The second feature acquisition unit 13 is used to extract features from the current high-resolution image and perform semantic alignment through the guide branch to obtain guide branch features; The information acquisition unit 14 is used to calculate the difference between the main branch feature and the guiding branch feature, obtain feature difference information, and determine the change weight map based on the feature difference information. The third feature acquisition unit 15 is used to adaptively weight and fuse the main branch features and the guiding branch features according to the change weight map to obtain the fused features; The fourth feature acquisition unit 16 is used to perform multi-scale convolutional attention processing on the fused features to obtain enhanced features; The feature decoding unit 17 is used to decode the enhanced features to obtain a high-resolution reconstructed image of the target.

[0076] In one embodiment, the first feature acquisition unit 12 is specifically used for: The historical low-resolution images are convolutionally mapped to obtain initial features; The initial features are hierarchically processed based on the residual Swin block to obtain the main branch features.

[0077] In one embodiment, the second feature acquisition unit 13 is specifically used for: The current high-resolution image is processed using a lightweight Swin Transformer encoder to obtain the first multi-scale features; Predict a learnable offset field based on the first multi-scale feature; Based on the learnable offset field, the first multi-scale features are adaptively sampled using deformable convolution to obtain the guiding branch features.

[0078] In one embodiment, the information acquisition unit 14 is specifically used for: Calculate the difference between the main branch features and the guiding branch features to obtain feature difference information; The feature difference information is input into a preset gating network; The changed weight map is output through the gating network.

[0079] In one embodiment, the third feature acquisition unit 15 is specifically used for: The first weighted feature is obtained by multiplying the change weight map with the guiding support feature; The second weighted feature is obtained by inverting the change weight map and multiplying it by the main branch feature. The first weighted feature and the second weighted feature are combined to obtain the fused feature.

[0080] In one embodiment, the fourth feature acquisition unit 16 is specifically used for: The fused features are convolved using convolution kernels of various scales to obtain multiple sets of convolutional features of different scales; The enhanced features are obtained by adaptively fusing multiple sets of convolutional features of different scales through a channel attention mechanism.

[0081] In one embodiment, the multi-scale convolution kernels include 3×3 convolution kernels, 5×5 convolution kernels, and 7×7 convolution kernels.

[0082] In one embodiment, when the fourth feature acquisition unit 16 performs the step of adaptively fusing the multiple sets of convolutional features of different scales through a channel attention mechanism to obtain the enhanced features, it specifically performs the following operations: Global average pooling is performed on each group of convolutional features to determine the importance weights of each group of convolutional features; The enhanced features are obtained by weighted summation of the multiple sets of convolutional features at different scales according to the importance weights.

[0083] In one embodiment, the apparatus further includes: An optimization unit is used to train and optimize the method based on a joint loss function.

[0084] In one embodiment, the joint loss function includes reconstruction loss, perception loss, edge loss, and gating constraint loss.

[0085] In one embodiment, the gating constraint loss is determined based on the difference between the change weight map and the true change mask.

[0086] In one embodiment, the target high-resolution reconstructed image is used to accurately restore historical scenes in complex terrain areas, and the change weight map is used to indicate the areas of ground feature change between the historical low-resolution image and the current high-resolution image.

[0087] In this embodiment, historical low-resolution images and current high-resolution images are first acquired. Main branch features are extracted from the historical low-resolution images by reconstructing the main branch to construct a structural baseline. Simultaneously, guide branch features are extracted from the current high-resolution images by extracting features and performing semantic alignment to obtain guide branch features, thus acquiring high-frequency priors. Next, feature difference information is obtained by calculating the difference between the main branch features and the guide branch features. Based on this information, a change weight map is determined. Then, the main branch features and guide branch features are adaptively weighted and fused according to this change weight map. This process explicitly models ground feature changes through a gating mechanism, thereby maintaining temporal consistency in invariant areas. First, erroneous texture migration is suppressed in changing regions. Then, multi-scale convolutional attention processing is applied to the fused features. This process uses convolutional kernels of multiple scales to simultaneously capture macroscopic structures and microscopic details, thereby enhancing the ability to restore building edges and road textures under complex terrain. Finally, the enhanced features are decoded to generate a high-resolution reconstructed image of the target. Therefore, through the synergistic effect of the above steps, this scheme overcomes the structural blurring and artifact problems caused by the single use of information, unstable registration and fusion, and lack of change perception in existing technologies. Thus, without relying on explicit terrain data, temporal consistency and high-fidelity reconstruction of remote sensing images of buildings in complex terrain are achieved.

[0088] It should be noted that the super-resolution reconstruction device for complex terrain and building remote sensing images provided in the above embodiments is only illustrated by the division of the above functional modules when executing the super-resolution reconstruction method for complex terrain and building remote sensing images. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the super-resolution reconstruction device for complex terrain and building remote sensing images provided in the above embodiments and the super-resolution reconstruction method embodiments for complex terrain and building remote sensing images belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0089] The sequence numbers of the embodiments described above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0090] This application embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described functionality. Figures 2-7 The super-resolution reconstruction method for remote sensing images of complex terrain and buildings in the illustrated embodiment can be found in the following documentation for the specific execution process: Figures 2-7 The specific details of the illustrated embodiments will not be elaborated here.

[0091] Please see Figure 9 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this application. The electronic device in this application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0092] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or more of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0093] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0094] The memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly.

[0095] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0096] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 can be a touch display screen.

[0097] The touch display screen can be designed as a full-screen, curved screen, or irregularly shaped screen. It can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; however, this application does not limit the specific design in this regard.

[0098] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, WiFi modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0099] exist Figure 9 In the illustrated electronic device, the processor 110 can be used to call computer applications stored in the memory 120 and specifically perform the following operations: Acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographic area acquired at different times. The main branch features are obtained by reconstructing the main branch and extracting features from the historical low-resolution image. The guide branch features are obtained by extracting features from the current high-resolution image and performing semantic alignment. Calculate the difference between the main branch feature and the guiding branch feature to obtain feature difference information, and determine the change weight map based on the feature difference information; Based on the change weight map, the main branch features and the guiding branch features are adaptively weighted and fused to obtain the fused features; The fused features are subjected to multi-scale convolutional attention processing to obtain enhanced features; The enhanced features are decoded to obtain a high-resolution reconstructed image of the target.

[0100] In one embodiment, when the processor 110 performs feature extraction on the historical low-resolution image by reconstructing the main branch to obtain the main branch features, it specifically performs the following operations: The historical low-resolution images are convolutionally mapped to obtain initial features; The initial features are hierarchically processed based on the residual Swin block to obtain the main branch features.

[0101] In one embodiment, when the processor 110 performs feature extraction and semantic alignment on the current high-resolution image using the guide branch to obtain guide branch features, it specifically performs the following operations: The current high-resolution image is processed using a lightweight Swin Transformer encoder to obtain the first multi-scale features; Predict a learnable offset field based on the first multi-scale feature; Based on the learnable offset field, the first multi-scale features are adaptively sampled using deformable convolution to obtain the guiding branch features.

[0102] In one embodiment, when the processor 110 executes the operation of determining the change weight map based on the feature difference information, it specifically performs the following operations: The feature difference information is input into a preset gating network; The changed weight map is output through the gating network.

[0103] In one embodiment, when the processor 110 performs adaptive weighted fusion of the main branch features and the guiding branch features based on the changing weight map to obtain fused features, it specifically performs the following operations: The first weighted feature is obtained by multiplying the change weight map with the guiding support feature; The second weighted feature is obtained by inverting the change weight map and multiplying it by the main branch feature. The first weighted feature and the second weighted feature are combined to obtain the fused feature.

[0104] In one embodiment, when the processor 110 performs multi-scale convolutional attention processing on the fused features to obtain enhanced features, it specifically performs the following operations: The fused features are convolved using convolution kernels of various scales to obtain multiple sets of convolutional features of different scales; The enhanced features are obtained by adaptively fusing multiple sets of convolutional features of different scales through a channel attention mechanism.

[0105] In one embodiment, the multi-scale convolution kernels include 3×3 convolution kernels, 5×5 convolution kernels, and 7×7 convolution kernels.

[0106] In one embodiment, when the processor 110 performs adaptive fusion of the multiple sets of convolutional features of different scales through a channel attention mechanism to obtain the enhanced features, it specifically performs the following operations: Global average pooling is performed on each group of convolutional features to determine the importance weights of each group of convolutional features; The enhanced features are obtained by weighted summation of the multiple sets of convolutional features at different scales according to the importance weights.

[0107] In one embodiment, the processor 110 also performs the following operations: The method is trained and optimized based on the joint loss function.

[0108] In one embodiment, the joint loss function includes reconstruction loss, perception loss, edge loss, and gating constraint loss.

[0109] In one embodiment, the gating constraint loss is determined based on the difference between the change weight map and the true change mask.

[0110] In one embodiment, the target high-resolution reconstructed image is used to accurately restore historical scenes in complex terrain areas, and the change weight map is used to indicate the areas of ground feature change between the historical low-resolution image and the current high-resolution image.

[0111] In this embodiment, historical low-resolution images and current high-resolution images are first acquired. Main branch features are extracted from the historical low-resolution images by reconstructing the main branch to construct a structural baseline. Simultaneously, guide branch features are extracted from the current high-resolution images by extracting features and performing semantic alignment to obtain guide branch features, thus acquiring high-frequency priors. Next, feature difference information is obtained by calculating the difference between the main branch features and the guide branch features. Based on this information, a change weight map is determined. Then, the main branch features and guide branch features are adaptively weighted and fused according to this change weight map. This process explicitly models ground feature changes through a gating mechanism, thereby maintaining temporal consistency in invariant areas. First, erroneous texture migration is suppressed in changing regions. Then, multi-scale convolutional attention processing is applied to the fused features. This process uses convolutional kernels of multiple scales to simultaneously capture macroscopic structures and microscopic details, thereby enhancing the ability to restore building edges and road textures under complex terrain. Finally, the enhanced features are decoded to generate a high-resolution reconstructed image of the target. Therefore, through the synergistic effect of the above steps, this scheme overcomes the structural blurring and artifact problems caused by the single use of information, unstable registration and fusion, and lack of change perception in existing technologies. Thus, without relying on explicit terrain data, temporal consistency and high-fidelity reconstruction of remote sensing images of buildings in complex terrain are achieved.

[0112] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0113] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for super-resolution reconstruction of remote sensing images of buildings in complex terrain, characterized in that, include: Acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographic area acquired at different times. The main branch features are obtained by reconstructing the main branch and extracting features from the historical low-resolution image. The guide branch features are obtained by extracting features from the current high-resolution image and performing semantic alignment. The difference between the main branch feature and the guiding branch feature is calculated to obtain feature difference information. The feature difference information is then input into a preset gating network, and a change weight map is output through the gating network. The change weight map is used to control the ratio of the main branch feature and the guiding branch feature during fusion. The first weighted feature is obtained by multiplying the change weight map with the guiding support feature; The second weighted feature is obtained by inverting the change weight map and multiplying it by the main branch feature. The first weighted feature and the second weighted feature are combined to obtain the fused feature; The fused features are subjected to multi-scale convolutional attention processing to obtain enhanced features; Decoding the enhanced features yields a high-resolution reconstructed image of the target. The method of controlling the ratio of the main branch features to the guiding branch features during fusion based on the change weight map includes: For regions with small feature differences, the change weight map enhances the influence of the guiding branch features; For regions with large feature differences, the change weight map reduces the influence of the guiding branch features and increases the influence of the main branch features.

2. The method according to claim 1, characterized in that, The step of extracting features from the historical low-resolution image by reconstructing the main branch to obtain the main branch features includes: The historical low-resolution images are convolutionally mapped to obtain initial features; The initial features are hierarchically processed based on the residual Swin block to obtain the main branch features.

3. The method according to claim 1, characterized in that, The step of extracting features from the current high-resolution image using a guide branch and performing semantic alignment to obtain guide branch features includes: The current high-resolution image is processed using a lightweight Swin Transformer encoder to obtain the first multi-scale features; Predict a learnable offset field based on the first multi-scale feature; Based on the learnable offset field, the first multi-scale features are adaptively sampled using deformable convolution to obtain the guiding branch features.

4. The method according to claim 1, characterized in that, The process of performing multi-scale convolutional attention processing on the fused features to obtain enhanced features includes: The fused features are convolved using convolution kernels of various scales to obtain multiple sets of convolutional features of different scales; The enhanced features are obtained by adaptively fusing multiple sets of convolutional features of different scales through a channel attention mechanism.

5. The method according to claim 4, characterized in that, The various convolutional kernels include 3×3, 5×5, and 7×7 kernels.

6. The method according to claim 4, characterized in that, The enhanced features are obtained by adaptively fusing multiple sets of convolutional features of different scales through a channel attention mechanism, including: Global average pooling is performed on each group of convolutional features to determine the importance weights of each group of convolutional features; The enhanced features are obtained by weighted summation of the multiple sets of convolutional features at different scales according to the importance weights.

7. The method according to claim 1, characterized in that, The method further includes: The method is trained and optimized based on the joint loss function.

8. The method according to claim 7, characterized in that, The joint loss function includes reconstruction loss, perception loss, edge loss, and gating constraint loss.

9. The method according to claim 8, characterized in that, The gated constraint loss is determined based on the difference between the change weight map and the actual change mask.

10. The method according to claim 1, characterized in that, The target high-resolution reconstructed image is used to accurately restore historical scenes in complex terrain areas, and the change weight map is used to indicate the area of ​​ground feature change between the historical low-resolution image and the current high-resolution image.

11. A super-resolution reconstruction device for remote sensing images of buildings in complex terrain, characterized in that, include: The image acquisition unit is used to acquire historical low-resolution images and current high-resolution images, wherein the historical low-resolution images and the current high-resolution images are remote sensing images of the same geographical area acquired at different time points. The first feature acquisition unit is used to extract features from the historical low-resolution image by reconstructing the main branch, and obtain the main branch features. The second feature acquisition unit is used to extract features from the current high-resolution image and perform semantic alignment through the guide branch to obtain guide branch features; An information acquisition unit is used to calculate the difference between the main branch feature and the guiding branch feature to obtain feature difference information, and input the feature difference information into a preset gating network. The gating network outputs a change weight map, which is used to control the ratio of the main branch feature and the guiding branch feature during fusion. The third feature acquisition unit is used to perform a dot product between the change weight map and the guiding branch feature to obtain the first weighted feature; The third feature acquisition unit is further configured to invert the change weight map and then multiply it by the main branch feature to obtain the second weighted feature; The third feature acquisition unit is also used to merge the first weighted feature and the second weighted feature to obtain a fused feature; The fourth feature acquisition unit is used to perform multi-scale convolutional attention processing on the fused features to obtain enhanced features; A feature decoding unit is used to decode the enhanced features to obtain a high-resolution reconstructed image of the target; The processing unit is configured to, for regions with small feature differences, increase the influence of the guiding branch feature using the change weight map; and for regions with large feature differences, decrease the influence of the guiding branch feature and increase the influence of the main branch feature using the change weight map.

12. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1 to 10.

Images