A DEM super-resolution reconstruction system based on RGB optical image guidance

Abstract

The application discloses a DEM super-resolution reconstruction system based on RGB optical image guidance, which comprises a multi-scale feature extraction module, a cross-modal fusion unit, a three-level progressive cross-modal fusion module and a multi-scale progressive reconstruction module based on global residual learning. The multi-scale feature extraction module is used for receiving DEM data and RGB optical images respectively, and generating first-layer RGB features and first-layer DEM features; the cross-modal fusion unit outputs channel attention input features for a DEM branch and refined guidance features for an RGB branch; the three-level progressive cross-modal fusion module iteratively refines multi-scale terrain features from shallow to deep, and completes layer-by-layer fusion alignment of local micro-landform details and middle-layer and high-layer terrain semantic features; finally, the multi-scale refined fusion features are input into the multi-scale progressive reconstruction module, and high-resolution DEM reconstruction results are output. The application can fully tap the cross-modal complementary advantages of RGB and DEM, and realize efficient and high-precision DEM super-resolution reconstruction.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to a DEM super-resolution reconstruction system based on RGB optical image guidance. Background Technology

[0002] Digital Elevation Models (DEMs) are core geospatial data that characterize the undulations of the Earth's surface. They are widely used in various geographic national condition analysis tasks, such as hydrological runoff simulation, landslide geological disaster early warning, soil and water conservation monitoring, urban topographic planning, and ecological environment assessment. The spatial resolution and accuracy of topographic details directly determine the reliability of geographic modeling, quantitative analysis, and engineering applications. In actual remote sensing observation and geographic data acquisition scenarios, acquiring high-precision DEM data with large-scale, full-coverage coverage is extremely costly. Due to multiple limitations, including satellite remote sensing imaging resolution, terrain occlusion, acquisition equipment accuracy, and field surveying conditions, existing open-source and batch-acquired DEM data generally suffer from low spatial resolution, missing micro-topographic textures, blurred areas of elevation abrupt changes, and local topographic distortion. They are unable to accurately represent fine topographic structures such as gullies, slope breaks, and micro-undulations, and cannot meet the application requirements of high-precision topographic analysis, refined disaster prediction, and high-precision three-dimensional topographic reconstruction.

[0003] Currently, DEM super-resolution reconstruction technology is the mainstream method for improving the accuracy of DEM data and restoring fine terrain features at low cost. Traditional interpolation methods can only achieve simple smooth upsampling at the pixel scale, making it difficult to learn the spatial distribution patterns of real terrain, which easily leads to blurred terrain details and structural distortion. Although deep learning methods based on convolutional neural networks can mine the correlation between local terrain features and textures, they are limited by the inherent defect of the fixed local receptive field of the convolution operator, making it difficult to adaptively capture multi-scale terrain context information. Moreover, most methods do not sufficiently mine the cross-modal complementary information between RGB optical imagery and DEM elevation data, resulting in problems such as shallow modal fusion layers, weak texture detail guidance ability, and poor accuracy in reconstructing elevation abrupt changes and micro-topographic regions, making it difficult to achieve coordinated and accurate reconstruction of global terrain structure and local micro-topographic details.

[0004] High-resolution RGB remote sensing imagery contains rich information on surface texture, edge contours, and topographic boundaries, exhibiting a strong spatial coupling with DEM elevation changes. This information can serve as high-precision prior information to guide detailed DEM reconstruction. Addressing the shortcomings of existing technologies, such as insufficient modal fusion, weak multi-scale terrain detail representation capabilities, and inadequate flexibility of conventional convolutional modeling, there is an urgent need for a method or system that can fully leverage the complementary advantages of RGB and DEM across modalities to achieve efficient and high-precision DEM super-resolution reconstruction. Summary of the Invention

[0005] In view of this, the present invention provides a DEM super-resolution reconstruction system based on RGB optical image guidance, so as to fully exploit the complementary advantages of RGB and DEM across modes and achieve efficient and high-precision DEM super-resolution reconstruction.

[0006] A DEM super-resolution reconstruction system based on RGB optical images includes a multi-scale feature extraction module, a cross-modal fusion unit, a three-level progressive cross-modal fusion module, and a multi-scale progressive reconstruction module based on global residual learning. The multi-scale feature extraction module is used to receive DEM data and registered RGB optical images respectively. Through parallel RGB branches and DEM branches, it extracts the initial shallow texture features of the RGB image and the initial shallow elevation structure features of the DEM data respectively, and generates the first-layer RGB features and the first-layer DEM features. The cross-modal fusion unit is used to receive and align the RGB features and DEM features to be fused, and then generate intermediate fusion features through channel dimension concatenation, convolutional compression and deep convolutional gated feature modulation. Based on the intermediate fusion features, it outputs channel attention input features for the DEM branch and refined guidance features for the RGB branch. In the three-layer progressive cross-modal fusion module, the first layer of the fusion module receives the first-layer RGB features, the first-layer DEM features, and the corresponding channel attention input features and refined guidance features of the first layer. Through residual fusion and residual group feature extraction, it generates the second-layer RGB features and the second-layer DEM features. The second layer of the fusion module receives the second-layer RGB features, the second-layer DEM features, and the corresponding channel attention input features and refined guidance features of the first layer. It reuses the fusion paradigm of the first layer to generate the third-layer RGB features and the third-layer DEM features. The third layer of the fusion module receives the third-layer RGB features, the third-layer DEM features, and the corresponding channel attention input features and refined guidance features of the third layer. It uses a differentiated structure to perform high-level semantic modeling to generate the top-layer RGB features and the top-layer DEM features. The multi-scale progressive reconstruction module based on global residual learning is used to receive top-level RGB features, first-level DEM features, second-level DEM features, third-level DEM features, and top-level DEM features. It adopts a multi-scale progressive reconstruction architecture from deep to shallow, fusing and generating intermediate reconstruction features layer by layer. Finally, it generates a depth residual map through tail and head processing, and fuses the depth residual map with the upsampled original DEM data element by element to output a high-resolution DEM reconstruction result.

[0007] The DEM super-resolution reconstruction system based on RGB optical image guidance provided by the present invention has the following beneficial effects: 1. To address the issues of blurred terrain details and structural distortion caused by traditional interpolation methods, this invention employs a multi-scale progressive reconstruction module based on global residual learning. This module fuses the depth residual map predicted by the network with the low-resolution DEM upsampled by bicubic interpolation element-wise, guiding the network to focus on compensating for high-frequency terrain details and elevation changes. This effectively restores micro-topographic textures and edge structures, avoiding over-smoothing.

[0008] 2. To address the problem that traditional convolutional networks have a fixed receptive field and are difficult to adaptively capture multi-scale terrain context information, this invention introduces a lightweight deep convolutional gated feature modulation unit. By expanding the receptive field through 5×5 deep convolution and combining it with gated branch adaptive modulation, the ability to model the context of complex terrain multi-scale features is enhanced, thus making up for the lack of flexibility in conventional convolutional modeling.

[0009] 3. To address the issues of shallow cross-modal fusion layers and weak RGB texture guidance capabilities in existing methods, this invention designs a cross-modal fusion unit and a three-level progressive cross-modal fusion module. Through iterative refinement layer by layer from shallow to deep, it can achieve precise alignment between RGB texture priors and DEM elevation features from local details to high-level semantics. This fully leverages the complementary advantages of RGB texture priors and DEM elevation features, effectively compensating for the shortcomings of traditional methods, such as single feature modeling and insufficient detail restoration capabilities.

[0010] 4. To address the issues of poor reconstruction accuracy and loss of micro-topographic details in areas with abrupt elevation changes, this invention employs a pixel-level L1 loss function, which is more robust to abnormal errors such as terrain noise and elevation changes, effectively suppressing over-smoothing. Combined with multi-scale progressive reconstruction and tail refinement modules, it significantly improves the reconstruction accuracy of areas with abrupt elevation changes and micro-topographic details.

[0011] 5. This invention adopts a lightweight deep gating design and selectively removes redundant gating units in the third-layer fusion. The overall network has low parameter quantity and computational overhead while ensuring high reconstruction accuracy, which is convenient for engineering deployment. It has important theoretical research value and engineering practice significance for low-cost and high-efficiency acquisition of high-precision DEM data across the entire region, promoting the application of refined geographic remote sensing analysis, precise monitoring of geological disasters and refined land space planning. Attached Figure Description

[0012] Figure 1 This is a structural block diagram of a DEM super-resolution reconstruction system based on RGB optical image guidance provided in an embodiment of the present invention. Detailed Implementation

[0013] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0014] Please see Figure 1 The DEM super-resolution reconstruction system based on RGB optical image guidance provided in this embodiment of the invention includes a multi-scale feature extraction module, a cross-modal fusion unit, a three-level progressive cross-modal fusion module, and a multi-scale progressive reconstruction module based on global residual learning.

[0015] The multi-scale feature extraction module is used to receive DEM data and registered RGB optical images respectively. Through parallel RGB branches and DEM branches, it extracts the initial shallow texture features of the RGB image and the initial shallow elevation structure features of the DEM data respectively, and generates the first-layer RGB features and the first-layer DEM features.

[0016] The cross-modal fusion unit is used to receive and align the RGB features and DEM features to be fused, and then generate intermediate fusion features through channel dimension concatenation, convolutional compression and deep convolutional gating feature modulation. Based on the intermediate fusion features, it outputs channel attention input features for the DEM branch and refined guiding features for the RGB branch.

[0017] In the three-layer progressive cross-modal fusion module, the first layer receives the first-layer RGB features, the first-layer DEM features, and the corresponding channel attention input features and refined guided features. Through residual fusion and residual group feature extraction, it generates the second-layer RGB features and the second-layer DEM features. The second layer receives the second-layer RGB features, the second-layer DEM features, and the corresponding channel attention input features and refined guided features. It reuses the first-layer fusion paradigm to generate the third-layer RGB features and the third-layer DEM features. The third layer receives the third-layer RGB features, the third-layer DEM features, and the corresponding channel attention input features and refined guided features. It uses a differentiated structure for high-level semantic modeling to generate the top-layer RGB features and the top-layer DEM features. This three-layer progressive cross-modal fusion module achieves hierarchical mining and precise alignment of multi-scale terrain features from local details to global semantics.

[0018] The multi-scale progressive reconstruction module based on global residual learning is used to receive top-level RGB features, first-level DEM features, second-level DEM features, third-level DEM features, and top-level DEM features. It adopts a multi-scale progressive reconstruction architecture from deep to shallow, fusing and generating intermediate reconstruction features layer by layer. Finally, it generates a depth residual map through tail and head processing, and fuses the depth residual map with the upsampled original DEM data element by element to output a high-resolution DEM reconstruction result.

[0019] The overall data flow of the system is as follows: Using low-resolution DEM data and high-resolution RGB optical imagery as dual-modal inputs, the system first extracts low-level features for both modalities through a multi-scale feature extraction module, separately mining the texture edge information of the RGB imagery and the elevation structure information of the DEM data. Then, the dual-modal features are fed into a cross-modal fusion unit to achieve adaptive alignment and preliminary interactive enhancement of single-layer RGB-DEM complementary features. Further, a three-level progressive cross-modal fusion module iteratively refines multi-scale terrain features from shallow to deep layers, completing the layer-by-layer fusion and alignment of local micro-topographic details with mid- and high-level terrain semantic features. Finally, the refined multi-scale fusion features are input into a multi-scale progressive reconstruction module, employing a deep-to-shallow feature reconstruction strategy combined with a global residual learning mechanism to accurately compensate for high-frequency terrain details and elevation abrupt changes, outputting a high-precision, detailed, high-resolution DEM reconstruction result.

[0020] The following is a detailed description of this embodiment. First, the basic components of the present invention are described. The basic components include a Residual Channel Attention Block (RCAB) and a residual group, as well as a Deep Convolutional Gated Feature Modulation Unit (DW-GateBlock).

[0021] The residual channel attention block is the core local feature extraction unit of this invention. It can adaptively enhance effective features and suppress redundant noise while preserving the original terrain structure information. At the same time, it can alleviate the gradient degradation problem of deep networks by leveraging the residual structure. The complete forward inference process of RCAB can be divided into four steps: local feature extraction, global channel information aggregation, channel attention weight generation, feature adaptive modulation and residual fusion.

[0022] Specifically, the residual channel attention block is used to perform the following operations: (1) Local feature extraction: for input features Two consecutive 3×3 standard convolutions are executed sequentially, with a PReLU non-linear activation function embedded between the convolutional layers to obtain local features. Extract information on local spatial texture, edges and elevation changes of the terrain to enhance basic features; (2) Global channel information aggregation: local features Global average pooling is performed to aggregate global statistical information for each channel, thereby achieving feature compression in the spatial dimension. (3) Attention weight generation: Attention weights in the [0,1] interval are generated by 1×1 convolution dimensionality reduction, PReLU activation, 1×1 convolution dimensionality increase, and then normalization by the Sigmoid function. Adaptive characterization of the importance of features in different channels; (4) Feature modulation and residual output: Attention weights With local features Element-wise multiplication is used to enhance key features and suppress invalid features. Finally, residual connections are used to combine the modulated features with the input features. Add them together, retain the underlying basic information, and output the features. .

[0023] The expression for the above process is:

[0024]

[0025]

[0026] in, PReLU is a non-linear activation function; This is a global average pooling operator; , These are 1×1 convolutions for channel-wise dimensionality reduction and dimensionality enhancement, respectively. Use the Sigmoid activation function; This represents element-wise Hadamard multiplication.

[0027] Based on residual channel attention blocks, this invention also defines residual groups. ,Depend on The residual channel attention blocks are stacked in series to iteratively extract refined multi-scale local spatial features, thereby enhancing the network's channel attention representation and deep feature extraction capabilities.

[0028] To balance computational efficiency and feature representation capabilities, this invention introduces a deep convolutional gated feature modulation unit for local context modeling and adaptive feature modulation. This unit abandons complex global modeling structures, employing lightweight deep convolution and channel gating mechanisms. It leverages deep convolution and adaptive gating to enhance multi-scale terrain context modeling capabilities, overcoming the shortcomings of traditional convolutional modeling in terms of flexibility and weak detail representation. This expands the receptive field with lower parameter overhead, strengthening the network's ability to capture complex terrain structures and micro-topographic details.

[0029] The deep convolutional gated feature modulation unit is used to perform the following operations: Using two sets of 1×1 convolutions to process the input features The number of channels is determined by Mapped to , to obtain projection features In the channel dimension Equal splitting is the main branch feature With gated branch features The main branch uses a 5×5 depthwise convolution to extract local spatial context features; the gated branch generates adaptive gate weights through a 1×1 linear mapping and sigmoid activation, dynamically modulating the main branch features; finally, the gate weights are multiplied element-wise with the main branch output, and then multiplied by the input features after another 1×1 linear mapping. Add together to output features While preserving the underlying infrastructure information, it enhances the response strength of key terrain features.

[0030] The expression for the above process is:

[0031] in, This represents a 5×5 depthwise convolution operator; This represents a 1×1 convolutional linear mapping.

[0032] Because topography and landforms exhibit typical multi-scale heterogeneity, shallow features contain details of micro-topographic textures, terrain edges, and local elevation undulations, while deep features characterize the overall topographic structure and large-scale elevation change trends. Single-scale feature representation capabilities are limited and cannot simultaneously meet the dual requirements of detail preservation and global structure modeling. Therefore, this invention designs a dual-modal multi-scale feature extraction module (MSFE), constructing a high-resolution RGB branch and a low-resolution DEM branch, specifically for initial shallow feature mining of the original data, providing fundamental feature support for subsequent cross-modal progressive fusion.

[0033] Let the original input high-resolution RGB image be... Low-resolution DEM data is .in, For the real number field, and These represent the height and width of the image, respectively; 3 represents the three-channel dimension of the RGB image, and 1 represents the single-channel elevation dimension of the DEM.

[0034] The RGB branch aims to extract rich texture and edge information, providing structural priors for DEM reconstruction; the DEM branch focuses on extracting terrain structural features and capturing elevation change patterns. This module uses convolutional layers and residual groups to complete the initial feature extraction, while the RGB branch additionally introduces a lightweight deep convolutional gated feature modulation unit for adaptive feature enhancement. This module is only responsible for generating the first layer of RGB features. and first-layer DEM features .

[0035] Specifically, the RGB branch is composed of a first 3×3 standard convolutional layer, a LeakyReLU activation function, a first residual group, and a deep convolutional gated feature modulation unit connected in series. The first residual group contains one residual channel attention block. The RGB branch satisfies the following equation:

[0036] in, For the first layer of RGB features, It is a deep convolutional gated feature modulation unit used to perform adaptive context enhancement on RGB branch texture features, thereby enhancing terrain edges and texture details; For a residual group containing one residual channel attention block, This is the LeakyReLU activation function, used to introduce nonlinearity and alleviate gradient vanishing. A standard 3×3 convolution is used to complete the initial spatial feature mapping; For input RGB optical images; The DEM branch is composed of a second 3×3 standard convolutional layer, a LeakyReLU activation function, and a second residual group concatenated in series. The second residual group contains four residual channel attention blocks. The DEM branch satisfies the following equation:

[0037] in, Features of the first layer DEM For a residual group containing 4 residual channel attention blocks, This refers to the input DEM data.

[0038] Multi-scale features of intermediate and deep layers and All features are generated through iterative refinement by a three-level progressive cross-modal fusion module, achieving gradual optimization of features from shallow to deep and from local details to high-level semantics.

[0039] Under the dual-branch structure, the RGB branch focuses on mining optical prior information such as surface texture, landform boundaries, and slope break edges, while the DEM branch focuses on extracting elevation features such as elevation undulation, terrain gradient, and basic landform structure. This provides high-quality and differentiated initial dual-modal feature input for subsequent three-level progressive cross-modal fusion, ensuring the effectiveness of subsequent multi-scale fusion and high-precision terrain reconstruction.

[0040] Because RGB optical images possess rich texture and edge detail information, and DEM depth features characterize terrain elevation changes, there is a strong spatial correlation between the two. To achieve adaptive depth fusion of RGB-DEM dual-modal features, this invention designs a cross-modal fusion unit. Through channel dimension alignment, spatial resolution matching, lightweight gating context modeling, and cross-modal feature modulation, complementary information between the two modalities is mined, enabling precise guidance of DEM elevation reconstruction by RGB texture.

[0041] The cross-modal fusion unit is used to perform the following operations: High-resolution RGB features to be fused are processed separately using 1×1 convolutions. and low-resolution DEM features Channel alignment is performed; bilinear interpolation upsampling is applied to the DEM features to make their spatial resolution consistent with that of the RGB features; then, bimodal features are concatenated along the channel dimension, and channel compression and cross-modal feature fusion are performed sequentially through 1×1 convolution and 3×3 convolution to obtain the fused intermediate features. ;Will The input is fed into a deep convolutional gated feature modulation unit, which fuses the local spatial context of the features to capture terrain neighborhood association information. After PReLU activation, the input features are output through two independent 1×1 convolutions, respectively, focusing on channel attention. and refined guidance features The expression is:

[0042]

[0043] in, Indicates concatenation of channel dimensions; It is a 1×1 standard convolution.

[0044] Topographic features possess inherent multi-scale hierarchical characteristics: shallow features focus on local edges, textures, and details of elevation abrupt changes, while deep features model the global terrain structure and geomorphic semantic information. Based on this, this invention employs a three-level progressive cross-modal fusion module and a hierarchical three-stage progressive cross-modal fusion strategy to achieve hierarchical iterative refinement of RGB-DEM features from local to global and from fine-grained to coarse-grained.

[0045] The first layer of the fusion module is specifically used for: First layer RGB features and first-layer DEM features The input is fed into the cross-modal fusion unit, which outputs the channel attention input features corresponding to the first layer. and refined guidance features ; Will The data is sequentially input into the third residual group and the first depthwise convolutional gated feature modulation unit to generate the second layer DEM features. The third residual group includes four residual channel attention blocks; Will and After element-wise addition, the data is sequentially input into the fourth residual group and the second deep convolutional gated feature modulation unit to generate the second layer of RGB features. The fourth residual group includes one residual channel attention block.

[0046] The expression for the above process is:

[0047] .

[0048] The second layer of the fusion module is specifically used for: The second layer of RGB features Second layer DEM features The input is fed into the cross-modal fusion unit, which outputs the corresponding channel attention input features for the second layer. and refined guidance features ; Will The data is sequentially input into the fifth residual group and the third deep convolutional gated feature modulation unit to generate the third layer DEM features. The fifth residual group includes four residual channel attention blocks; Will and After element-wise addition, the data is sequentially input into the sixth residual group and the fourth deep convolutional gated feature modulation unit to generate the third layer of RGB features. The sixth residual group includes one residual channel attention block.

[0049] The third layer of the fusion module is specifically used for: The third layer RGB features and third-layer DEM features The input is fed into the cross-modal fusion unit, which outputs the channel attention input features corresponding to the third layer. and refined guidance features ; Will The data is directly input into the seventh residual group to generate the top-level DEM features. The seventh residual group includes four residual channel attention blocks; Will and After element-wise addition, a 1×1 convolution is introduced for channel dimension calibration to generate the top-level RGB features. .

[0050] It should be noted that in the third layer of the fusion module, the DEM branch no longer introduces deep convolutional gated feature modulation units to avoid redundancy in context modeling. The aforementioned hierarchical architecture enables hierarchical mining of multi-scale terrain features, achieving precise alignment of cross-modal features from local details to global semantics.

[0051] To achieve accurate mapping of multi-scale fused features to high-resolution DEMs, this invention constructs a multi-scale progressive reconstruction module based on global residual learning. It employs a deep-to-shallow multi-scale progressive reconstruction architecture, combined with a global residual learning mechanism, to reduce model training difficulty and enhance the compensation capability for high-frequency terrain details and elevation abrupt changes. The first-layer reconstruction branch focuses on deep global semantic features, strengthening local spatial context modeling through residual groups, deep convolutional gated feature modulation units, and 1×1 convolutions. Subsequent layers use residual groups to iteratively refine multi-scale local details.

[0052] The multi-scale progressive reconstruction module based on global residual learning specifically performs a four-layer progressive reconstruction from deep to shallow: First layer reconstruction: Reconstructing the top-level RGB features The features are added to the upsampled top-level DEM features and then sequentially input into the sixth residual group, the fourth depthwise convolutional gated feature modulation unit, and the first 1×1 convolution to generate the first reconstructed intermediate features. The sixth residual group contains two residual channel attention blocks, and the expression for the reconstruction of the first layer is:

[0053] in, For a residual group containing two residual channel attention blocks, This indicates upsampling, used to match the spatial resolution of features at different scales; Second-level reconstruction: Compared with the upsampled third layer DEM features The sums are then fed sequentially into the seventh residual group and the second 1×1 convolution to generate the second reconstructed intermediate features. The seventh residual group contains four residual channel attention blocks, and the expression for the second-layer reconstruction is:

[0054] Third-level reconstruction: Compared with the upsampled second-layer DEM features The sums are then sequentially fed into the eighth residual group and the third 1×1 convolution to generate the third reconstructed intermediate features. The eighth residual group contains four residual channel attention blocks, and the expression for the third layer reconstruction is:

[0055] Fourth layer reconstruction: Features of the first layer of DEM after upsampling The sums are then fed sequentially into the ninth residual group and the fourth 1×1 convolution to generate the fourth reconstructed intermediate features. The ninth residual group contains four residual channel attention blocks, and the expression for the fourth layer reconstruction is:

[0056] The multi-scale progressive reconstruction module based on global residual learning is also used to incorporate the fourth reconstruction intermediate features. The input is fed into the tail refinement module for feature purification; the purified features are then sequentially passed through the transposed convolution upsampling module and the deep residual reconstruction head to generate a depth residual map; to ensure the geometric rationality of the reconstructed terrain, this invention introduces a global residual learning strategy, which compares the depth residual map with the original DEM data. After performing s-fold bicubic interpolation upsampling, the results are summed element-wise to output the high-resolution DEM reconstruction result. The expression is:

[0057] in, For deep residual reconstruction head; The tail refining module consists of two sets of residual channel attention block stacking units, each containing eight residual channel attention blocks. Indicates to The result of bicubic interpolation upsampling by s times.

[0058] The above strategy preserves the low-frequency main structure of the terrain through interpolation branches, guiding the network to focus on the accurate compensation of high-frequency terrain details, textures and elevation changes, effectively improving reconstruction accuracy and model convergence speed.

[0059] DEM super-resolution reconstruction tasks require the accurate preservation of terrain elevation details, edge contours, and local geometric undulations. Losses can easily lead to overly smoothed reconstruction results and loss of details regarding abrupt terrain changes. To ensure pixel-level accurate alignment between the network reconstruction output and the true high-resolution DEM, this invention selects a method more robust to anomalies. The loss function is used as the training objective.

[0060] Let the network super-resolution reconstruction output be The corresponding true depth value of the real high-resolution DEM is Then pixel level loss Defined as:

[0061] in, The total number of effective pixels within the reconstructed high-resolution DEM. For pixel index, High-resolution DEM reconstruction results The Middle Elevation value per pixel For the corresponding real high-resolution DEM, the first The true elevation value of each pixel.

[0062] Compared to loss, The loss function is more robust to anomalous errors such as terrain noise and abrupt elevation changes, effectively suppressing the problem of over-smoothing in the reconstruction results and maximizing the preservation of edge details and local geometric changes in the DEM. In summary, this invention will... As the system's loss function ,Right now: .Will As a global optimization goal for the network, end-to-end training is achieved.

[0063] In summary, the DEM super-resolution reconstruction system based on RGB optical image guidance according to the above embodiments has the following beneficial effects: 1. To address the issues of blurred terrain details and structural distortion caused by traditional interpolation methods, this invention employs a multi-scale progressive reconstruction module based on global residual learning. This module fuses the depth residual map predicted by the network with the low-resolution DEM upsampled by bicubic interpolation element-wise, guiding the network to focus on compensating for high-frequency terrain details and elevation changes. This effectively restores micro-topographic textures and edge structures, avoiding over-smoothing.

[0064] 2. To address the problem that traditional convolutional networks have a fixed receptive field and are difficult to adaptively capture multi-scale terrain context information, this invention introduces a lightweight deep convolutional gated feature modulation unit. By expanding the receptive field through 5×5 deep convolution and combining it with gated branch adaptive modulation, the ability to model the context of complex terrain multi-scale features is enhanced, thus making up for the lack of flexibility in conventional convolutional modeling.

[0065] 3. To address the issues of shallow cross-modal fusion layers and weak RGB texture guidance capabilities in existing methods, this invention designs a cross-modal fusion unit and a three-level progressive cross-modal fusion module. Through iterative refinement layer by layer from shallow to deep, it can achieve precise alignment between RGB texture priors and DEM elevation features from local details to high-level semantics. This fully leverages the complementary advantages of RGB texture priors and DEM elevation features, effectively compensating for the shortcomings of traditional methods, such as single feature modeling and insufficient detail restoration capabilities.

[0066] 4. To address the issues of poor reconstruction accuracy and loss of micro-topographic details in areas with abrupt elevation changes, this invention employs a pixel-level L1 loss function, which is more robust to abnormal errors such as terrain noise and elevation changes, effectively suppressing over-smoothing. Combined with multi-scale progressive reconstruction and tail refinement modules, it significantly improves the reconstruction accuracy of areas with abrupt elevation changes and micro-topographic details.

[0067] 5. This invention adopts a lightweight deep gating design and selectively removes redundant gating units in the third-layer fusion. The overall network has low parameter quantity and computational overhead while ensuring high reconstruction accuracy, which is convenient for engineering deployment. It has important theoretical research value and engineering practice significance for low-cost and high-efficiency acquisition of high-precision DEM data across the entire region, promoting the application of refined geographic remote sensing analysis, precise monitoring of geological disasters and refined land space planning.

[0068] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A DEM super-resolution reconstruction system based on RGB optical image guidance, characterized in that, It includes a multi-scale feature extraction module, a cross-modal fusion unit, a three-level progressive cross-modal fusion module, and a multi-scale progressive reconstruction module based on global residual learning; The multi-scale feature extraction module is used to receive DEM data and registered RGB optical images respectively. Through parallel RGB branches and DEM branches, it extracts the initial shallow texture features of the RGB image and the initial shallow elevation structure features of the DEM data respectively, and generates the first-layer RGB features and the first-layer DEM features. The cross-modal fusion unit is used to receive and align the RGB features and DEM features to be fused, and then generate intermediate fusion features through channel dimension concatenation, convolutional compression and deep convolutional gated feature modulation. Based on the intermediate fusion features, it outputs channel attention input features for the DEM branch and refined guidance features for the RGB branch. In the three-level progressive cross-modal fusion module, the first layer of the fusion module is used to receive the first-layer RGB features, the first-layer DEM features, and the corresponding channel attention input features and refined guidance features of the first layer. Through residual fusion and residual group feature extraction, the second-layer RGB features and the second-layer DEM features are generated. The second layer of the fusion module is used to receive the second layer RGB features, the second layer DEM features, and the corresponding channel attention input features and refined guidance features of the first layer. It reuses the first layer fusion paradigm to generate the third layer RGB features and the third layer DEM features. The third layer of the fusion module is used to receive the third-layer RGB features, the third-layer DEM features, and the corresponding channel attention input features and refined guidance features of the third layer. It uses a differentiated structure to perform high-level semantic modeling and generate the top-level RGB features and the top-level DEM features. The multi-scale progressive reconstruction module based on global residual learning is used to receive top-level RGB features, first-level DEM features, second-level DEM features, third-level DEM features and top-level DEM features. It adopts a multi-scale progressive reconstruction architecture from deep to shallow, fusing and generating intermediate reconstruction features layer by layer. Finally, it generates a depth residual map through tail and head processing, and fuses the depth residual map with the upsampled original DEM data element by element to output a high-resolution DEM reconstruction result. In the multi-scale feature extraction module: The RGB branch is composed of a first 3×3 standard convolutional layer, a LeakyReLU activation function, a first residual group, and a deep convolutional gated feature modulation unit connected in series. The first residual group contains one residual channel attention block. The RGB branch satisfies the following equation: in, For the first layer of RGB features, It is a deep convolutional gated feature modulation unit. For a residual group containing one residual channel attention block, The LeakyReLU activation function is used. It is a 3×3 standard convolution. For input RGB optical images; The DEM branch is composed of a second 3×3 standard convolutional layer, a LeakyReLU activation function, and a second residual group concatenated in series. The second residual group contains four residual channel attention blocks. The DEM branch satisfies the following equation: in, Features of the first layer DEM For a residual group containing 4 residual channel attention blocks, The input DEM data; The cross-modal fusion unit is used to perform the following operations: The RGB features to be fused are processed separately using 1×1 convolutions. and DEM features Channel alignment is performed; the DEM features are upsampled using bilinear interpolation to make their spatial resolution consistent with the RGB features; bimodal features are concatenated along the channel dimension, and then subjected to channel compression and cross-modal feature fusion via 1×1 and 3×3 convolutions to obtain fused intermediate features. ;Will The input is fed into a deep convolutional gated feature modulation unit, then activated by PReLU, and finally outputs channel attention input features through two independent 1×1 convolutions. and refined guidance features The expression is: in, The PReLU nonlinear activation function is used. Indicates concatenation of channel dimensions; It is a 1×1 standard convolution.

2. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 1, characterized in that, The residual channel attention block is used to perform the following operations: For input features Two consecutive 3×3 standard convolutions are executed sequentially, with a PReLU non-linear activation function embedded between the convolutional layers to obtain local features. ; for local features Perform global average pooling to aggregate global statistics for each channel; Attention weights are generated by dimensionality reduction via 1×1 convolution, PReLU activation, dimensionality increase via 1×1 convolution, and then normalization using the Sigmoid function. Pay attention weights With local features Element-wise multiplication, then combined with the input features Add together to output features ; The residual channel attention block satisfies the following formula: in, This is a global average pooling operator; , These are 1×1 convolutions for channel-wise dimensionality reduction and dimensionality enhancement, respectively. Use the Sigmoid activation function; This represents element-wise Hadamard multiplication.

3. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 2, characterized in that, The deep convolutional gated feature modulation unit is used to perform the following operations: Using two sets of 1×1 convolutions to process the input features The number of channels is determined by Mapped to , to obtain projection features In the channel dimension Equal splitting is the main branch feature With gated branch features The main branch uses 5×5 depthwise convolution to extract local spatial context features. The gated branch generates adaptive gate weights through a 1×1 linear mapping and Sigmoid activation; the gate weights are then multiplied element-wise by the main branch output, and after another 1×1 linear mapping, they are multiplied by the input features. Add together to output features The expression is: in, This represents a 5×5 depthwise convolution operator; This represents a 1×1 convolutional linear mapping.

4. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 3, characterized in that, The first layer of the fusion module is specifically used for: First layer RGB features and first-layer DEM features The input is fed into the cross-modal fusion unit, which outputs the channel attention input features corresponding to the first layer. and refined guidance features ; Will The data is sequentially input into the third residual group and the first depthwise convolutional gated feature modulation unit to generate the second layer DEM features. The third residual group includes four residual channel attention blocks; Will and After element-wise addition, the data is sequentially input into the fourth residual group and the second deep convolutional gated feature modulation unit to generate the second layer of RGB features. The fourth residual group includes one residual channel attention block.

5. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 4, characterized in that, The second layer of the fusion module is specifically used for: The second layer of RGB features Second layer DEM features The input is fed into the cross-modal fusion unit, which outputs the corresponding channel attention input features for the second layer. and refined guidance features ; Will The data is sequentially input into the fifth residual group and the third deep convolutional gated feature modulation unit to generate the third layer DEM features. The fifth residual group includes four residual channel attention blocks; Will and After element-wise addition, the data is sequentially input into the sixth residual group and the fourth deep convolutional gated feature modulation unit to generate the third layer of RGB features. The sixth residual group includes one residual channel attention block.

6. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 5, characterized in that, The third layer of the fusion module is specifically used for: The third layer RGB features and third-layer DEM features The input is fed into the cross-modal fusion unit, which outputs the channel attention input features corresponding to the third layer. and refined guidance features ; Will The data is directly input into the seventh residual group to generate the top-level DEM features. The seventh residual group includes four residual channel attention blocks; Will and After element-wise addition, a 1×1 convolution is introduced for channel dimension calibration to generate the top-level RGB features. .

7. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 6, characterized in that, The multi-scale progressive reconstruction module based on global residual learning specifically performs a four-layer progressive reconstruction from deep to shallow: First layer reconstruction: Reconstructing the top-level RGB features The features are added to the upsampled top-level DEM features and then sequentially input into the sixth residual group, the fourth depthwise convolutional gated feature modulation unit, and the first 1×1 convolution to generate the first reconstructed intermediate features. The sixth residual group contains two residual channel attention blocks, and the expression for the reconstruction of the first layer is: in, For a residual group containing two residual channel attention blocks, Indicates upsampling; Second-level reconstruction: Compared with the upsampled third layer DEM features The sums are then fed sequentially into the seventh residual group and the second 1×1 convolution to generate the second reconstructed intermediate features. The seventh residual group contains four residual channel attention blocks, and the expression for the second-layer reconstruction is: Third-level reconstruction: Compared with the upsampled second-layer DEM features The sums are then sequentially fed into the eighth residual group and the third 1×1 convolution to generate the third reconstructed intermediate features. The eighth residual group contains four residual channel attention blocks, and the expression for the third layer reconstruction is: Fourth layer reconstruction: Features of the first layer of DEM after upsampling The sums are then fed sequentially into the ninth residual group and the fourth 1×1 convolution to generate the fourth reconstructed intermediate features. The ninth residual group contains four residual channel attention blocks, and the expression for the fourth layer reconstruction is: The multi-scale progressive reconstruction module based on global residual learning is also used to incorporate the fourth reconstruction intermediate features. The input is fed into the tail refinement module for feature purification; the purified features are then sequentially passed through the transposed convolution upsampling module and the deep residual reconstruction head to generate a depth residual map; the depth residual map is then compared with the original DEM data. After performing s-fold bicubic interpolation upsampling, the results are summed element-wise to output the high-resolution DEM reconstruction result. The expression is: in, For deep residual reconstruction head; The tail refining module consists of two sets of residual channel attention block stacking units, each set of residual channel attention block stacking units contains 8 residual channel attention blocks; Indicates to The result of bicubic interpolation upsampling by s times.

8. The DEM super-resolution reconstruction system based on RGB optical image guidance according to claim 7, characterized in that, The loss function of the system for: in, The total number of effective pixels within the reconstructed high-resolution DEM. For pixel index, High-resolution DEM reconstruction results The Middle Elevation value per pixel For the corresponding real high-resolution DEM, the first The true elevation value of each pixel.

Images