Dem super-resolution reconstruction method and system based on state space model

Abstract

The application relates to a DEM super-resolution reconstruction method and system based on a state space model. First, a shallow feature extraction module of a trained residual network is used to perform feature extraction on input low-resolution DEM data to obtain shallow features. Then, a terrain feature perception analysis module is used to perform terrain feature perception analysis to obtain terrain features. Finally, an enhanced terrain feature reconstruction module is used to perform super-resolution reconstruction on the terrain features to obtain high-resolution DEM data. The terrain feature perception analysis module includes a state space model, which can model terrain trends and be used for DEM super-resolution reconstruction. The model not only focuses on details but also considers the overall trend of the terrain, improves the integrity and continuity of the reconstruction results, and thus improves the overall understanding and representation of the terrain, that is, the accuracy and quality of the DEM super-resolution reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of topographic mapping, in particular to a DEM super-resolution reconstruction method and system based on a state space model. BACKGROUND

[0002] A digital elevation model (DEM) is a model that discretely mathematically represents the topography of the Earth's surface. It simulates the elevation changes of the ground through a series of ordered numerical arrays. DEM not only contains the elevation information of the terrain, but also can derive the contour lines, slope maps and other terrain features. It is an important part of the digital terrain model (DTM). The data structure of DEM usually adopts a regular grid form, and the ground elevation data is stored in a matrix form, thereby providing an efficient digital means for terrain analysis and visualization.

[0003] DEM has a wide range of applications, and it plays an important role in surveying and mapping, engineering construction, landscape planning, hydrological analysis, scientific research, etc. For example, in engineering construction, DEM is used to calculate earthwork, plan roads and dam site selection; in hydrology, DEM is an important tool for watershed analysis and flood simulation; in scientific research, DEM can be used to analyze the influence of terrain on climate, ecology and soil erosion. In addition, DEM can be converted into other types of three-dimensional surface models for environmental science and urban planning, etc. It is one of the indispensable basic data in the field of geographic information science.

[0004] Traditionally, high-resolution DEM data needs to be obtained by high-precision sensors. However, due to technical limitations, it takes a lot of time and cost to obtain high-precision DEM data using traditional mapping methods. There are also some methods that obtain high-resolution DEM data by interpolating on low-resolution DEM data, such as bilinear interpolation, bicubic interpolation, etc. This kind of method is simple and convenient, and does not need prior high-resolution DEM information, but the super-resolution accuracy of this kind of method is not high enough to meet the actual application requirements.

[0005] With the development of deep learning, especially the development of super-resolution in the field of image processing, learning-based DEM super-resolution technology has become a research hotspot in recent years. From the network structure, the existing learning-based DEM super-resolution can be divided into CNN (Convolutional Neural Network) based methods and Transformer (Deep Learning Model Architecture) based methods.

[0006] However, the above-mentioned traditional deep learning-based DEM super-resolution reconstruction method has the technical problems of low reconstruction accuracy and poor application effect. SUMMARY

[0007] Therefore, it is necessary to provide a DEM super-resolution reconstruction method based on a state space model and a DEM super-resolution reconstruction system based on a state space model in view of the above technical problems.

[0008] To achieve the above object, embodiments of the present application adopt the following technical solutions:

[0009] In one aspect, a DEM super-resolution reconstruction method based on a state space model is provided, comprising:

[0010] The shallow feature extraction module of the trained residual network is used to extract features of the input low-resolution DEM data, so as to obtain shallow features of the low-resolution DEM data; the residual network comprises the shallow feature extraction module, a terrain feature perception analysis module and an enhanced terrain feature reconstruction module;

[0011] The terrain feature perception analysis module performs terrain feature perception analysis according to the shallow features, so as to obtain terrain features of the low-resolution DEM data; the terrain feature perception analysis module comprises a plurality of terrain feature perception modules, the terrain feature perception modules are combined in a residual connection manner, each terrain feature perception module comprises a plurality of terrain feature perception sub-modules, and the terrain feature perception sub-modules comprise visual state space models;

[0012] The enhanced terrain feature reconstruction module performs super-resolution reconstruction on the terrain features, so as to obtain high-resolution DEM data; the enhanced terrain feature reconstruction module comprises a convolution layer, a pixel recombination layer and a 2-dimensional selective scanning layer.

[0013] In another aspect, a DEM super-resolution reconstruction system based on a state space model is also provided, comprising a trained residual network;

[0014] The shallow feature extraction module of the residual network is used to extract features of the input low-resolution DEM data, so as to obtain shallow features of the low-resolution DEM data; the residual network comprises the shallow feature extraction module, a terrain feature perception analysis module and an enhanced terrain feature reconstruction module;

[0015] The terrain feature perception analysis module is used to perform terrain feature perception analysis according to the shallow features, so as to obtain terrain features of the low-resolution DEM data; the terrain feature perception analysis module comprises a plurality of terrain feature perception modules, the terrain feature perception modules are combined in a residual connection manner, each terrain feature perception module comprises a plurality of terrain feature perception sub-modules, and the terrain feature perception sub-modules comprise visual state space models;

[0016] The enhanced terrain feature reconstruction module performs super-resolution reconstruction on the terrain features, so as to obtain high-resolution DEM data; the enhanced terrain feature reconstruction module comprises a convolution layer, a pixel recombination layer and a 2-dimensional selective scanning layer.

[0017] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0018] The DEM super-resolution reconstruction method and system based on the state space model described above use the state space model to model the terrain trend and for DEM super-resolution reconstruction, not only paying attention to details and considering the overall trend of the terrain, but also improving the integrity and continuity of the reconstruction results, thereby improving the overall understanding and characterization of the terrain, i.e., improving the accuracy and quality of the DEM super-resolution reconstruction, and in the application field, more comprehensive and accurate terrain information can be obtained, providing a scientific basis for the application fields of terrain analysis, geomorphology evaluation, water resource management, etc., and improving the application effect and reliability. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0020] Figure 1 A flowchart of the DEM super-resolution reconstruction method based on the state space model in an embodiment;

[0021] Figure 2 A residual network diagram in an embodiment;

[0022] Figure 3 A terrain feature perception sub-module diagram in an embodiment;

[0023] Figure 4 A visual state space model diagram in an embodiment;

[0024] Figure 5 A design concept diagram of 2D-SSM in DEM super-resolution in an embodiment;

[0025] Figure 6 A principle diagram of 2D-SSM in an embodiment;

[0026] Figure 7 A deformable convolution diagram in an embodiment;

[0027] Figure 8 Input low-resolution DEM data in an embodiment, wherein (a), (b) and (c) respectively represent three different sets of low-resolution DEM data;

[0028] Figure 9For actual high-resolution DEM data in one embodiment, the actual high-resolution DEM data of (a), (b) and (c) correspond to Figure 8 low-resolution DEM data of (a), (b) and (c);

[0029] Figure 10 For reconstructed high-resolution DEM data in one embodiment, the reconstructed high-resolution DEM data of (a), (b) and (c) correspond to Figure 8 low-resolution DEM data of (a), (b) and (c). DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0032] It should be noted that the reference herein to "embodiments" means that the particular features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the application. The phrase is exhibited at various places in the specification does not necessarily mean the same embodiment, nor is it independent or alternative embodiments that are not mutually exclusive with other embodiments.

[0033] The embodiments of the present application will be described in detail below in combination with the drawings in the embodiment diagram of the present application.

[0034] In one embodiment, as shown in Figure 1 The embodiment of the present application provides a DEM super-resolution reconstruction method based on a state space model, which includes the following processing steps S12-S16:

[0035] S12, using the shallow feature extraction module of the trained residual network to perform feature extraction on the input low-resolution DEM data to obtain the shallow features of the low-resolution DEM data; the residual network includes a shallow feature extraction module, a terrain feature perception analysis module and an enhanced terrain feature reconstruction module.

[0036] It is understandable that the residual network can be trained using L1 loss, or other training methods (such as L2 loss or perceptual loss) to obtain a trained residual network. The residual network consists of three parts: a shallow feature extraction module, a terrain feature perceptual analysis module, and an enhanced terrain feature reconstruction module.

[0037] First, in the shallow feature extraction module, low-resolution DEM data is input, and a 3x3 convolutional layer can be used to extract DEM data features to obtain shallow features.

[0038] S14, the terrain feature perception and analysis module performs terrain feature perception and analysis based on shallow features to obtain the terrain features of low-resolution DEM data; the terrain feature perception and analysis module includes multiple terrain feature perception modules, which are combined by residual connection. Each terrain feature perception module includes multiple terrain feature perception sub-modules, and the terrain feature perception sub-modules include a visual state space model.

[0039] It is understandable that the terrain feature perception and analysis module performs terrain feature perception and analysis to obtain the corresponding terrain features. For example, Figure 2 As shown in the residual network diagram, the terrain feature perception and analysis module includes multiple terrain feature perception modules (TFAGs). Each terrain feature perception module (TFAG) includes multiple terrain feature perception sub-modules (TFAMs) and a convolutional layer. The terrain feature perception modules (TFAGs) are combined using residual connections. The terrain feature perception sub-modules (TFAMs) also include a visual state space model, used to model the trend features of the terrain.

[0040] Note: Figure 2 The diagram contains 6 TFAGs and 6 TFAMs; the number can be changed in actual applications.

[0041] S16, the enhanced terrain feature reconstruction module performs super-resolution reconstruction of terrain features to obtain high-resolution DEM data; the enhanced terrain feature reconstruction module includes convolutional layers, pixel recombination layers and 2D selective scanning layers.

[0042] It is understandable that the enhanced terrain feature reconstruction module performs super-resolution reconstruction using terrain features. This module consists of convolutional layers, pixel recombination layers, and 2D selective scanning layers. First, two convolutional layers integrate the previously learned terrain features, and then pixel recombination operations are used to upsample the data dimension from the low-resolution DEM data size to the high-resolution DEM data size. Next, a 2D selective scanning module uses terrain trends to infer the upsampled terrain features, and then a convolutional layer reduces the channel dimension to 1 to obtain the final high-resolution DEM data.

[0043] The DEM super-resolution reconstruction method based on the state space model uses the state space model to model the terrain trend and reconstruct the DEM super-resolution, which not only pays attention to the details but also considers the overall trend of the terrain, improves the integrity and continuity of the reconstruction results, and thus improves the overall understanding and characterization of the terrain, i.e. improves the accuracy and quality of the DEM super-resolution reconstruction, and in the application field, more comprehensive and accurate terrain information can be obtained, which provides a scientific basis for terrain analysis, landform evaluation, water resource management and other application fields, and improves the application effect and reliability.

[0044] In one embodiment, in the DEM super-resolution reconstruction method based on the state space model, the residual network is trained using L1 loss.

[0045] It can be understood that L1 loss measures the absolute difference between the model prediction value and the true value, and by minimizing L1 loss, the model is trained to produce reconstruction results that match the true terrain features. This training method is evaluated by comparing with the real high-resolution DEM data, including evaluating the accuracy, detail retention and consistency with terrain features of the reconstruction results.

[0046] In one embodiment, in the DEM super-resolution reconstruction method based on the state space model, the terrain feature perception sub-module includes a global trend modeling unit and a local spatial heterogeneity modeling unit, the global trend modeling unit includes a layer normalization operation and a visual state space model, and the local spatial heterogeneity modeling unit includes a layer normalization operation, a deformable convolution layer, an activation layer and a channel attention mechanism.

[0047] It can be understood that, as shown in Figure 3 As shown in (schematic diagram of terrain feature perception sub-module), the terrain feature perception sub-module (TFAM) includes two parts, the first part is a global trend modeling unit, which can include a layer normalization operation (LayerNorm) and a visual state space model (VSSM); the second part is a local spatial heterogeneity modeling unit, which can include a layer normalization operation, a deformable convolution layer (DCN), an activation layer (GeLu) and a channel attention mechanism (CAM). The global trend modeling unit and the local spatial heterogeneity modeling unit are connected through a residual manner.

[0048] In one embodiment, in the DEM super-resolution reconstruction method based on the state space model, the visual state space model includes a linear neural network layer, a depth separable convolution operation, an activation layer and a 2-dimensional selective scanning operation.

[0049] It can be understood that, as shown in Figure 4(Diagram of the visual state space model) The visual state space model includes a linear neural network layer, a depthwise separable convolution operation (DWConv), an activation layer (SiLU), and a 2D selective scan operation (2D-SSM) to model the trend features of the terrain.

[0050] In addition, the design concept of 2D-SSM in DEM super-resolution is as follows: Figure 5 As shown, the design aims to simulate terrain trends by scanning the terrain in four directions (up, down, horizontal, and vertical) to model terrain trends and aggregate terrain features.

[0051] The specific principle of 2D-SSM is as follows: Figure 6 As shown, the DEM or its feature map is first unfolded into a 1D sequence in four directions: top, bottom, horizontal, and vertical, denoted as X = [x1, x2, ..., x]. N There are four such sequences. Then, the terrain features are calculated and aggregated using the following formula to obtain Y = [y1, y2, ..., y3]. N There are four such sequences.

[0052] Finally, these four sequences are added together and restored to a 2D DEM image.

[0053]

[0054]

[0055] y t =Ch t-1 +Dx t

[0056] In the formula, the input x t Let h be the t-th point of the input state-space model. t-1 This refers to the states of the first t-1 points calculated by the state-space model. Based on the current input x... + and the state h obtained from all the previous inputs. t-1 Using parameters and Update the state to get h t At the same time, using h t-1 and x t The input y at point t is calculated using parameters C and D. t y t This can be understood as the result of a state-space model performing feature aggregation on the first t points of a sequence.

[0057] In the formula, △, B, and C are obtained by operating on the input sequence X through a linear neural network, while A and D are obtained by transforming the sequence X, thus enabling adaptive operation on the input sequence X.

[0058] A schematic diagram of deformable convolutional layers is shown below. Figure 7 As shown, it aims to model the local spatial heterogeneity of terrain. Unlike traditional convolution, which extracts features from only a regular rectangular region, deformable convolution can learn local spatial heterogeneity, fit the local terrain, and thus better aggregate terrain features.

[0059] The aforementioned DEM super-resolution reconstruction method based on the state-space model overcomes the shortcomings of previous DEM super-resolution reconstruction methods in that they could not model terrain trends, and improves the performance of DEM super-resolution reconstruction by training a neural network that combines the state-space model and deformable convolution.

[0060] In one embodiment, the above-mentioned DEM super-resolution reconstruction method based on the state space model has 6 terrain feature perception modules and 6 terrain feature perception sub-modules.

[0061] It is understandable that by selecting an appropriate number of terrain feature perception modules and sub-modules, a good reconstruction effect can be achieved while saving computing resources and storage space, thus achieving a balance in practice.

[0062] In some implementations, to more intuitively and comprehensively illustrate the above-described state-space model-based DEM super-resolution reconstruction method, application examples of this method are provided: Figure 8 This is the input low-resolution DEM data in one embodiment, where (a), (b), and (c) represent three different sets of low-resolution DEM data, respectively. Figure 9 For one embodiment, the actual high-resolution DEM data are shown, where the actual high-resolution DEM data in (a), (b), and (c) respectively correspond to... Figure 8 Low-resolution DEM data in (a), (b) and (c) Figure 10 The high-resolution DEM data reconstructed in one embodiment are shown, wherein the reconstructed high-resolution DEM data in (a), (b), and (c) correspond to respectively Figure 8 Low-resolution DEM data for (a), (b) and (c).

[0063] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least one of the steps in the above method can include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but can be performed at different times, and the order of execution of these sub-steps or stages is not necessarily sequential, but can be performed alternately or alternately with at least one part of other steps or sub-steps or stages of other steps.

[0064] In one embodiment, a state space model-based DEM super-resolution reconstruction system is provided, comprising a trained residual network; a shallow feature extraction module of the residual network, configured to perform feature extraction on input low-resolution DEM data to obtain shallow features of the low-resolution DEM data; the residual network comprises the shallow feature extraction module, a terrain feature perception analysis module, and an enhanced terrain feature reconstruction module; the terrain feature perception analysis module is configured to perform terrain feature perception analysis based on the shallow features to obtain terrain features of the low-resolution DEM data; the terrain feature perception analysis module comprises a plurality of terrain feature perception modules combined in a residual connection manner, each terrain feature perception module comprises a plurality of terrain feature perception sub-modules, and the terrain feature perception sub-module comprises a visual state space model; the enhanced terrain feature reconstruction module performs super-resolution reconstruction on the terrain features to obtain high-resolution DEM data; and the enhanced terrain feature reconstruction module comprises a convolution layer, a pixel reorganization layer, and a 2D selective scanning layer.

[0065] The above state space model-based DEM super-resolution reconstruction system uses a state space model to model terrain trends and reconstruct DEM super-resolution, not only focusing on details but also considering the overall trend of the terrain, improving the overall and coherence of the reconstruction results, thereby improving the overall understanding and representation of the terrain, i.e., improving the accuracy and quality of DEM super-resolution reconstruction, and in the application field, more comprehensive and accurate terrain information can be obtained, providing a scientific basis for terrain analysis, geomorphology evaluation, water resource management, etc., and improving application effect and reliability.

[0066] In one embodiment, in the above state space model-based DEM super-resolution reconstruction system, the residual network is trained using L1 loss.

[0067] In one embodiment, in the above state space model-based DEM super-resolution reconstruction system, the terrain feature perception sub-module comprises a global trend modeling unit and a local spatial heterogeneity modeling unit, the global trend modeling unit comprises a layer normalization operation and a visual state space model, and the local spatial heterogeneity modeling unit comprises a layer normalization operation, a deformable convolution layer, an activation layer, and a channel attention mechanism.

[0068] In one embodiment, in the DEM super-resolution reconstruction system based on the state space model, the visual state space model comprises a linear neural network layer, a depth separable convolution operation, an activation layer and a 2-dimensional selective scanning operation.

[0069] In one embodiment, in the DEM super-resolution reconstruction system based on the state space model, the number of terrain feature perception modules is 6, and the number of terrain feature perception sub-modules is 6.

[0070] The specific limitations of the DEM super-resolution reconstruction system based on the state space model can refer to the limitations of the DEM super-resolution reconstruction method based on the state space model described above, and will not be repeated here.

[0071] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0072] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A DEM super-resolution reconstruction method based on a state-space model, characterized in that, Including the following steps: The shallow feature extraction module of the trained residual network is used to extract features from the input low-resolution DEM data to obtain the shallow features of the low-resolution DEM data; the residual network includes the shallow feature extraction module, the terrain feature perception and analysis module, and the enhanced terrain feature reconstruction module; The terrain feature perception and analysis module performs terrain feature perception and analysis based on the shallow features to obtain the terrain features of the low-resolution DEM data. The terrain feature perception and analysis module includes multiple terrain feature perception modules, which are combined using a residual connection method. Each terrain feature perception module includes multiple terrain feature perception sub-modules and a convolutional layer. The terrain feature perception sub-modules include a global trend modeling unit and a local spatial heterogeneity modeling unit. The global trend modeling unit includes layer normalization operations and a visual state space model. The local spatial heterogeneity modeling unit includes layer normalization operations, deformable convolutional layers, activation layers, and channel attention mechanisms. The enhanced terrain feature reconstruction module performs super-resolution reconstruction of the terrain features to obtain high-resolution DEM data; the enhanced terrain feature reconstruction module includes a convolutional layer, a pixel recombination layer and a 2D selective scanning layer.

2. The DEM super-resolution reconstruction method based on a state-space model according to claim 1, characterized in that, The residual network is trained using L1 loss.

3. The DEM super-resolution reconstruction method based on a state-space model according to claim 1, characterized in that, The visual state space model includes a linear neural network layer, a depthwise separable convolution operation, an activation layer, and a 2D selective scanning operation.

4. The DEM super-resolution reconstruction method based on a state-space model according to claim 1, characterized in that, The terrain feature perception module consists of 6 modules, and the terrain feature perception sub-module consists of 6 sub-modules.

5. A DEM super-resolution reconstruction system based on a state-space model, characterized in that, Including the trained residual network; The shallow feature extraction module of the residual network is used to extract features from the input low-resolution DEM data to obtain shallow features of the low-resolution DEM data; the residual network includes the shallow feature extraction module, the terrain feature perception and analysis module, and the enhanced terrain feature reconstruction module. The terrain feature perception and analysis module is used to perform terrain feature perception and analysis based on the shallow features to obtain the terrain features of the low-resolution DEM data. The terrain feature perception and analysis module includes multiple terrain feature perception modules, which are combined using a residual connection method. Each terrain feature perception module includes multiple terrain feature perception sub-modules and a convolutional layer. The terrain feature perception sub-modules include a global trend modeling unit and a local spatial heterogeneity modeling unit. The global trend modeling unit includes layer normalization operations and a visual state space model. The local spatial heterogeneity modeling unit includes layer normalization operations, deformable convolutional layers, activation layers, and channel attention mechanisms. The enhanced terrain feature reconstruction module performs super-resolution reconstruction of the terrain features to obtain high-resolution DEM data; the enhanced terrain feature reconstruction module includes a convolutional layer, a pixel recombination layer and a 2D selective scanning layer.

6. The DEM super-resolution reconstruction system based on a state-space model according to claim 5, characterized in that, The residual network is trained using L1 loss.

7. The DEM super-resolution reconstruction system based on a state-space model according to claim 5, characterized in that, The visual state space model includes a linear neural network layer, a depthwise separable convolution operation, an activation layer, and a 2D selective scanning operation.

8. The DEM super-resolution reconstruction system based on a state-space model according to claim 5, characterized in that, The terrain feature perception module consists of 6 modules, and the terrain feature perception sub-module consists of 6 sub-modules.

Images