A DEM super-resolution reconstruction method based on self-similar features and Transformer technology

Abstract

The present invention discloses a digital elevation model (DEM) super-resolution reconstruction algorithm based on self-similar features and Transformer technology. The method can more accurately improve the resolution of digital elevation model data, and includes the following steps: 1) pre-processing high-resolution digital elevation model training data, and using the self-similarity principle of terrain to obtain digital elevation model pre-processed data with similar detail features to reference data; 2) constructing a Transformer model, wherein the Transformer model is used to perform super-resolution reconstruction on the digital elevation model pre-processed data to generate high-resolution digital elevation model data; 3) inputting the digital elevation model pre-processed data into the Transformer model for model training to obtain a trained Transformer model; and 4) pre-processing low-resolution digital elevation model data and inputting the pre-processed data into the trained Transformer model for super-resolution reconstruction of the digital elevation model to obtain high-resolution digital elevation model data.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular relates to a digital elevation model (DEM) super-resolution reconstruction method based on self-similar features and Transformer technology. Background Art

[0002] A digital elevation model (DEM) is a digital representation of the morphological attributes of a terrain surface. It is a digital description with spatial location characteristics and terrain attribute characteristics. DEMs have been widely used in 3D visualization, path planning, terrain surface analysis, and other fields. Due to the diversity of application areas, the generation of DEMs has been studied in many fields using different technologies, including LiDAR. LiDAR is an optical remote sensing technology that measures the distance between the sensor and the object, as well as the energy reflected by the object. LiDAR data has been used as the primary source of high-resolution, high-precision DEMs. Despite the widespread use of LiDAR data, DEMs still have problems and systematic errors. The resolution and accuracy of DEMs have a significant impact on the analysis results in the application field.

[0003] The resolution of a digital elevation model refers to the size of the land covered by a grid cell. For example, if the resolution of a digital elevation model is 3 meters, each grid cell in the digital elevation model stores elevation data for a 3m x 3m area. The resolution and information content of the digital elevation model have a significant impact on the calculation of terrain indices.

[0004] Improving the quality of digital elevation models (DEMs) has attracted the attention of many researchers. Early research focused on generating DEMs from raw data sources, such as airborne lidar or other geographic information systems (GIS). Traditional interpolation methods, such as inverse distance weighted (IDW), bilinear methods, and kriging, have been used to generate high-resolution DEMs with more spatial information. However, due to the lack of high-frequency information, these methods tend to smooth the terrain surface. Subsequently, several studies have attempted to improve the quality of DEMs by combining multi-scale DEM datasets. The learning sample method incorporates high-resolution measurements into the DEM and introduces nonlocal algorithms to generate super-resolution (SR) images. Due to the high-frequency information extracted from the learning samples, this method achieves higher accuracy than classical interpolation methods. However, the accuracy of nonlocal algorithms depends heavily on the learning samples. Subsequently, convolutional neural network (CNN)-based methods have been applied to DEM super-resolution reconstruction as D-SRCNN, where the input and corresponding output of the CNN are low-resolution and high-resolution DEMs, respectively. In addition to CNN, generative adversarial networks (GAN) are also applied to super-resolution. The GAN-based digital elevation model super-resolution method D-SRGAN improves the resolution of a 50-foot digital elevation model to that of a 3-foot digital elevation model.

[0005] Although super-resolution image methods based on generative adversarial networks (GANs) have achieved good results, the hallucinations and artifacts generated by GANs further pose a huge challenge to the image super-resolution reconstruction task. Reference image-based super-resolution reconstruction (RefSR) technology has made some progress. It can transfer high-resolution textures from a given reference image to produce satisfactory results. However, matching a suitable reference DEM data for each low-resolution DEM data is a huge workload, which is difficult to meet the requirements of big data training in neural networks. In view of this, this method, based on the self-similar characteristics of terrain data, proposes a method to automatically obtain reference data using the original data itself, and applies it to the Transformer model to complete high-quality high-resolution DEM data reconstruction. Summary of the Invention

[0006] The purpose of the present invention is to improve the existing digital elevation model super-resolution reconstruction method and enhance the resolution of digital elevation model data.

[0007] To this end, the present invention discloses a method for super-resolution reconstruction of digital elevation models based on self-similar features and Transformer technology. The method can more accurately improve the resolution of digital elevation model data, comprising the following steps:

[0008] 1) Preprocessing the high-resolution digital elevation model training data, using the self-similarity principle of terrain to obtain digital elevation model preprocessing data with similar detailed features to the reference data;

[0009] 2) constructing a Transformer model, wherein the Transformer model is used to perform super-resolution reconstruction on the digital elevation model preprocessing data to generate high-resolution digital elevation model data;

[0010] 3) inputting the digital elevation model preprocessing data into the Transformer model for model training to obtain a trained Transformer model;

[0011] 4) After preprocessing the low-resolution digital elevation model data, the data is input into the trained Transformer model for digital elevation model super-resolution reconstruction to obtain high-resolution digital elevation model data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Flowchart of the digital elevation model super-resolution reconstruction method based on self-similar features and Transformer technology

[0013] Figure 2 A training process framework diagram for the digital elevation model super-resolution reconstruction method based on self-similar features and Transformer technology

[0014] Figure 3 An example diagram of the downsampling method

[0015] Figure 4 An example of the upsampling method

[0016] Figure 5 This is a reconstruction process framework diagram of the digital elevation model super-resolution reconstruction method based on self-similar features and Transformer technology. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings, but the present invention should not be construed as limiting other specific embodiments not described in detail herein.

[0018] See Figure 1 The embodiment of the present application provides a method for super-resolution reconstruction of a digital elevation model based on self-similar features and Transformer technology, which specifically includes the following steps:

[0019] Step 1: Preprocess the high-resolution digital elevation model data and obtain digital elevation model preprocessed data with similar detailed features to the reference data by using the principle of self-similarity of terrain.

[0020] In the DEM data, uncollected non-realistic data has a value of 0 and is referred to as boundary data. By removing non-realistic data with a value of 0 from the high-resolution initial DEM data, the model input is high-quality data sources, resulting in a more accurate model training. After boundary removal, the initial high-resolution DEM data is cropped to obtain the high-resolution DEM-HR data with an n×n resolution.

[0021] The high-resolution digital elevation model data after the boundary removal processing is down-sampled by a times to obtain low-resolution digital elevation model initial data; the low-resolution digital elevation model data is cropped to obtain low-resolution digital elevation model data DEM-LR with a resolution of n / a×n / a; and the digital elevation model reference data DEM-Ref with a resolution of n×n and similar detail features is cropped with the low-resolution digital elevation model data DEM-LR as the center.

[0022] The digital elevation model reference data DEM-Ref is obtained by utilizing the self-similarity principle of terrain. The self-similarity principle can be expressed as follows: if two physical phenomena of the same type have proportional scalar physical quantities at corresponding points in time and space, and all physical quantities have the same direction in addition to proportionality, then the two phenomena are said to be similar. Because there is no ready-made reference terrain dataset, the workload of constructing a dataset by oneself is enormous. Given the self-similarity of terrain, unlike the traditional reference image-based super-resolution reconstruction method that uses a high-resolution image as a reference image, this method utilizes the self-similarity principle of terrain and uses large-scale digital elevation model data centered on low-resolution digital elevation model data as reference data, and applies it to super-resolution reconstruction.

[0023] like Figure 2 As shown, the low-resolution digital elevation model data DEM-LR is upsampled a times to obtain low-resolution digital elevation model upsampled data DEM-LR-U, and the digital elevation model reference data DEM-Ref is downsampled and upsampled a times in sequence to obtain digital elevation model reference downsampled upsampled data DEM-Ref-DU.

[0024] In order to retain the original data information to the greatest extent, the downsampling method is as follows: Figure 3 As shown in the figure, assume that the size of image A is 4×4, and the size of image B after 2 times downsampling is 2×2. The corresponding coordinate formula of image B and image A is:

[0025] B(x,y)=A(x*2,y*2) (1)

[0026] The upsampling method is as follows Figure 4As shown, it is divided into two steps: bicubic interpolation calculation and real coordinate replacement:

[0027] First, perform bicubic interpolation calculation:

[0028]

[0029] Among them, a ij is the weighting coefficient, and the formula of the weighting coefficient is as follows:

[0030]

[0031] Here, a is -0.5.

[0032] Then use the value of the original image to replace the value of the corresponding coordinate after upsampling. The corresponding coordinate formula is:

[0033] A(x*2,y*2)=B(x,y) (4)

[0034] The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model up-sampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference down-sampled up-sampled data DEM-Ref-DU.

[0035] Step 2: constructing a Transformer model, wherein the Transformer model is used to perform super-resolution reconstruction on the input digital elevation model data to generate high-resolution digital elevation model data.

[0036] The Transformer model includes feature extraction and self-attention calculation parts. Figure 2 As shown, the Transformer model obtains the feature F of the low-resolution digital elevation model data DEM-LR, the feature Q of the low-resolution digital elevation model upsampled data DEM-LR-U, the feature V of the digital elevation model reference data DEM-Ref, and the feature K of the digital elevation model reference downsampled upsampled data DEM-Ref-DU through feature extraction.

[0037] The Transformer model performs self-attention calculations including correlation calculation, feature transfer, and feature fusion on features K, Q, and V, and the results are added to feature F to finally output high-resolution digital elevation model data.

[0038] Step 3: Input the preprocessed digital elevation model data into the Transformer model for model training.

[0039] like Figure 2As shown, the input high-resolution digital elevation model initial data is preprocessed to obtain digital elevation model preprocessed data;

[0040] The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU;

[0041] Inputting the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU into the Transformer model for training to obtain synthetic feature data;

[0042] Calculating the structural similarity between the DEM-HR and the synthetic features and performing model evaluation;

[0043] If the evaluation results are satisfactory, the training ends; if not, the Transformer model continues to be trained.

[0044] Step 4: After preprocessing the low-resolution digital elevation model data, the low-resolution digital elevation model data is input into the trained Transformer model for digital elevation model super-resolution reconstruction to obtain high-resolution digital elevation model data.

[0045] like Figure 5 As shown, the input low-resolution digital elevation model data is preprocessed to obtain digital elevation model preprocessed data;

[0046] The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU;

[0047] The low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU are input into the trained Transformer model for super-resolution reconstruction to obtain high-resolution digital elevation model data.

[0048] In short, the embodiments of the present invention disclose preferred embodiments thereof, but are not limited thereto. A person skilled in the art will readily understand the spirit of the present invention based on the above embodiments and make various extensions and modifications. As long as these modifications do not depart from the spirit of the present invention, they are all within the scope of protection of the present invention.

Claims

1. A digital elevation model super-resolution reconstruction method based on self-similar features and Transformer technology, characterized in that: The following steps are involved: 1) Preprocessing the high-resolution digital elevation model training data, using the self-similarity principle of terrain to obtain digital elevation model preprocessing data with similar detailed features to the reference data; 2) constructing a Transformer model, wherein the Transformer model is used to perform super-resolution reconstruction on the digital elevation model preprocessing data to generate high-resolution digital elevation model data; 3) inputting the digital elevation model preprocessing data into the Transformer model for model training to obtain a trained Transformer model; 4) After preprocessing the low-resolution digital elevation model data, the data is input into the trained Transformer model for digital elevation model super-resolution reconstruction to obtain high-resolution digital elevation model data; The step 1) of preprocessing the high-resolution digital elevation model training data to obtain digital elevation model preprocessed data having similar detail features to the reference data by utilizing the self-similarity principle of terrain includes: The high-resolution digital elevation model training data is processed and cropped to obtain the high-resolution digital elevation model data DEM-HR with a resolution of n×n; Performing a-fold downsampling processing on the high-resolution digital elevation model initial data after the boundary removal processing to obtain low-resolution digital elevation model initial data; The low-resolution digital elevation model initial data is clipped to obtain low-resolution digital elevation model data DEM-LR with a resolution of n / a×n / a; and digital elevation model reference data DEM-Ref with a resolution of n×n having similar detail features is clipped with the low-resolution digital elevation model data DEM-LR as the center; The low-resolution digital elevation model data DEM-LR is upsampled by a times to obtain low-resolution digital elevation model upsampled data DEM-LR-U, and the digital elevation model reference data DEM-Ref is downsampled and upsampled by a times in sequence to obtain digital elevation model reference downsampled upsampled data DEM-Ref-DU; The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model up-sampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference down-sampled up-sampled data DEM-Ref-DU.

2. The method for super-resolution reconstruction of a digital elevation model based on self-similar features and Transformer technology according to claim 1, characterized in that: The Transformer model in step 2) obtains the feature F of the low-resolution digital elevation model data DEM-LR, the feature Q of the low-resolution digital elevation model upsampled data DEM-LR-U, the feature V of the digital elevation model reference data DEM-Ref, and the feature K of the digital elevation model reference downsampled upsampled data DEM-Ref-DU through feature extraction; The Transformer model performs self-attention calculation on feature K, feature Q, and feature V, and adds the result to feature F to obtain the final output high-resolution digital elevation model data.

3. The method for super-resolution reconstruction of a digital elevation model based on self-similar features and Transformer technology according to claim 2, characterized in that: The step 3) of inputting the digital elevation model preprocessing data into the Transformer model for model training includes: The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU; Inputting the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU into the Transformer model for training to obtain synthetic feature data; Calculating the structural similarity between the synthetic feature data and the high-resolution digital elevation model data DEM-HR, and performing model evaluation; If the evaluation results are satisfactory, the training ends; if not, the Transformer model continues to be trained.

4. The method for super-resolution reconstruction of a digital elevation model based on self-similar features and Transformer technology according to claim 1, wherein: The low-resolution digital elevation model data is pre-processed in step 4) and then input into the trained Transformer model for digital elevation model super-resolution reconstruction to obtain high-resolution digital elevation model data, including: Preprocessing the input low-resolution digital elevation model data to obtain digital elevation model preprocessing data; The digital elevation model preprocessing data includes: the low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU; The low-resolution digital elevation model data DEM-LR, the low-resolution digital elevation model upsampled data DEM-LR-U, the digital elevation model reference data DEM-Ref, and the digital elevation model reference downsampled upsampled data DEM-Ref-DU are input into the trained Transformer model for super-resolution reconstruction to obtain high-resolution digital elevation model data.

Images