High-resolution under-ice terrain generation method based on filling super-resolution

Abstract

The invention provides an under-ice terrain generation method based on filling and super-division, which combines filling and super-division models of deep learning and utilizes sparse airborne radar data to accurately generate a high-resolution terrain. According to the method, firstly, a terrain trend is extracted at a low resolution (1000m) through a filling model, and then a super-division model is used for super-division of the low-resolution terrain to generate a high-resolution (250m) terrain. The filling model adopts a twin structure, macroscopic features are extracted through codec branches, and kernel prediction branches learn kernel generation trend details of each pixel. The super-division model uses a pyramid structure, combines a kernel prediction structure with pixel recombination and bilinear interpolation fusion, and generates rich terrain details. The method shows high robustness and wide applicability in an under-ice terrain generation task, provides a new thought and method for the field, and has important scientific value and application prospect.

Description

Technical Field

[0001] The present invention relates to subglacial terrain reconstruction technology, and in particular to a method for high-precision subglacial terrain digital elevation model (DEM) based on filling super-resolution, which belongs to the cross-technical field of remote sensing data and deep learning. Background Art

[0002] Subglacial terrain reconstruction is of great significance for glaciological research and climate change prediction. At present, direct measurement of subglacial terrain mainly relies on airborne ice-penetrating radar bathymetry technology, but due to the high cost of data acquisition and the limitation of flight routes, the acquired data is usually sparse. Existing subglacial terrain reconstruction methods mainly include traditional interpolation, inversion methods and deep learning methods. Although traditional interpolation methods can generate subglacial terrain DEM, they have low resolution and problems such as over-smoothing and loss of details. Inversion methods use ice sheet surface measurement data and ice flow velocity and other information to infer the subglacial terrain, but the accuracy and stability are limited by data quality and model assumptions. Deep learning methods can generate higher-resolution subglacial terrain DEM, but there are still a lot of artifacts and dependence on data density.

[0003] In addition, most existing subglacial terrain reconstruction methods reconstruct the overall terrain and lack the detailed description of local terrain features such as . The detailed description of subglacial terrain features is of great significance for understanding glacial dynamics, groundwater circulation, and ice sheet evolution. Therefore, how to use sparse aerial radar bathymetric data to generate high-precision and high-resolution subglacial terrain DEM and achieve detailed description of local terrain features such as subglacial lakes and canyons is an important challenge in glaciology research. Summary of the invention

[0004] Based on the image filling and super-resolution technology in deep learning, this paper provides a method for generating a high-precision subglacial terrain digital elevation model (DEM) using sparse aerial radar bathymetric data. This method uses a twin network structure and integrates key technologies such as kernel prediction and deformable convolution to effectively solve the problems of over-smoothing, loss of details, low accuracy and poor stability in existing Antarctic subglacial terrain reconstruction methods.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] 1. In step 1, construct a data set and select the Shuttle Radar Topography Mission (SRTM) global continental elevation data and the polar digital elevation model data ArcticDEM. The track shape mask is obtained by processing the open source data of the Antarctic ice radar data BEDMAP series. The radar data sequence containing longitude and latitude and corresponding elevation values ​​is mapped to a two-dimensional plane, and the track valid data shape is extracted as a binary mask.

[0007] 2. In step 1, the model training adopts the method of pre-training and fine-tuning: the filling model uses low-resolution global elevation data and masks for pre-training, and uses part of the low-resolution Arctic elevation data and masks for fine-tuning; the super-resolution model uses low-resolution and high-resolution global elevation data for pre-training, and uses part of the low-resolution and high-resolution Arctic elevation data for fine-tuning.

[0008] 3. In step 2, a twin terrain filling model is constructed. The model contains two branches: the encoder-decoder branch and the kernel prediction branch. The encoder-decoder branch uses deformable convolution in the encoder, and the intermediate processing layer includes multiple residual blocks for further processing of the extracted features; the decoder uses jump connections to apply bias to the encoded feature map through deformable convolution, which can increase the geometric deformation robustness of the convolution kernel and expand the receptive field of the convolution kernel. It can effectively capture the large-scale trend characteristics of the subglacial terrain. By applying jump connections to the decoder part, more original detail information can be retained. The kernel prediction branch learns an independent kernel for each pixel of the input, and uses linear combinations of pixels in the neighborhood to generate more natural local terrain details, making up for the information loss caused by data holes, and can achieve better results through multi-level applications at the encoder level and the final output.

[0009] 4. In step 3, a terrain super-resolution model is constructed. The network structure includes an encoder and two 2x upsampling decoders. The encoder part includes multiple convolutional layers for extracting features of the input image; the decoder part includes bilinear interpolation and pixel reorganization upsampling layers and convolutional layers. By fusing the two upsampling methods, the feature map can be super-resolved into a high-resolution image while reducing artifacts. In addition, the encoder layer and the decoder output layer add a kernel prediction structure to reorganize the feature map and super-resolution results pixel by pixel to obtain more realistic terrain details.

[0010] 5. In the process of super-resolution model training, the present invention proposes dynamic terrain loss, which dynamically adjusts the loss weight of each pixel by assigning different importance to the loss value of each pixel, thereby improving the efficiency and effect of the network in the training process. Specifically, the weight is dynamically calculated based on the difference between the generated terrain and the real terrain, and the loss function is shown in formula (1).

[0011]

[0012] where dw ij is a dynamic weight, as shown in formula (2), e is a very small constant (here e = 0.001) to avoid division by zero, x ij is the real terrain input, The 4×4 average pooling is applied to the super-resolution terrain output by the super-resolution model, as shown in formula (3). It is the super-resolution terrain output by the super-resolution model.

[0013]

[0014] For each pooled output and the corresponding input low-resolution true value x ij , we calculate the square of the difference Then, using the dynamic weight dw calculated above ij These squared errors are weighted. Finally, all weighted squared errors are averaged to get the final loss value loss.

[0015] 6. The main content of this invention is to propose a subglacial terrain generation method based on filling super-resolution, including using transfer learning to train and fine-tune the model in the absence of complete subglacial terrain training data. The generated subglacial terrain DEM can provide data support for the details of subglacial terrain features, which is helpful for in-depth understanding of the evolution of polar bedrock and the subglacial hydrological cycle mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Model training and generation flow chart

[0017] Figure 2 This is the proposed terrain filling network structure diagram for track terrain filling

[0018] Figure 3 The proposed network structure for low-resolution terrain super-resolution

[0019] Figure 4 This is the verification result map of the complete Arctic terrain of the present invention.

[0020] Figure 5 The present invention generates a sub-ice terrain result map for the Antarctic sparse radar track area DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings:

[0022] Figure 1 In order to fill the super-resolution model and generate a schematic diagram of the subglacial terrain, the global continental elevation data and the selected Arctic elevation data were clipped into terrain blocks of the same geographical range and different sizes of 250m resolution and 1000m resolution. Due to the same geographical range, the 250m resolution was clipped into a square with a side length of 448 pixels, and the 1000m resolution was clipped into a square with a side length of 112 pixels. The track mask was mapped to a two-dimensional plane at a resolution of 1000m through the Antarctic track sequence and clipped to the same size as the complete terrain block with a resolution of 1000m, that is, a square with a side length of 112 pixels.

[0023] For the filling model, the complete terrain data superimposed with the track mask is used as the model input, and the complete terrain data is used as the ground truth for training. For the super-resolution model, the low-resolution complete terrain data block is input, and the high-resolution complete terrain data block is used as the ground truth for training.

[0024] Figure 2 The track terrain filling network proposed by the present invention is as follows: Firstly, the low-resolution track terrain data blocks mapped to two dimensions are input into the two twin branches of the network.

[0025] The encoder consists of 3 layers of sub-encoders:

[0026] In the first encoder layer, 3 pixels of reflection padding are first added to each side of the input tensor so that the convolution layer can capture more edge information. Then it is input to a 2D convolution layer with 2 input channels, 32 output channels, 7x7 kernel size, 1 stride, and no padding. This is followed by an instance normalization layer and a ReLU (Rectified Linear Unit) activation function. At the same time, a deformable 2D convolution layer with 3 input channels, 32 output channels, 3x3 kernel size, 1 stride, and no padding is connected in parallel and channel-wise.

[0027] The second encoder layer consists of a 2D convolutional layer with 64 input channels, 64 output channels, 4x4 kernel size, stride 2, and padding 1, an instance normalization layer, and a ReLU activation function. At the same time, a deformable 2D convolutional layer with 64 input channels, 64 output channels, 3x3 kernel size, stride 1, and no padding is connected in parallel and channel-wise.

[0028] The output feature map is also input into the twin branch. The kernel prediction branch connects the feature map by pooling the original input. Through convolution, a 3x3 convolution kernel with the same number of feature pixels as the third-layer encoder is generated, and pixel-by-pixel convolution is performed on the output feature map of the third-layer encoder.

[0029] The third encoder layer consists of a 2D convolutional layer with 128 input channels, 256 output channels, a kernel size of 3x3, a stride of 1, and a padding of 1, an instance normalization layer, and a ReLU activation function.

[0030] The middle layer consists of 8 residual blocks, each of which contains two 1-pixel reflection padding layers, a convolution kernel size of 3x3 with a spectral normalization layer, a stride of 1, a dilation rate of 1, and a 2D convolution layer without padding. Each convolution layer is followed by an instance normalization layer and a ReLU activation function.

[0031] The decoder consists of 3 layers of sub-decoders:

[0032] The first decoder layer consists of a transposed convolution with 256 input channels, 128 output channels, a kernel size of 4x4, a stride of 2, and a padding of 1, an instance normalization layer, and a ReLU activation function. It is then connected to the second encoder output feature map by channel, and the channel is reduced by a two-dimensional convolution layer with 256 input channels, 128 output channels, a kernel size of 3x3, a stride of 1, and a padding of 1.

[0033] The second decoder layer consists of a transposed convolution with 128 input channels, 64 output channels, 4x4 kernel size, stride 2, and padding 1, an instance normalization layer and a ReLU activation function, a 3-pixel reflection padding, and a 2D convolution layer with 64 input channels, 64 output channels, 7x7 kernel size, stride 0, and padding 0. It is then connected to the output feature map of the first encoder layer by channel, and the channels are reduced by a 2D convolution layer with 128 input channels, 64 output channels, 3x3 kernel size, stride 1, and padding 1.

[0034] The third-layer decoder consists of a two-dimensional convolution with 64 input channels, 1 output channel, a convolution kernel size of 7x7, a stride of 1, and a padding of 3, and a ReLU activation function. At the same time, the kernel prediction structure generates a 7x7 convolution kernel with the same number of pixels as the decoder output result (112x112) for pixel-by-pixel convolution to obtain a low-resolution padding result.

[0035] Figure 3 The super-resolution network proposed in the present invention is firstly inputting the filled complete low-resolution subglacial terrain data block into the two twin branches of the network.

[0036] The encoder is input into a module consisting of multiple 2D convolutional layers with kernel size 3x3, stride 1, padding 1, instance normalization layer, and ReLU activation function. The number of input terrain channels is enlarged from 1 to 64 and then to 256 for subsequent pixel reorganization, and input into the same residual block of the padding network for feature extraction.

[0037] The decoder consists of 2 layers of sub-decoders:

[0038] The first decoder layer consists of a transposed convolution with 256 input channels, 128 output channels, a kernel size of 4x4, a stride of 2, a padding of 1, an instance normalization layer, and a ReLU activation function.

[0039] The second layer decoder is composed of two parallel upsampling modules connected by channels, which are: 2 2x bilinear interpolation upsampling, input channel number is 128, output channel number is 16, convolution kernel size is 3, step size is 1, padding is 1 two-dimensional convolution layer and 4x pixel reorganization upsampling, input channel number is 32, output channel number is 16, convolution kernel size is 3, step size is 1, padding is 1 two-dimensional convolution layer one instance normalization layer and a ReLU activation function. Finally, there is a 2D convolution layer with input channel number 32, output channel number 1, convolution kernel size 7, and padding is 3 to adjust the number of channels. At the same time, the kernel prediction structure generates a 7x7 convolution kernel with the same number of pixels as the decoder output result (448x448) for pixel-by-pixel convolution to obtain the super-resolution result, which is the final result of the present invention.

[0040] Since there is no large-scale complete subglacial terrain, in order to verify the effectiveness of this method, we first use the complete Arctic data that is not included in the training set to generate it. A mask with a track spacing of 15 km is manually made and superimposed on the complete data. The method of the present invention is used to generate a complete and smooth subglacial terrain, such as Figure 4 shown.

[0041] In addition, in the 45°E area along the coast of the Antarctic continent, there is a lack of radar detection, and the track interval is about 50km. The invention is used to generate the track data before 2000 of the BEDMAP series as a test set, and a small amount of intensive detection data after 2000 in the same area is used as verification, such as Figure 5 The accuracy of the terrain generated by the present invention is calculated using the mean absolute error (MAE) and the mean relative error (MRE). Formulas (4) and (5) are shown in the table below. i is the actual true value, Generates values ​​for the model.

[0042]

[0043] The existing technology faces the challenge of not being able to generate coherent terrain when the track interval exceeds 20 kilometers in terrain generation in this area, and its relative error is close to 30% when facing areas with large terrain fluctuations. The average absolute error of the terrain generated by the present invention with a track interval of about 50km and the actual radar detection data is 222.9m, and the average relative error is 12.7%, which demonstrates the effectiveness of the present invention.

Claims

1. A high-resolution subglacial terrain generation method based on filling super-resolution, Features The following steps are involved: Step 1: Based on the public digital elevation model data, cut it into complete terrain data blocks with consistent range and different resolution as the training set, and then perform batch normalization operation as the training input of the two models; Step 2: Establish a twin filling model using a twin structure, in which the codec structure extracts large-scale trends and the kernel prediction structure generates terrain textures for pre-filling; input the sub-ice elevation data mapped to the two-dimensional 1000m resolution sparse track into the model to generate a rough complete sub-ice elevation data block; Step 3: Establish a super-resolution model, perform terrain super-resolution on the pre-filled rough complete elevation data blocks, perform upsampling through convolution combined with pixel reconstruction and bilinear interpolation, generate detailed terrain textures through the kernel prediction structure, and generate the final high-resolution subglacial terrain.

2. The method according to claim 1, Features In step 1, for the filling model, cropping is performed at a resolution of 1000 m as the complete input of the training set of the filling model, and the sparse input of the training set of the filling model is generated by combining the mask of the track shape detected by the airborne radar.

3. The method according to claim 1, Features In step 1, for the super-resolution model, cropping is performed at a resolution of 1000m as the low-resolution input of the model training set, and cropping is performed at a resolution of 250m at the same location as the high-resolution input of the model training set.

4. The method according to claim 1, Features In step 2, the filling model learns and generates the data from step 1; the input is encoded into a feature map through the deformable convolution in the encoder-decoder branch, and then decoded into a complete terrain. At the same time, the kernel prediction branch generates a convolution kernel for each pixel, interacting at the feature map and output levels, and generating terrain texture by linearly combining neighborhood pixels.

5. The method according to claim 1, Features In step three, the pre-filled low-resolution terrain is first encoded into a high-dimensional feature map through convolution, and then super-resolution is performed through two times of bilinear interpolation and double upsampling combined with pixel reorganization to obtain 4 times upsampling; at the same time, the kernel prediction structure generates a convolution kernel for each pixel of the high-dimensional feature map and the output super-resolution result.

6. The method according to claim 1, Features In step 3, dynamic terrain loss is used during super-resolution model training.

7. The method according to claim 1, Features: The filling model will be mapped to two-dimensional low-resolution track terrain data blocks input into the two twin branches of the network; The encoder consists of 3 layers of sub-encoders: In the first encoder layer, a 3-pixel reflection padding is first added to each side of the input tensor; then it is input into a 2D convolutional layer with 2 input channels, 32 output channels, 7x7 kernel size, stride 1, and no padding; followed by an instance normalization layer and a ReLU activation function; at the same time, a deformable 2D convolutional layer with 3 input channels, 32 output channels, 3x3 kernel size, stride 1, and no padding is connected in parallel and connected by channels; The second encoder layer consists of a 2D convolutional layer with 64 input channels, 64 output channels, 4x4 kernel size, stride 2, and padding 1, an instance normalization layer, and a ReLU activation function; at the same time, a deformable 2D convolutional layer with 64 input channels, 64 output channels, 3x3 kernel size, stride 1, and no padding is connected in parallel with it and connected by channels; The output feature map is also input into the twin branch. The kernel prediction branch connects the feature map by pooling the original input, generates a 3x3 convolution kernel with the same number of pixels as the feature pixels of the third-layer encoder through convolution, and performs pixel-by-pixel convolution on the output feature map of the third-layer encoder. The third encoder layer consists of a 2D convolutional layer with 128 input channels, 256 output channels, a kernel size of 3x3, a stride of 1, and a padding of 1, an instance normalization layer, and a ReLU activation function; The middle layer consists of 8 residual blocks, each of which contains two 1-pixel reflection padding layers, a convolutional kernel size of 3x3 with a spectral normalization layer, a stride of 1, a dilation rate of 1, and a 2D convolutional layer without padding. Each convolutional layer is followed by an instance normalization layer and a ReLU activation function. The decoder consists of 3 layers of sub-decoders: The first decoder layer consists of a transposed convolution with 256 input channels, 128 output channels, a kernel size of 4x4, a stride of 2, and a padding of 1, an instance normalization layer, and a ReLU activation function. It is then connected to the output feature map of the second encoder layer by channel, and the channels are reduced by a two-dimensional convolution layer with 256 input channels, 128 output channels, a kernel size of 3x3, a stride of 1, and a padding of 1. The second decoder layer consists of a transposed convolution with 128 input channels, 64 output channels, 4x4 kernel size, stride 2, and padding of 1, an instance normalization layer and a ReLU activation function, a 3-pixel reflection padding, and a 2D convolution layer with 64 input channels, 64 output channels, 7x7 kernel size, stride 0, and padding of 0. It is then connected to the output feature map of the first encoder layer by channel, and the channels are reduced by a 2D convolution layer with 128 input channels, 64 output channels, 3x3 kernel size, stride 1, and padding of 1. The third-layer decoder consists of a two-dimensional convolution with 64 input channels, 1 output channel, a convolution kernel size of 7x7, a stride of 1, and a padding of 3, and a ReLU activation function. At the same time, the kernel prediction structure generates the same number of pixels as the decoder output result, that is, 112x112 7x7 convolution kernels, and performs pixel-by-pixel convolution to obtain a low-resolution padding result.

8. The method according to claim 1, Features: In the super-resolution model, the filled complete low-resolution subglacial terrain data block is first input into the two twin branches of the network; The encoder is input into a module consisting of multiple 2D convolutional layers with kernel size 3x3, stride 1, padding 1, instance normalization layer, and ReLU activation function. The number of input terrain channels is enlarged from 1 to 64 and then to 256 for subsequent pixel reorganization, and input into the same residual block of the padding network for feature extraction. The decoder consists of 2 layers of sub-decoders: The first decoder layer consists of a transposed convolution with 256 input channels, 128 output channels, a kernel size of 4x4, a stride of 2, a padding of 1, an instance normalization layer, and a ReLU activation function. The second layer decoder is composed of two parallel upsampling modules connected by channels, namely: 2 2x bilinear interpolation upsampling, input channel number 128, output channel number 16, convolution kernel size 3, step size 1, padding 1 two-dimensional convolution layer and 4x pixel reorganization upsampling, input channel number 32, output channel number 16, convolution kernel size 3, step size 1, padding 1 two-dimensional convolution layer an instance normalization layer and a ReLU activation function; finally, a two-dimensional convolution layer with input channel number 32, output channel number 1, convolution kernel size 7, and padding 3 adjusts the number of channels; at the same time, the kernel prediction structure generates the same number of pixels as the decoder output result, that is, 448x448 7x7 convolution kernels, and performs pixel-by-pixel convolution to obtain a super-resolution result, which is the final result of the present invention.

Images