A lunar DEM super-resolution method based on a residual network

Abstract

The application discloses a kind of lunar DEM super-resolution methods based on residual network, belong to remote sensing technical field.The application includes: obtaining DEM image dataset and carrying out pretreatment to it, then using generative adversarial network to carry out data enhancement processing, then high-low resolution image pair set is obtained by downsampling, and training and test set division is carried out;Image super-resolution reconstruction network model is constructed and training is carried out based on training set, then forward inference is carried out on test image set based on the current trained model, if the effect of output reconstruction image reaches expected goal, then the final reconstruction network model for lunar DEM super-resolution reconstruction is obtained;Otherwise, the current trained network model is used as training object to continue model training.The application uses specific residual learning method for existing low-resolution lunar DEM image, improves image quality and detail;Meanwhile, by introducing generative adversarial network, training data is expanded.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing, and particularly relates to a lunar DEM super-resolution method based on a residual network. BACKGROUND

[0002] High-resolution digital elevation models (DEM) and terrain data of the lunar surface are of great significance for scientific research, space exploration missions, and resource exploration.

[0003] However, existing lunar DEM data is generally low-resolution, limiting accurate acquisition and analysis of lunar surface details and topographic features. In recent years, with the rapid development of deep learning and computer vision technologies, various image processing methods have emerged, such as reconstruction-based methods and learning-based methods. Although these methods have achieved some success in Earth remote sensing image processing, they still face challenges when applied to lunar DEM super-resolution methods. The topographic features, lighting conditions, and data noise of the lunar surface are very different from general images, so specific methods need to be used to adapt to the lunar DEM super-resolution task. SUMMARY

[0004] The application aims to provide a lunar DEM super-resolution method based on a residual network, which can effectively generate high-resolution terrain models from low-resolution lunar DEM data to obtain more detailed and accurate lunar terrain information. Thus, it can be applied to lunar scientific research, space exploration, and resource exploration, providing new possibilities for lunar exploration.

[0005] The technical solution adopted by the application is as follows:

[0006] A lunar DEM super-resolution method based on a residual network, comprising the following steps:

[0007] Step 1: Obtain a digital elevation model (DEM) image dataset of a target lunar region and perform data preprocessing on it;

[0008] Step 2: Perform data augmentation processing on the DEM image dataset after data preprocessing using a generative adversarial network;

[0009] Step 3: Perform downsampling on the DEM image dataset after data augmentation processing to obtain a low-resolution image set, and label the low-resolution image set with the labels corresponding to the DEM image dataset after data augmentation processing in Step 2;

[0010] Then, divide the labeled images into a training image set and a test image set;

[0011] Step 4, constructing an image super-resolution reconstruction network model based on a deep convolutional neural network, the reconstruction network model comprising in sequence: an input layer, a residual network, an activation layer and an output layer;

[0012] wherein the input layer is a convolutional layer with a convolution kernel size larger than a specified size, used as a low-resolution image patch, the convolution kernel size of the input layer is defined as k x k, the stride is s, and the padding is 0, and the output feature map of the input layer is represented as:

[0013]

[0014] wherein I(x, y) represents the pixel value of the output feature map of the input layer at coordinates (x, y), w and h represent the width and height of the input image of the input layer respectively, and p represents the number of pixels of padding;

[0015] The residual network of the reconstruction network model comprises M residual blocks, each residual block comprising a plurality of stacked residual basic blocks, each residual basic block comprising four convolutional activations, and the output feature map of the residual basic block is obtained by adding the input of the residual basic block and the convolutional activation result of the fourth layer of the residual basic block, wherein the value range of M is [4, 6]; and the convolution kernel of each residual block is 3 x 3, and the activation function is f(x) = max(x, 0), wherein x represents the input of the activation function;

[0016] Step 5, training the network parameters of the reconstruction network model based on the training image set, the loss function used in the training is the root mean square error RMSE, and the training is stopped when a predetermined number of training times is reached or the reconstruction error reaches a predetermined accuracy, thereby obtaining a trained reconstruction network model;

[0017] Step 6, based on the current trained reconstruction network model, performing forward inference on the test image set, and saving the reconstructed image (super-resolution image) output by the reconstruction network model;

[0018] determining whether the reconstruction effect of the reconstructed image of the test image reaches the expected target, if yes, obtaining a final reconstruction network model for lunar DEM super-resolution reconstruction based on the current trained reconstruction network model; if no, taking the current trained reconstruction network model as a training object and continuing to perform step 5.

[0019] Further, in step 1, the data preprocessing specifically includes: image enhancement (including random rotation, flipping and scaling operations, etc.), noise removal, image cropping and segmentation, and image normalization processing.

[0020] Further, the image normalization processing refers to scaling the pixel value to the interval [0, 1].

[0021] Further, the application also includes: when obtaining the target affected lunar DEM super-resolution image based on the final reconstruction network model obtained in step 6, the output of the final reconstruction network model is processed by a sharpening filter for terrain edge enhancement, and then the reconstructed lunar DEM super-resolution image is obtained through inverse normalization restoration processing. In this processing process, the sharpening filter is used for denoising to reduce possible reconstruction noise, so that a clearer and more accurate lunar DEM super-resolution image is obtained.

[0022] Further, in step 2, the generator of the generative adversarial network includes several convolution modules, each convolution block includes, in sequence, a convolution operation, a batch normalization operation and an activation function mapping; and the input of the generator is a random noise vector, and the dimension of the generated image output by the generator is consistent with the dimension of the DEM image after data preprocessing, that is, a new high-resolution image sample is generated based on the input random noise vector through the network; the discriminator of the generative adversarial network includes several convolution modules, and a convolution layer and a sigmod function are connected after the last convolution module; the input of the discriminator is the generated image output by the generator and the DEM image after data preprocessing, and the similarity between the two is output by the discriminator, which is used to adjust the network parameters of the generator, so that the generated image output by the generator is as close as possible to the DEM image after data preprocessing.

[0023] Further, in step 5, the loss function used in training is:

[0024]

[0025] Wherein, n represents the number of training images participating in training, N and T are labels and corresponding reconstructed images respectively, subscript i is a training image identifier, λ represents a preset regularization coefficient for controlling the weight of the regularization term, and ω represents the weight vector of the reconstruction network model. Preferably, the order of magnitude of the regularization coefficient λ is 1e-5.

[0026] The technical scheme provided by the application at least brings the following beneficial effects:

[0027] The application uses a specific residual learning method for the existing low-resolution lunar DEM image, improves the image quality and details, and at the same time, introduces the generative adversarial network to expand the training data, so that the application method has a wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical scheme in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 The processing process schematic diagram of the lunar DEM super-resolution method based on the residual network provided by the embodiment of the application is shown in the figure.

[0030] Figure 2 The GAN network structure schematic diagram used in the embodiment of the application is shown in the figure, wherein 2a is a network structure schematic diagram of a generator, and 2b is a network structure schematic diagram of a discriminator.

[0031] Figure 3 The RES network (residual network) network structure schematic diagram used in the embodiment of the application is shown in the figure.

[0032] Figure 4 The reconstruction structure schematic diagram of the embodiment of the application is shown in the figure, wherein 4a is a 5m resolution DEM image of a study area, and 4b is a 2.5m high-resolution lunar DEM image obtained after reconstruction. DETAILED DESCRIPTION

[0033] To make the purpose, technical scheme and advantages of the embodiment of the application clearer, the technical scheme in the embodiment of the application will be described in detail and completely below by combining the figures in the embodiment of the application. Obviously, the described embodiment is only a part of the embodiments of the application, rather than all the embodiments. Generally, the components of the embodiment of the application described and shown in the figures can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiment of the application provided in the figures is not only to limit the scope of the claimed application, but only to represent selected embodiments of the application.

[0034] The embodiment of the application provides a lunar DEM super-resolution method based on a residual network, so as to effectively generate a high-resolution terrain model from low-resolution lunar DEM data, obtain more detailed and accurate lunar terrain information, and thus the method can be applied to the fields of lunar scientific research, space exploration and resource exploration, and the like, thereby providing a new possibility for lunar exploration.

[0035] As a possible implementation manner, the specific implementation steps of the lunar DEM super-resolution method based on the residual network provided by the embodiment of the application include:

[0036] (1) Data preprocessing: Data preprocessing is a key step in super-resolution, which helps to improve the quality and adaptability of data, so as to obtain better results in subsequent training and processing. This step mainly includes the following operations: image enhancement, noise removal, image cropping segmentation, image normalization and standardization.

[0037] (2) Data augmentation: Data augmentation is a technique commonly used in machine learning and deep learning tasks to improve the performance of the model by introducing a generative adversarial network (GAN) to expand the diversity and quantity of the original data. In processing lunar DEM (Digital Elevation Model) images, GAN generates a large number of lunar DEM samples based on the original samples. GAN is an unsupervised generative model, the main architecture consists of a generator and a discriminator, the generator is trained to generate a dataset close to the real sample, and the discriminator is used to evaluate the similarity of the generated data and the real sample. In the training process of the generator, the method of randomness and data transformation is adopted to increase the diversity of the generated sample set. The image augmented by the GAN model not only retains the topographic features of the original image, but also increases more details and clarity, which is crucial for the robustness and generalization performance of the super-resolution model.

[0038] (3) Data set preparation: In the embodiment of the present application, the processed data set is divided into training set images and test set images according to 7:3, and the training set images are down-sampled by n times (based on the actual situation). The width and height of the original high-resolution training set image are w and h, respectively, and the width and height of the low-resolution training set image obtained by down-sampling are w / n and h / n, respectively. Label the low-resolution data set with the label corresponding to the original high-resolution image to obtain labeled training data, and select the low-resolution training data set as unlabeled training data.

[0039] (4) Design network structure: Based on the image super-resolution reconstruction SRCNN based on deep convolutional neural network, ResNet (Residual Network) is used to build the network architecture. Since the goal of image super-resolution SR task is to improve low-resolution images to high-resolution images, the loss function selects RMSE, which measures the difference between the generated high-resolution image and the real high-resolution image. The smaller the value, the closer the generated image is to the real high-resolution image, so that a very deep neural network can be trained. In the training process, the learning rate needs to be adjusted dynamically, the connection weights W and the bias b of each layer are randomly initialized, the optimization algorithm is selected as Adam, and the activation function is selected as ReLU. Through parameter setting, the best performance of the model is ensured to converge.

[0040] (5) Network optimization: At the front end of the residual network, a convolutional layer with a larger kernel (e.g., 5x5 or 7x7) is added to more effectively capture the wide terrain features in the lunar DEM image. Considering the large amount of details and terrain features in the lunar DEM image, the model's expression ability is improved by increasing the number of residual blocks. According to the experimental results, the number of residual blocks is adjusted from 4 to 6. Increasing the depth of the residual network (i.e., the number of residual blocks) can help the network learn more complex features, but it also increases the training difficulty and computational cost. For complex lunar terrain, appropriately increasing the network depth helps capture more detailed terrain features. The enlarged image is activated using 6 residual blocks. The convolution kernel of each residual block is 3x3, and the activation function is f(x) = max(x, 0).

[0041] (6) Using the residual principle, the convolution activation results of the first layer and the convolution activation results of the fourth layer are added, and the added results are passed to the activation function (ReLU). The output of the activation function is passed to the next layer of the network for processing. Similarly, the output of the previous layer is added to the convolution activation results four layers away from the current layer.

[0042] To evaluate the super-resolution image reconstruction effect of the method of the embodiment of the application, the embodiment of the application selects a group of representative lunar pictures as the test data set, which includes typical lunar landform features and image samples at different scales. The PSNR (Peak Signal-to-Noise Ratio) index is used to evaluate the performance of the lunar DEM super-resolution method based on the residual network. PSNR is one of the most commonly used indicators to measure the quality of image reconstruction. It is based on the mean square error (MSE) between the reconstructed image and the original high-resolution image, and is used to measure the similarity between the reconstructed image and the original image.

[0043] Embodiment

[0044] Referring to Figure 1 , the processing data of the embodiment is a high-resolution lunar digital elevation model (DEM) generated by the lunar orbiter laser altimeter (LOLA). This DEM data reaches a level of 5 meters per pixel and covers the geographical area studied. The lunar DEM super-resolution method based on the residual network of the embodiment specifically includes the following steps:

[0045] Step 1: Since the amount of high-resolution DEM data on the lunar surface is limited, the embodiment introduces a generative adversarial network (GAN) model (as shown in Figure 2 ). Figure 22a is the generator used in this embodiment, which is composed of several convolution modules, each convolution block in turn includes: convolution operation, batch normalization operation and activation function mapping, this embodiment adopts three convolution modules to constitute the generator. Wherein, the network input of the generator is a random noise vector, this noise vector generates new data samples through the network, the convolution block in the figure represents the convolution layer, these layers are used to extract features in the image. Each convolution layer will apply multiple filters (or convolution kernels) to scan the input image and produce a set of feature maps (images), which represent certain features in the image, such as edges, corners, etc. The training data set is expanded by generating synthetic high-resolution elevation data from existing DEM data to better study and verify the super-resolution method. In this embodiment, the discriminator used is shown in 2b, which includes a plurality of convolution modules, and a convolution layer and a sigmod function are connected after the last convolution module. In this embodiment, the discriminator includes 3 convolution modules. Figure 2

[0046] This embodiment is realized based on the Matconvnet tool package, and is performed on a computer equipped with an NVIDIA GeForce GTX 3090 GPU and 32 GB RAM. Before inputting the network, all DEM images are normalized, that is, the pixel value is scaled to the interval [0, 1]. In addition, in order to improve the generalization ability of the network, random rotation, flipping and scaling operations are performed on the data set.

[0047] Step 2: Design the network structure, the network main architecture used is the SRCNN model, which includes an input layer, a convolution layer, a residual network, an activation layer and an output layer. The residual network is ResNet-32, which includes six residual blocks, each residual block is composed of three convolution layers, and the convolution kernel size is 3x3 and the step is 1. In order to avoid the problem of gradient disappearance, batch normalization (Batch Normalization) and ReLU activation function are connected after each convolution layer. The output layer uses bilinear interpolation up-sampling to restore to the original high-resolution size.

[0048] The size of the convolution kernel is usually fixed, the input layer is used as a low-resolution image patch, the convolution kernel size is 9x9, the step is 6, and the padding is set to 0, which helps to keep the size of the output feature map unchanged. Assuming that w represents the image width and h represents the height, the size of the first output feature map obtained by input convolution operation can be calculated by the following formula:

[0049]

[0050] ​In the formula, (x, y) represents the pixel coordinates of the output feature map, k is the kernel size (k×k), s is the stride, and p is the number of padding pixels. Indicates the output size in the width direction. This represents the output size in the height direction. The convolution kernel starts from the top left corner of the image and slides across the input image in both horizontal and vertical directions with a stride of 's', performing convolution operations on different parts of the input image. The convolution kernel size of the convolutional layer is 5x5, with the stride and padding remaining constant. Its function is to achieve non-linear multiple mappings of the input layer's output image features.

[0051] In this embodiment, the high-level features of the preliminarily learned images are further input into ResNet-32 (network schematic diagram as shown below). Figure 3 As shown, ResNet employs the residual principle, using both identity mapping and residual mapping. The outputs of these two branches are summed to form the final output of the residual block. An input image of size w×h is processed by a 3×3 convolutional kernel in the first residual block, padded with zeros to maintain the output size. This allows the output after the activation layer to be added to the input. The image propagates sequentially throughout the ResNet network, learning feature representations at different levels. Each residual block maintains the same input and output size, meaning the entire network does not introduce size changes.

[0052] Step 3: Model Training: The optimizer used is Adam, with an initial learning rate set to 1e-4. This learning rate is adjusted during training using the ReduceLROnPlateau callback function. The learning rate is halved if the loss on the validation set fails to improve for five consecutive epochs. The loss function used is the mean squared error (RMSE), described by the formula:

[0053]

[0054] Where N and T are the original resolution DEM image and the reconstructed DEM image, respectively. An L2 regularization term was added to prevent overfitting, with a regularization coefficient of 1e-5.

[0055] L2=λ||ω|| 2

[0056] Where λ is the regularization coefficient, used to control the weight of the regularization term, and ω represents the model's weight vector, which contains the learned feature representations. In the super-resolution process, these features involve information such as texture, edge, and color.

[0057] Step 4: Batch setting: The batch size for each training is set to 16, which is the optimal value determined considering the GPU memory limit and model complexity. The data for each batch is selected from the training set by random sampling. The model training is performed for a total of 100 epochs. After each epoch, the model performance is evaluated using the validation set, and the training is terminated by the EarlyStopping mechanism when the performance no longer improves. The model uses the Adam optimization method with β1 = 0.9, β2 = 0.99, and ε = 1 × 10 -8 During the use of the Adam optimizer, a smaller initial learning rate and more detailed parameters (such as β1 = 0.9, β2 = 0.999) are explored to improve training stability.

[0058] Step 2 is repeated until the neural network output layer error reaches the preset accuracy requirement or the number of training iterations reaches the maximum number of iterations, and the training is ended. The network structure and parameters are saved, i.e., the trained neural network model is obtained, and the entire algorithm flow is as shown in Figure 1 .

[0059] Step 5: Image quality evaluation: In order to comprehensively evaluate the performance of the proposed super-resolution method, the peak signal-to-noise ratio (PSNR) is used as the image quality evaluation index for this experiment. Its formula is:

[0060]

[0061] (MSE is the mean square error between the current original image and the reference image; MAX is the maximum value of the original image color, if each sampling point is 8 bits, then MAX = 255). The higher the PSNR value, the better the image quality. This shows that the model can effectively reconstruct high-quality lunar DEM images. The test results are divided into 10 batches, and the average PSNR value of each batch is calculated, and the test results of the model are as shown in Figure 4 , where 4a is the 5m resolution DEM image of the study area, and 4b is the reconstructed 2.5m high-resolution lunar DEM image.

[0062] Step 6: Input any low-resolution image, use the trained reconstruction network model, and the output of the network is the reconstructed super-resolution image. The high-resolution image output by the network is further enhanced by a sharpening filter to enhance the terrain edge and a denoising technique to reduce possible reconstruction noise, thereby obtaining a clearer and more accurate lunar DEM super-resolution image.

[0063] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some technical features therein can be replaced by equivalents; and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0064] The above only describes some embodiments of the present application. For those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A lunar DEM super-resolution method based on residual networks, characterized in that, Includes the following steps: Step 1: Obtain the digital elevation model (DEM) image dataset of the target area on the moon and perform data preprocessing. Step 2: Generative adversarial networks are used to perform data augmentation on the preprocessed DEM image dataset; Step 3: After data augmentation, the DEM image dataset is downsampled to obtain a low-resolution image set. The low-resolution image set is labeled with the corresponding labels of the DEM image dataset after data augmentation in Step 2. The labeled images are then divided into a training image set and a test image set. Step 4: Construct an image super-resolution reconstruction network model based on a deep convolutional neural network. This reconstruction network model includes, in sequence: an input layer, a residual network, an activation layer, and an output layer. The input layer is a convolutional layer with a kernel size larger than a specified size, used as a low-resolution image patch. The kernel size of the input layer is defined as k×k, the stride as s, and the padding as 0. The output feature map of the input layer is then represented as: Where I(x,y) represents the pixel value of the output feature map of the input layer at coordinates (x,y), w and h represent the width and height of the input image of the input layer, respectively, and p represents the number of pixels to be filled; The residual network of the reconstructed network model includes M residual blocks, each residual block includes several stacked residual base blocks, and each residual base block includes four convolutional activation layers. The output feature map of the residual base block is obtained by adding the input of the residual base block with the convolutional activation result of the fourth layer of the residual base block. The value of M is in the range of [4,6]. The convolutional kernel of each residual block is 3×3, and the activation function is f(x)=max(x,0), where x represents the input of the activation function. Step 5: Train the network parameters of the reconstruction network model based on the training image set. The loss function used during training is the root mean square error (RMSE). The training stops when a predetermined number of training iterations is reached or the reconstruction error reaches a preset accuracy, and the trained reconstruction network model is obtained. Step 6: Based on the currently trained reconstruction network model, perform forward inference on the test image set and save the reconstructed super-resolution image output by the reconstruction network model. Determine whether the reconstruction effect of the super-resolution image of the test image meets the expected goal. If yes, obtain the final reconstruction network model for the super-resolution reconstruction of the lunar DEM based on the currently trained reconstruction network model. If not, use the currently trained reconstruction network model as the training object and continue to execute step 5.

2. The method as described in claim 1, characterized in that, In step 1, data preprocessing specifically includes: image enhancement, noise removal, image cropping and segmentation, and image normalization.

3. The method as described in claim 2, characterized in that, Image enhancement includes random rotation, flipping, and scaling operations.

4. The method as described in claim 2, characterized in that, Image normalization refers to scaling pixel values ​​to the [0,1] range.

5. The method as described in claim 1, characterized in that, Also includes: When acquiring lunar DEM super-resolution images of the target impact based on the final reconstruction network model obtained in step 6, the output of the final reconstruction network model is processed by a sharpening filter to enhance the terrain edges, and then subjected to inverse normalization restoration to obtain the reconstructed lunar DEM super-resolution image.

6. The method as described in claim 1, characterized in that, In step 2, the generator of the generative adversarial network (GAN) includes several convolutional modules. Each convolutional module sequentially includes a convolution operation, a batch normalization operation, and an activation function mapping. The input of the generator is a random noise vector, and the dimension of the generated image output is consistent with the dimension of the DEM image after data preprocessing. The discriminator of the GAN includes several convolutional modules, and the last convolutional module is connected to a convolutional layer and a sigmoid function. The input of the discriminator is the generated image output by the generator and the DEM image after data preprocessing. The discriminator outputs the similarity between the two, which is used to adjust the network parameters of the generator.

7. The method as described in claim 1, characterized in that, In step 3, the labeled images are divided into a training image set and a test image set in a 7:3 ratio.

8. The method as described in claim 1, characterized in that, In step 5, the loss function used during training is: Where n represents the number of training images participating in the training, N and T are the label and the corresponding reconstructed super-resolution image, respectively, the subscript i is the training image identifier, λ represents the preset regularization coefficient used to control the weight of the regularization term, and ω represents the weight vector of the reconstruction network model.

9. The method as described in claim 8, characterized in that, The regularization coefficient λ is on the order of 1e-5.

10. The method as described in claim 1, characterized in that, The kernel size of the input layer of the reconstructed network model is greater than or equal to 5×5.

Images