Dtm-guided diffusion model dem hole filling method, device and medium

Abstract

The application provides a DEM hole filling method of a terrain line guided diffusion model, including: acquiring a DEM image, and constructing a training set of the DEM image; extracting ridge lines and valley lines in the DEM image through a hydrological analysis method, and constructing a terrain line mask; under the condition of the terrain line mask, training a diffusion model through the training set; and filling holes in an unknown region through iterative denoising by using the trained diffusion model and a terrain line mask of a known region.

Description

Technical Field

[0001] This application relates to the field of DEM image processing, and in particular to a method, apparatus and medium for filling holes in a DEM using a topographically guided diffusion model. Background Technology

[0002] Digital elevation models (DEMs) are digital terrain models. DEM data can describe the undulating features of the ground and is therefore widely used in geographical research, such as disaster assessment and reconstruction, and monitoring dynamic changes in climate-sensitive areas like lakes and glaciers. However, in rugged mountainous areas, complex terrain interferes with the measurement signals emitted by sensors, especially those carried on spacecraft. This prevents the measurement signals from completely covering the ground, and current measurement techniques also have limitations, often resulting in data voids in DEM data. Numerous data voids exist in areas with significant changes in elevation and slope, typically concentrated around ridgelines and valleys. These data voids severely impact the application of DEM data in scientific research. Therefore, accurately filling DEM data voids to meet the high requirements of geoscientific research is of paramount importance.

[0003] Interpolation algorithms, such as IDW and Kriging, are commonly used methods for filling holes in DEMs, and can achieve good results under certain conditions. However, spatial interpolation is easily affected by the complex terrain around the holes, failing to reconstruct complex terrain features, and the results obtained by interpolation methods have the problem of discontinuity in the terrain surface at the hole boundaries. Spatial interpolation algorithms that integrate terrain feature information improve the accuracy of interpolation results to some extent, but they only utilize local elevation information around the holes, rather than the overall contextual semantic information of the image, so they are still not suitable for complex terrains. In recent years, with the continuous breakthroughs of deep learning in the field of image inpainting, some studies have attempted to use deep generative models to overcome the limitations of traditional interpolation methods. Previous studies have demonstrated that Generative Adversarial Networks (GANs) can achieve excellent performance in DEM hole filling tasks. Some studies have integrated terrain information into the network based on Conditional Generative Adversarial Networks (CGANs), generating DEM surfaces with rich semantics while ensuring that the surfaces meet certain conditions. These conditional constraint strategies have improved the performance of GAN networks in DEM hole filling tasks to some extent. However, these methods suffer from unstable training and are prone to mode collapse, requiring significant effort and time to adjust network training. Moreover, the DEM images reconstructed by conventional methods exhibit significant unnatural artifacts, and even in highly complex terrain, GANs still cannot reconstruct complete ridge and valley features. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, apparatus, and medium for filling voids in a topographically guided diffusion model of a DEM.

[0005] The above-mentioned objective of this application is achieved through the following technical solution:

[0006] S1: Obtain DEM images and construct a training set of the DEM images;

[0007] S2: Extract ridgelines and valleylines from the DEM image using hydrological analysis to construct a topographic mask;

[0008] S3: Under the conditions of the terrain line mask, train the diffusion model using the training set;

[0009] S4: Using the trained diffusion model and the terrain line mask of the known area, the holes in the unknown area are filled through iterative denoising.

[0010] Optionally, step S1 includes:

[0011] Using QGIS's function of exporting images and cropping them into blocks of a specified size, the DEM image is constructed into image blocks of 256×256. The size of the image block is limited by the GPU's computing power. When the computing power allows, the larger the size of the image block, the better the performance.

[0012] Optionally, step S2 includes:

[0013] S21: Use ArcGIS spatial analysis tools and hydrological analysis methods to extract the ridgeline grid ridge_1, and calculate the positive terrain DEM_pos and negative terrain DEM_neg of the DEM image;

[0014] S22: Refine the raster ridge_1 and multiply it with the positive terrain DEM_pos to obtain the final ridge line;

[0015] S23: Using the ABS function, the inverse terrain is calculated, the ridgeline is extracted from the inverse terrain, and the valley line is obtained;

[0016] S24: Obtain the terrain mask tf_mask through the ridge line and valley line, with the ridge line pixel value being 1, the valley line pixel value being -1, and the background pixel value being 0.

[0017] Optionally, step S3 includes:

[0018] S31: Divide the terrain line mask into DEM blocks corresponding to the DEM image to form the terrain line part of the training set;

[0019] S32: Initialize the hyperparameters of the diffusion model; set the total number of forward noise addition steps T of the diffusion model; assume the relationship between noise and time t is a cosine function;

[0020] S33: The diffusion model randomly loads DEM blocks and corresponding terrain line masks (tf_mask) from the training set; randomly obtains time t and random Gaussian noise ∈ t ~N(0,I);

[0021] The diffusion model defines the forward diffusion process of a DEM image from time zero to time T by gradually adding random Gaussian noise:

[0022]

[0023] Define α t =1-β t Then the following relationship exists:

[0024]

[0025] Using the two formulas above, we can derive x. t The relationship between x0 and x0 is as follows:

[0026]

[0027] and:

[0028]

[0029] Where x0 represents sample data selected from the training set; x t The image represents the DEM noise image at time t; the forward diffusion process of the diffusion model applies a reparameterization technique to ensure that the image x after adding noise is properly positioned. t For noise ∈ t It is differentiable, which facilitates gradient calculation and thus facilitates neural network training; the diffusion model defines T fixed variance tables, and the variance at time t is β. t ; I represents the identity matrix; x t The relationship between x and x0 indicates that the diffusion model does not require t noise additions to go from x0 to x t , in β t Given that α t It is known that Given the information, noise can be added in one step; q(x) t |x t-1 ) indicates from x t-1 The obtained x t The distribution it follows; : = represents the distribution defined as follows; N(b; C, D) represents the Gaussian distribution that random variable B follows, with mean C and variance D; N represents the Gaussian distribution N; α i =1-βi i = 1, 2, ..., t;

[0030] Based on the diffusion process, the noise image x at time t is obtained. t , to noisy image x t Tensor concatenation is performed with the terrain line mask tf_mask, and the resulting tensor is used as the input to the diffusion model;

[0031] S34: Train a U-Net neural network to predict the inverse denoising process of the diffusion model from x0 to x. t The added noise; determine the predicted noise ∈ θ With actual added noise ∈ t The error between them, and the variational lower bound L between the noise distribution and the true noise distribution. vlb ; Variational lower bound L vlb As the loss function of the diffusion model;

[0032]

[0033] L hybrid =L simple +L vlb

[0034] Among them, c rf For terrain line masking; L simple This represents the simplified MSE loss function; L represents the expectation operation; hybrid This represents the final comprehensive loss function used. This represents the noise predicted by the neural network under parameter θ;

[0035] S35: Randomly select time t within the range [0, 1000], where t is an integer, and repeat step S34 to train the learning ability of the diffusion model under the condition of terrain line mask; the learning ability represents the learning of the mapping relationship between terrain lines and DEM images.

[0036] Optionally, step S4 includes:

[0037] S41: Visually draw the ridgeline and valley line corresponding to the known area from satellite imagery or other known DEM data at the same location, obtain the topographic mask of the known area, and input the hole DEM and the topographic mask of the known area into the trained diffusion model.

[0038] S42: The strategy for inverse denoising and image generation using the diffusion model is as follows:

[0039]

[0040] in, This represents the derived x under the noise predicted by the neural network with parameter θ. t-1 Based on the noise addition principle of the diffusion model described above, the distribution-related parameters are derived. The x derived from the model t-1 μ θ Σ represents the mean derived from the model; N represents the Gaussian distribution; Σ θ This represents the variance derived from the model.

[0041] Obtain the noise in the diffusion model prediction and the variance Σ in the Gaussian transform. θ (x t ,t,c tf );

[0042] Based on the above strategy, noise is gradually denoised in reverse from time T, where t = 0, 1, ..., T;

[0043] S43: In the task of filling holes in DEM images, the part inside the hole is generated by a diffusion model, while the known DEM image data outside the hole is kept unchanged.

[0044] Let a complete DEM image be x, the known region be m⊙x, the hole region be (1-m)⊙x, the hole mask be m, and the known region after forward noise addition be m. as follows:

[0045]

[0046] Where x0 represents the training sample selected from the training set;

[0047] Hollow regions inferred from the diffusion model as follows:

[0048]

[0049] The known regions and predicted hole regions are merged to obtain the x-image. t-1 as follows:

[0050]

[0051] Merge image x t-1 Ten steps of adding noise are performed to obtain a noisy image with more noise at time T; the noisy image is then denoised in reverse for 10 steps. During this process, the known information outside the hole will constrain the diffusion model, making the semantics on both sides of the hole boundary consistent.

[0052] S44: Repeat step S42 a preset number of times, and then denoise for 10 more steps until the image at the next time step is obtained;

[0053] S45. Repeat steps S43 and S44 until t=0 to obtain a noise-free image, thus completing the hole filling of the unknown area of ​​the DEM image.

[0054] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform a DEM void filling method based on a topographically guided diffusion model.

[0055] A computer-readable storage medium storing instructions that, when executed, perform a DEM void filling method for a topographically guided diffusion model.

[0056] The beneficial effects of the technical solution provided in this application are:

[0057] Under the condition of topographic masking, a diffusion model is trained using the training set. The diffusion model can more completely learn the distribution of the training set, and under the control of the external information of the hole, it can smooth the DEM semantics on both sides of the boundary, overcoming the shortcomings of interpolation methods and GAN methods in terms of elevation jumps at the boundary. Moreover, the model does not generate abnormal artifacts and has a powerful DEM image generation capability. The topographic mask and the DEM image information of the known area around the hole provide sufficient control information for filling the hole, which greatly reduces the model's sensitivity to resolution. The diffusion model has great application potential in DEM hole filling tasks. Attached Figure Description

[0058] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0059] Figure 1 This is a step diagram of the DEM cavity filling method of the topographically guided diffusion model in the embodiments of this application;

[0060] Figure 2 This is a DEM block image (a) and a corresponding topographic mask image (b) in the training set of the DEM hole filling method of the topographic line-guided diffusion model in the embodiments of this application.

[0061] Figure 3 This is a schematic diagram illustrating the working principle of the diffusion model of the DEM cavity filling method guided by terrain lines in the embodiments of this application.

[0062] Figure 4 This is a schematic diagram of the iterative denoising strategy of the DEM hole filling method of the topographically guided diffusion model in the embodiments of this application;

[0063] Figure 5This is a DEM cavity filling result diagram of the DEM cavity filling method of the topographic line-guided diffusion model in the embodiments of this application;

[0064] Figure 6 This is a multi-resolution DEM filling result image of the DEM hole filling method of the topographic line-guided diffusion model in the embodiments of this application;

[0065] Figure 7 This is a schematic diagram of the electronic device structure of the DEM void filling method of the topographically guided diffusion model in the embodiments of this application. Detailed Implementation

[0066] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0067] The Denoising Diffusion Probabilistic Model (DDPM) has gained widespread attention in recent years due to its powerful generative capabilities, becoming the highest-performing and most mainstream generative model. Compared to GANs, the diffusion model has a simpler and more stable training process, enabling it to learn the distribution of the training data more completely. The trained diffusion model can generate more diverse and visually appealing images. The powerful generative capabilities of the diffusion model can be used to improve the visual quality of DEM hole filling results, eliminating unnatural artifacts and elevation jumps at boundaries. However, the generation process of the unconditional diffusion model is completely random. The highest requirement for DEM hole filling tasks is that the content inside the holes matches the true elevation in nature, which requires additional known information to provide clues to the diffusion model's generation process.

[0068] Therefore, it is essential to integrate simple and easily expressible terrain knowledge into the diffusion model, reduce the randomness of the model's generated content, and guide the process of filling the holes in the DEM.

[0069] A comprehensive analysis of current research on DEM hole filling and the development status of diffusion models reveals that diffusion models have significant application potential in DEM hole filling tasks. Diffusion models possess powerful generative capabilities and the ability to fuse multimodal data. Their strong generative capabilities overcome the artifact problem in the filled DEM. Guided by known terrain knowledge, the model can accurately reconstruct missing terrain features inside the holes, improving elevation accuracy, slope accuracy, and image similarity.

[0070] Embodiments of this application provide a method for filling DEM voids in a topographically guided diffusion model, comprising:

[0071] S1: Obtain DEM images and construct a training set of the DEM images;

[0072] S2: Extract ridgelines and valleylines from the DEM image using hydrological analysis to construct a topographic mask;

[0073] S3: Under the conditions of the terrain line mask, train the diffusion model using the training set;

[0074] S4: Using the trained diffusion model and the terrain line mask of the known area, the holes in the unknown area are filled through iterative denoising.

[0075] Specifically, the diffusion model has powerful generation capabilities and the ability to fuse multimodal data. Its powerful generation capabilities overcome the problem of artifacts in the filled DEM. Guided by known terrain knowledge, the diffusion model can accurately reconstruct the missing terrain features inside the holes, improving elevation accuracy, slope accuracy, and image similarity.

[0076] Specifically, the software used to extract ridgelines and valley lines from DEM images through hydrological analysis can be ArcGIS professional geographic information software.

[0077] Step S1 includes: using the QGIS function to export images and crop them into blocks of a specified size, the DEM image is constructed into image blocks of size 256×256. The size of the image blocks is limited by the GPU computing power. When the computing power allows, the larger the size of the image blocks, the better the performance.

[0078] Step S2 includes:

[0079] S21: Use ArcGIS spatial analysis tools and hydrological analysis methods to extract the ridgeline grid ridge_1, and calculate the positive terrain DEM_pos and negative terrain DEM_neg of the DEM image;

[0080] S22: Refine the raster ridge_1 and multiply it with the positive terrain DEM_pos to obtain the final ridge line;

[0081] S23: Using the ABS function, the inverse terrain is calculated, the ridgeline is extracted from the inverse terrain, and the valley line is obtained;

[0082] S24: Obtain the terrain mask tf_mask through the ridge line and valley line, with the ridge line pixel value being 1, the valley line pixel value being -1, and the background pixel value being 0.

[0083] Step S3 includes:

[0084] S31: Divide the terrain line mask into DEM blocks corresponding to the DEM image to form the terrain line part of the training set;

[0085] S32: Initialize the hyperparameters of the diffusion model; set the total number of forward noise addition steps T of the diffusion model; assume the relationship between noise and time t is a cosine function;

[0086] S33: The diffusion model randomly loads DEM blocks and corresponding terrain line masks (tf_mask) from the training set; randomly obtains time t and random Gaussian noise ∈ t ~N(0,I);

[0087] The diffusion model defines the forward diffusion process of a DEM image from time zero to time T by gradually adding random Gaussian noise:

[0088]

[0089] Define α t =1-β t Then the following relationship exists:

[0090]

[0091] Using the two formulas above, we can derive x. t The relationship between x0 and x0 is as follows:

[0092]

[0093] and:

[0094]

[0095] Where x0 represents sample data selected from the training set; x t The image represents the DEM noise image at time t; the forward diffusion process of the diffusion model applies a reparameterization technique to ensure that the image x after adding noise is properly positioned. t For noise ∈ t It is differentiable, which facilitates gradient calculation and thus facilitates neural network training; the diffusion model defines T fixed variance tables, and the variance at time t is β. t ; I represents the identity matrix; x t The relationship between x and x0 indicates that the diffusion model does not require t noise additions to go from x0 to x t , in β t Given that α t It is known that Given the information, noise can be added in one step; q(x) t |x t-1 ) indicates from x t-1 The obtained x t The distribution it follows; : = represents the distribution defined as follows; N(b; C, D) represents the Gaussian distribution that random variable B follows, with mean C and variance D; N represents the Gaussian distribution N; α i =1-βi i = 1, 2, ..., t;

[0096] Based on the diffusion process, the noise image x at time t is obtained. t , to noisy image x t Tensor concatenation is performed with the terrain line mask tf_mask, and the resulting tensor is used as the input to the diffusion model;

[0097] S34: Train a U-Net neural network to predict the inverse denoising process of the diffusion model from x0 to x. t The added noise; determine the predicted noise ∈ θ With actual added noise ∈ t The error between them, and the variational lower bound L between the noise distribution and the true noise distribution. vlb ; Variational lower bound L vlb As the loss function of the diffusion model;

[0098]

[0099] L hybrid =L simple +L vlb

[0100] Among them, c rf For terrain line masking; L simple This represents the simplified MSE loss function; L represents the expectation operation; hybrid This represents the final comprehensive loss function used. This represents the noise predicted by the neural network under parameter θ;

[0101] S35: Randomly select time t within the range [0, 1000], where t is an integer, and repeat step S34 to train the learning ability of the diffusion model under the condition of terrain line mask; the learning ability represents the learning of the mapping relationship between terrain lines and DEM images.

[0102] Specifically, random noise ∈ is added from x0. t Get x t Using neural networks to predict random noise ∈ t Then we can know only x t In the case of x0, restore x0. Add random noise ∈ from x0. t-1 Get x t-1 x t-1 With x t Between, can be seen as from x t-1 Add random noise ∈ to obtain x t In obtaining x t ,x0,xt-1 Then, ∈ can also be derived, which completes the derivation from x. t Denoising can be achieved in one step. t-1 .

[0103] Specifically, DEM images contain ridge and valley features, which we summarize using ridge lines and valley lines. Pixel values ​​on and near ridge lines are higher than pixel values ​​at other locations, while pixel values ​​on and near valley lines are lower than pixel values ​​at other locations. This relationship is not explicitly expressed, but rather learned by training a diffusion model using DEM and topographic line data, allowing the model to understand the relationship between the elevation values ​​corresponding to topographic lines being higher / lower than other locations during training.

[0104] The forward diffusion process in a diffusion model destroys the image, while the reverse denoising process generates an image from the noise. If the diffusion model can predict the noisy image x at time t... t The diffusion model's ability to predict noise added between the clear image x0 at time 0 and the noise source allows it to reconstruct a clear image by subtracting the noise from the noisy image. This prediction is achieved through a neural network. We can leverage this ability to predict the noise difference between a given random noise source and a clear image, and then remove the noise to obtain the original image corresponding to the noisy one. This corresponds to the model's process of generating an image from scratch. In practical applications, the model starts with random noise, and the specific content of the generated image is unpredictable. Therefore, we added terrain lines. The model not only generates a clear DEM image from the noise, but the contour of the generated DEM image also matches the contour described by the terrain lines.

[0105] Step S4 includes:

[0106] S41: Visually draw the ridgeline and valley line corresponding to the known area from satellite imagery or other known DEM data at the same location, obtain the topographic mask of the known area, and input the hole DEM and the topographic mask of the known area into the trained diffusion model.

[0107] S42: The strategy for inverse denoising and image generation using the diffusion model is as follows:

[0108]

[0109] in, This represents the derived x under the noise predicted by the neural network with parameter θ. t-1 Based on the noise addition principle of the diffusion model described above, the distribution-related parameters are derived. The x derived from the model t-1 μ θΣ represents the mean derived from the model; N represents the Gaussian distribution; Σ θ This represents the variance derived from the model.

[0110] Obtain the noise in the diffusion model prediction and the variance Σ in the Gaussian transform. θ (x t ,t,c tf );

[0111] Based on the above strategy, noise is gradually denoised in reverse from time T, where t = 0, 1, ..., T;

[0112] S43: In the task of filling holes in DEM images, the part inside the hole is generated by a diffusion model, while the known DEM image data outside the hole is kept unchanged.

[0113] Let a complete DEM image be x, the known region be m⊙x, the hole region be (1-m)⊙x, the hole mask be m, and the known region after forward noise addition be m. as follows:

[0114]

[0115] Where x0 represents the training sample selected from the training set;

[0116] Hollow regions inferred from the diffusion model as follows:

[0117]

[0118] The known regions and predicted hole regions are merged to obtain the x-image. t-1 as follows:

[0119]

[0120] Merge image x t-1 Ten steps of adding noise are performed to obtain a noisy image with more noise at time T; the noisy image is then denoised in reverse for 10 steps. During this process, the known information outside the hole will constrain the diffusion model, making the semantics on both sides of the hole boundary consistent.

[0121] S44: Repeat step S42 a preset number of times, and then denoise for 10 more steps until the image at the next time step is obtained;

[0122] S45. Repeat steps S43 and S44 until t=0 to obtain a noise-free image, thus completing the hole filling of the unknown area of ​​the DEM image.

[0123] Specifically, visual drawing uses vector lines in GIS software to draw ridgelines and valley lines, which are then converted into raster maps.

[0124] Specifically, too few preset iterations are insufficient to fully leverage the constraints imposed by the known portions of the image on the generation of content within the holes. After a certain number of repetitions, the model performance stabilizes; increasing the number of iterations only lengthens the model's runtime without further improving performance. The empirical value for the preset number of iterations in this application is 10.

[0125] Specifically, initialize the hyperparameters of the diffusion model, including batch size and learning rate. Set the total number of forward noise addition steps T of the diffusion model to 1000.

[0126] In one embodiment, it includes:

[0127] Step 1: Download DEM images and construct a training set.

[0128] Step 1.1: Download ASTERGDEMv3 images of the Tibetan Plateau, Rocky Mountains, and Andes Mountains at a resolution of 1 Arc-Second and TanDEM-X images at a resolution of 3 Arc-Second.

[0129] Step 1.2: Using QGIS's image export and cropping function, the DEM image is cropped into 256×256 DEM blocks, resulting in 260,000 ASTER GDEM blocks and 2,700 TanDEM-X DEM blocks. Blocks containing holes in the TanDEM-X DEM have been removed. The cropped DEM blocks are shown below. Figure 2 As shown in (a).

[0130] Step 1.3: Construct the ASTER GDEM training set and the TanDEM-X DEM training set respectively.

[0131] Step 2: Use hydrological analysis to obtain ridgelines and valley lines, and construct topographic line masks.

[0132] Step 2.1: Extract the ridgeline raster ridge_1 using ArcGIS spatial analysis tools and hydrological analysis methods, and calculate the positive terrain DEM_pos and negative terrain DEM_neg of the DEM;

[0133] Step 2.2: Refine ridge_1 to obtain a ridgeline grid with a width of 1 pixel, and multiply it with the positive terrain DEM_pos to obtain the final ridgeline;

[0134] Step 2.3: Use the formula for calculating abs (original DEM-max elevation value) to obtain the reverse terrain. Extract the ridgeline from the reverse terrain to obtain the valley line.

[0135] Step 2.4: The ridge-valley model yields a terrain mask (tf_mask). Ridge lines have a pixel value of 1, valley lines have a pixel value of -1, and the background has a pixel value of 0. The entire terrain mask is exported from QGIS and divided into smaller blocks corresponding to the DEM to form the terrain line portion of the training set. The terrain mask is shown below. Figure 2 (b) shows the step 3: Predict the noise in the reverse denoising process using the U-Net neural network; train the diffusion model to learn the mapping relationship between the DEM image and the corresponding topographic lines.

[0136] Specifically, the diffusion model includes a forward diffusion process and a backward denoising process. The backward denoising process requires a neural network to predict the noise that needs to be removed at each step. This neural network is usually U-Net.

[0137] Step 3.1: Initialize network training hyperparameters: batch size = 1, learning rate = 0.002, etc. Set the total number of forward noise addition steps T of the diffusion model to 1000, and the relationship between noise and time t is a cosine function.

[0138] Step 3.2: Randomly load DEM blocks and corresponding terrain line masks (tf_mask) from the training set, randomly obtain time t, and random Gaussian noise ∈ t ~N(0,I), the diffusion model defines the forward diffusion process of an image from time 0 to time T by gradually adding standard Gaussian noise:

[0139]

[0140]

[0141] Where x0 is the sample DEM loaded from the training set. The DEM noise image x at time t is obtained according to the above diffusion process. t The noisy image is tensor-stitched with the terrain line mask tf_mask;

[0142] Step 3.3: The noise added during training of the diffusion model, predicting the noise ∈ θ The error between the predicted noise distribution and the actual noise distribution, and the variational lower bound L between the predicted noise distribution and the actual noise distribution. vlb Variational lower bound L vlb As the loss function of the model.

[0143] Step 3.4: Randomly select a time t in the range [0,1000], where t is an integer, and repeat step 3.3 to train the diffusion model's ability to predict noise under terrain line mask conditions.

[0144] The trained parameters are stored in three checkpoint files, and the DEM holes are filled using the model parameters averaged by the EMA (Exponential Moving Average) method. Figure 3 This is a schematic diagram of forward noise addition and backward noise reduction in a diffusion model, revealing the basic working principle of the diffusion model.

[0145] Step 3.5: Train on two separate training sets to obtain two different diffusion models.

[0146] Step 4: Fill the holes in the DEM image by iterative denoising.

[0147] Step 4.1: Visually draw the ridgeline and valley line corresponding to the test area from satellite imagery or other known DEM data at the same location to obtain the topographic mask. Input the hole DEM and topographic mask into the network.

[0148] Step 4.2: Based on the strategy of reverse denoising using the diffusion model to generate images, the model predicts the noise to be removed from the current noisy image to the next step and the variance Σ in the Gaussian transformation. θ (x t ,t,c tf According to the above formula, noise is gradually reduced from time T. Based on the principle of accelerated sampling using the diffusion model, the skip sampling step of the 1000-step noise reduction process is set to 250 steps.

[0149] Step 4.3: In the DEM hole filling task, only the part inside the hole needs to be network-generated; the known DEM data outside the hole needs to be preserved. The DEM of the hole to be filled is x, and the known area after forward noise addition is: The void region inferred by the diffusion model is:

[0150] By combining the known region and the predicted void region, we obtain x. t-1

[0151] The merged result is denoised for 10 steps to obtain an image with more noise closer to time T. This image is then denoised in reverse for 10 steps. During this process, the known content outside the hole constrains the diffusion model, making the semantics on both sides of the hole boundary consistent.

[0152] Step 4.4: Repeat step 4.2 10 times, then perform 10 more noise reduction steps before moving to the next time step; Figure 4 This invention demonstrates the strategy of stitching together known DEM information and iteratively denoising.

[0153] Step 4.5: Repeat steps 4.3 and 4.4 until t=0 to obtain a noise-free image, thus completing the DEM hole filling. The reference coordinate system of the filled DEM image is lost and needs to be reset. Figure 5 The results of the void filling are shown.

[0154] Step 5: Test the effectiveness of the invention on DEM images at different resolutions.

[0155] Step 5.1: Acquire TanDEM-X DEM data at resolutions of 12m, 30m, and 90m. Based on the corresponding relative elevation error map, set a threshold to remove DEM pixels with large errors as holes. The threshold setting method is as follows:

[0156]

[0157] Among them, V hem This represents the HEM image corresponding to a DEM image. hem Std represents the mean of all HEM pixels except for holes. hem This represents its standard deviation. ΔH i This represents the pixels on the HEM image. P is the set of data hole pixels, and i represents the pixels on the HEM image.

[0158] Step 5.2: Hole filling is performed on the models trained on the STER GDEM training set and the TanDEM-X DEM training set. The results of hole filling for DEMs at different resolutions are shown below. Figure 6 As shown. It should be noted that the 30m model was trained on ASTER GDEM data obtained by optical stereo photogrammetry. The test data here, however, are TanDEM-X DEMs obtained through InSAR technology. The almost identical filling results demonstrate that the present invention also exhibits robustness to DEM datasets obtained using different measurement techniques.

[0159] This application demonstrates that models trained on DEM datasets with different resolutions than the training set can also achieve excellent results on 12m or 90m DEM test data. For the same 30m test DEM, models trained on the 30m training set and models trained on the 90m training set exhibit almost identical accuracy.

[0160] The diffusion model was trained using a training set of DEM images, and it was able to generate rich surface textures inside the holes. Under the joint constraints of the known DEM image and topographic lines outside the holes, the diffusion model could generate DEM surfaces that were consistent with the real terrain and had no elevation jumps at the hole boundaries. The content filled inside the holes was consistent with the topographic line conditions, indicating that the results of multiple tests on the same test data were almost consistent. Compared with other deep learning-based methods, the stability of the diffusion model was greatly improved.

[0161] The DEM void filling method of this application has the following three significant characteristics:

[0162] The first notable feature is its powerful DEM image generation capability: the diffusion model can learn the distribution of the training set more completely, and under the control of the external information of the holes, it can smooth the DEM semantics on both sides of the boundary, overcoming the defects of the interpolation method and the GAN method in elevation jump at the boundary, and the model will not generate abnormal artifacts.

[0163] The second significant feature is its ability to learn the mapping relationship between topographic lines and the DEM. During the testing phase, the elevation information inside the holes in the DEM is completely missing. The trained model can reconstruct the DEM based on the learned mapping relationship, even with only the topographic line information inside the holes. Assuming the topographic lines are accurate, the model can accurately reconstruct the missing surface contours. Furthermore, guided by the topographic lines, the model's results on multiple tests of the same test data are very similar, improving the model's stability.

[0164] The third notable feature is its applicability to DEMs of different resolutions. A model trained on a 30m resolution DEM training set shows almost no fluctuation in accuracy when filling DEMs with resolutions of 12m, 30m, and 90m. This feature allows the model to be extended to more DEM datasets. For example, when a DEM at a certain resolution is insufficient to form a training set for model training, a diffusion model can be trained on a publicly available global DEM dataset, and the trained model can be used to fill holes in the target resolution DEM.

[0165] In summary, the DEM hole filling method based on the topographic line-guided diffusion model proposed in this application has the following advantages: the model training is simple and does not require careful adjustment of hyperparameters and model training strategies; the filled DEM hole images show excellent performance in both quantitative and qualitative evaluation; the model is stable and can accurately reconstruct complex terrain features; under the guidance of the topographic line, the uncertainty of the content filled inside the holes is greatly reduced, and the model has strong robustness to DEM resolution.

[0166] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0167] This application also discloses an electronic device. (See reference...) Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0168] The communication bus 502 is used to enable communication between these components.

[0169] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0170] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0171] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0172] The memory 505 may include random access memory (RAM) or read-only memory.

[0173] Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. The memory 505 may also optionally include, but is not limited to, at least one storage device located remotely from the aforementioned processor 501. (Refer to...) Figure 7 The memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a DEM void filling method based on a topographically guided diffusion model.

[0174] exist Figure 7 In the illustrated electronic device 500, the user interface 503 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 501 can be used to call an application stored in the memory 505 for a DEM hole filling method of a terrain-guided diffusion model. When executed by one or more processors 501, the electronic device 500 performs one or more methods as described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0175] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0176] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed include, but are not limited to, indirect couplings or communication connections between apparatuses or units through some service interfaces, including but not limited to electrical or other forms.

[0177] The units described as separate components include, but are not limited to, physically separate units. The components shown as units include, but are not limited to, physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of these units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or may exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0180] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will readily conceive of those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0181] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for filling holes in a DEM (Digital Elevation Model) guided by topographic lines, characterized in that, The method includes the following steps: S1: Obtain DEM images and construct a training set of the DEM images; S2: Extract ridgelines and valleylines from the DEM image using hydrological analysis to construct a topographic mask; S3: Under the conditions of the terrain line mask, train the diffusion model using the training set; Step S3 includes: S31: Divide the terrain line mask into DEM blocks corresponding to the DEM image to form the terrain line part of the training set; S32: Initialize the hyperparameters of the diffusion model; set the total number of forward noise addition steps T for the diffusion model; set the noise versus time... The relationship is a cosine function; S33: The diffusion model randomly loads DEM blocks and corresponding terrain line masks (tf_mask) from the training set; randomly obtains time steps. and random Gaussian noise ; The diffusion model defines the forward diffusion process of a DEM image from time zero to time T by gradually adding random Gaussian noise: definition Then the following relationship exists: Using the two formulas above, we can derive... and The relationship is as follows: and: in, The sample data selected from the training set; The image represents the DEM noise image at time t; the forward diffusion process of the diffusion model applies a reparameterization technique to ensure the image quality after noise addition. noise It is differentiable, which facilitates gradient calculation and thus facilitates neural network training; the diffusion model defines T fixed variance tables, and the variance at time t is... ; Represents the identity matrix; and The relationship indicates that the diffusion model does not require t noise additions to achieve the desired effect. arrive ,exist Given that It is known that Given the information, noise can be added in one step. Indicates from Obtained The distribution it follows; It is defined as; Let C represent the Gaussian distribution that random variable B follows, with mean C and variance D. Indicates that it follows a Gaussian distribution ; i = 1, 2, ..., t; Based on the diffusion process, the noise image at time t is obtained. Noisy image Tensor concatenation is performed with the terrain line mask tf_mask, and the resulting tensor is used as the input to the diffusion model; S34: Train a U-Net neural network to predict the inverse denoising process of the diffusion model. arrive The added noise; determine the predicted noise. Compared to actual added noise The error between them, and the variational lower bound between the noise distribution and the true noise distribution. ; Variational lower bound As the loss function of the diffusion model; in, For terrain line masking; This represents the simplified MSE loss function; This represents the expectation operation; This represents the final comprehensive loss function used. Indicates in the parameter The noise predicted by the neural network under the given conditions; S35: Randomly select a time t within the range [0, 1000], where t is an integer, and repeat step S34 to train the learning ability of the diffusion model under the condition of terrain line mask; the learning ability represents the learning of the mapping relationship between terrain lines and DEM images. S4: Using the trained diffusion model and the terrain line mask of the known area, fill the holes in the unknown area through iterative denoising; Step S4 includes: S41: Visually draw the ridgeline and valley line corresponding to the known area from satellite imagery or other known DEM data at the same location, obtain the topographic mask of the known area, and input the hole DEM and the topographic mask of the known area into the trained diffusion model; S42: The strategy for inverse denoising and image generation using the diffusion model is as follows: in, Indicates in the parameter The following is derived from the neural network's prediction under noise conditions. Based on the noise addition principle of the diffusion model described above, the distribution-related parameters are derived. Indicates the model derived ; This represents the mean derived from the model; Indicates a Gaussian distribution; This represents the variance derived from the model. Obtain the noise in the predictions of the diffusion model and the variance in the Gaussian transform. ; Based on the above strategy, noise reduction is performed step by step in reverse from time T. ; S43: In the task of filling holes in DEM images, the part inside the hole is generated by a diffusion model, while the known DEM image data outside the hole is kept unchanged. Let a complete DEM image be The known region is m x, the cavity region is (1-m) x, with hole mask m, is the known region after forward noise addition. ,as follows: in, This represents the training samples selected from the training set; Hollow regions inferred from the diffusion model ,as follows: The known regions and predicted hole regions are merged to obtain the merged image. as follows: Merge images Ten steps of adding noise are performed to obtain a noisy image with more noise at time T; the noisy image is then denoised in reverse for 10 steps. During this process, the known information outside the hole will constrain the diffusion model, making the semantics on both sides of the hole boundary consistent. S44: Repeat step S42 a preset number of times, and then denoise for 10 more steps until the image at the next time step is obtained; S45: Repeat steps S43 and S44 until... This yields a noise-free image, thus completing the hole filling in the unknown areas of the DEM image.

2. The method for filling DEM holes in a topographically guided diffusion model as described in claim 1, characterized in that, Step S1 includes: Using QGIS's function of exporting images and cropping them into blocks of a specified size, the DEM image is constructed into image blocks of 256×256. The size of the image block is limited by the GPU's computing power. When the computing power allows, the larger the size of the image block, the better the performance.

3. The method for filling DEM voids in a topographically guided diffusion model as described in claim 1, characterized in that, Step S2 includes: S21: Use ArcGIS spatial analysis tools and hydrological analysis methods to extract the ridgeline grid ridge_1, and calculate the positive topography DEM_pos and negative topography DEM_neg of the DEM image; S22: Refine the raster ridge_1 and multiply it with the positive terrain DEM_pos to obtain the final ridge line; S23: Using the ABS function, the inverse terrain is calculated, the ridgeline is extracted from the inverse terrain, and the valley line is obtained; S24: Obtain the terrain mask tf_mask through the ridge line and valley line, with the ridge line pixel value being 1, the valley line pixel value being -1, and the background pixel value being 0.

4. An electronic device, characterized in that, The device includes a processor (501), a memory (505), a user interface (503), and a network interface (504). The memory (505) is used to store instructions. The user interface (503) and the network interface (504) are used to communicate with other devices. The processor (501) is used to execute the instructions stored in the memory (505) to cause the electronic device (500) to perform a DEM void filling method for a topographically guided diffusion model as described in any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform a DEM void filling method for a topographically guided diffusion model as described in any one of claims 1-3.

Images