Satellite-borne laser radar digital elevation model generation method based on generative adversarial network
By combining the generative adversarial network with terrain features and spatial interpolation knowledge constraints, the problem of sparse distribution of spaceborne lidar data was solved, a high-resolution digital elevation model was generated, and high-quality terrain data generation was achieved.
Patent Information
- Application Number
- CN202510817971.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-18
- Publication Date
- 2025-09-19
AI Technical Summary
Traditional spatial interpolation methods cannot effectively handle the sparse and non-random distribution of spaceborne lidar data, resulting in the inability to generate high-resolution digital elevation models.
A generative adversarial network is used to combine terrain feature constraints and spatial interpolation knowledge constraints. The U-Net network generator and the fully convolutional network discriminator are used to train the generative adversarial network model to generate a high-resolution digital elevation model.
It achieves the goal of learning rich terrain information from sparse spaceborne lidar data and generating high-resolution digital elevation models, avoiding the problem of violating terrain features that may be caused by unconstrained generative adversarial networks.
Smart Images
Figure CN120669223A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital elevation model generation, and in particular to a method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network. Background Art
[0002] LiDAR is an active remote sensing technology that uses laser pulses to accurately measure the three-dimensional coordinates of target objects. Thanks to its short wavelength and dense point cloud, laser pulses can penetrate the vegetation canopy through the gaps between leaves and branches to obtain information about the terrain under the forest. Therefore, compared with traditional optical remote sensing and synthetic aperture radar, LiDAR has a strong vegetation penetration capability, which gives it a significant advantage in obtaining digital elevation model (DEM) terrain data in vegetation-covered areas. However, the spatial coverage of ground-based or airborne LiDAR is limited, and large-scale DEM production is not possible. Satellite-borne LiDAR orbits at a high altitude, and its footprint can periodically cover most of the global surface. Using appropriate interpolation techniques, large-scale, high-precision terrain elevation data can be generated.
[0003] However, the spatial distribution of spaceborne lidar ground footprints is relatively sparse and non-random. For example, the latest Ice, Cloud, and Elevation Satellite 2 (ICESat-2) and the Global Ecosystem Dynamics Investigation (GEDI) satellites have ground footprints with an on-track spacing of 0.7 m and a maximum cross-track spacing of 3.3 km. GEDI ground footprints have on-track spacing of 60 m and a cross-track spacing of 600 m. Traditional spatial interpolation methods, such as kriging and inverse distance weighted interpolation, are not well suited to such sparsely and non-randomly distributed data. Therefore, traditional spatial interpolation methods cannot produce high-resolution DEMs based on spaceborne lidar data.
[0004] Currently, numerous deep learning models are used for interpolation, reconstruction, and gap filling in digital elevation models. Generative Adversarial Networks (GANs) are widely used in DEM research due to their excellent ability to capture and represent data distribution characteristics. However, as a direct data-driven approach, GANs can produce undesirable results, such as violations of terrain geometry and lack of detailed terrain features, if constraints are not introduced to control the interpolation process. Summary of the Invention
[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for generating a digital elevation model of a space-borne lidar based on a generative adversarial network.
[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: A method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network comprises the following steps: Extract terrain elevation point data based on spaceborne lidar data; Performing rasterization processing on the terrain elevation point data to obtain rasterized terrain elevation point data; Calculate Kriging interpolation results and spatial interpolation knowledge constraints based on rasterized terrain elevation point data; Compute terrain feature constraints from multispectral data; A generative adversarial network model is trained based on rasterized terrain elevation point data, kriging interpolation results, spatial interpolation knowledge constraints, terrain feature constraints, and ground truth, and the data overlap is determined based on the training results. The application input data is constructed according to the data overlap, and the trained generative adversarial network model is input to generate a digital elevation model subset. The digital elevation model subset is then edge cropped and weighted average fused to obtain the final digital elevation model.
[0007] Furthermore, the terrain elevation point data is rasterized to obtain rasterized terrain elevation point data, including: In the digital elevation model generation area, a regular grid of a set size is established, and the average value of the elevation point data falling in the grid is used as the grid value, otherwise the grid value is zero.
[0008] Furthermore, the knowledge constraints for calculating spatial interpolation based on rasterized terrain elevation point data include: Calculate the shortest spatial distance between the point to be interpolated and the known elevation point based on the rasterized terrain elevation point data.
[0009] Furthermore, computing terrain feature constraints based on multispectral data includes: The shadow segmentation index is calculated based on the near-infrared band data and the short-wave infrared band data, and then the spectral image is divided into shadow area and non-shadow area, that is, shady slope and sunny slope according to the set index threshold.
[0010] Furthermore, the shadow segmentation index is calculated as follows:
[0011] in, is the shadow segmentation index, For band In position The normalized pixel intensity, For band The empirical parameters of is the total number of bands.
[0012] Furthermore, the generative adversarial network model includes a generator and a discriminator. The generator adopts the U-Net network and the discriminator adopts the fully convolutional network.
[0013] Furthermore, the input data of the generator includes rasterized terrain elevation point data, Kriging interpolation results, spatial interpolation knowledge constraints, and terrain feature constraints.
[0014] Furthermore, the input data of the discriminator includes the generator generation results, ground truth, Kriging interpolation results and spatial interpolation knowledge constraints and terrain feature constraints.
[0015] Furthermore, the generator is trained by minimizing the loss function, which is expressed as:
[0016]
[0017] in, The loss for the difference between the generator's generated results and the local ground truth, Generate results for the generator, is the ground truth, is the loss function of the generator, is the mathematical expectation, is the difference between the generator's generated result and the local ground truth, is the weight coefficient of the gap loss.
[0018] Furthermore, the discriminator is trained by maximizing the loss function, which is expressed as:
[0019] in, is the loss function of the discriminator.
[0020] The present invention has the following beneficial effects: (1) Due to the sparse ground footprint of spaceborne lidar and its uniform spatial distribution according to orbit, traditional spatial interpolation methods cannot meet the production requirements of high-resolution spaceborne lidar digital elevation models. This paper uses generative adversarial networks to replace traditional spatial interpolation methods, breaking through the interpolation bottleneck of traditional spatial interpolation methods. It can learn rich terrain information from limited altimetry sample points and perform reasonable extrapolation, thereby achieving the production of high-resolution spaceborne lidar digital elevation models.
[0021] (2) For the direct data-driven approach of generative adversarial networks, this paper establishes the terrain feature constraints of shady and sunny slopes and the spatial interpolation knowledge constraints of Euclidean distance from the perspective of geographic space, thus realizing the control of the elevation interpolation process of the generative adversarial network and avoiding the problem that the unconstrained generative adversarial network may generate elevation data that violates the terrain features and terrain details. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A schematic flow chart of a method for generating a digital elevation model of a space-borne lidar based on a generative adversarial network according to an embodiment of the present invention; Figure 2 A schematic diagram of the elevation point distribution of a space-borne laser radar provided in an embodiment of the present invention; Figure 3 A schematic diagram of a data set provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a generative adversarial network provided by an embodiment of the present invention; Figure 5 Schematic diagram of edge clipping and weighted average fusion provided by an embodiment of the present invention; (a) is edge clipping, and (b) is weighted average fusion; Figure 6 This is a schematic diagram of the final generated digital elevation model provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.
[0024] like Figure 1 As shown, an embodiment of the present invention provides a method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network, comprising the following steps S1 to S6: S1. Extract terrain elevation point data based on space-borne lidar data; In an optional embodiment of the present invention, step S1 selects an area with complex topography and rich vegetation as an example, and obtains the ground elevation points of the satellite-borne lidar in the area (altimetry data combined with ICESat-2 and GEDI), such as Figure 2 In addition, the ground truth needs to cover a portion of the instance area.
[0025] This embodiment preprocesses the acquired spaceborne lidar data. The preprocessing includes denoising and outlier removal. Terrain elevation point data is then extracted.
[0026] S2. performing rasterization processing on the terrain elevation point data to obtain rasterized terrain elevation point data; In an optional embodiment of the present invention, step S2 establishes a regular grid of a certain size (such as 5m) within the digital elevation model generation area, and takes the average value of all elevation point data falling in the grid cell as the grid cell elevation value. If there is no elevation point in the cell, the grid cell elevation value is zero.
[0027] S3. Calculate the Kriging interpolation results and spatial interpolation knowledge constraints based on the rasterized terrain elevation point data; In an optional embodiment of the present invention, step S3 uses the generated rasterized terrain elevation point data to calculate its Kriging interpolation result and spatial interpolation knowledge constraints, such as Figure 3 As shown in , it is the shortest spatial distance from the point to be interpolated to the known elevation point.
[0028] Spatial interpolation knowledge constraints are usually constructed based on spatial distance. According to the first law of geography, the closer the spatial distance between objects, the greater the correlation. Therefore, this embodiment introduces spatial distance to constrain the degree of influence of known elevation grids on unknown elevation grids during the interpolation process. The spatial distance here refers to the two-dimensional plane Euclidean distance, which is calculated as follows:
[0029] in, , represents the plane coordinates of a known elevation grid, , Plane coordinates representing the unknown elevation grid.
[0030] S4, calculating terrain feature constraints based on multispectral data; In an optional embodiment of the present invention, step S4 uses the near-infrared and two short-wave infrared data from Sentinel-2 to calculate the shadow segmentation index, and then extracts the shady slope and sunny slope through index threshold segmentation.
[0031] This embodiment takes into account that DEM is a digital expression of the terrain surface, and constructs terrain feature constraints and spatial interpolation knowledge constraints from a geographic space perspective to control the interpolation process of the generative adversarial network.
[0032] When constructing terrain feature constraints, this embodiment uses an intuitive natural perception knowledge - shady slopes and sunny slopes, which can be regarded as a special case of mountain shadows. The formation of shady slopes and sunny slopes is directly affected by the solar altitude angle and the solar azimuth angle. The solar altitude angle controls the formation and distribution of shadows. The larger the altitude angle, the fewer shadows are produced; the smaller the altitude angle, the more shadows are produced, and the larger the coverage range. When the solar altitude angle is around 45°, the shadows formed can best reflect the changes in terrain undulations. The construction of terrain feature constraints is the extraction of shady slopes and sunny slopes. We use Sentinel-2 multispectral data as the basis for shadow extraction and segmentation. When selecting Sentinel-2 images, it is necessary to limit the solar altitude angle to around 45° as much as possible. The shadow segmentation index is calculated using the near-infrared band and two short-wave infrared bands of Sentinel-2, and then the image is divided into shadow areas and non-shadow areas, namely shady slopes and sunny slopes, according to the selected index threshold. The shadow segmentation index calculation formula is:
[0033] in, is the shadow segmentation index, For band In position The normalized pixel intensity, For band The empirical parameters of is the total number of bands. The larger (brighter) the pixel value of each band is, the The smaller (tends to 0); on the contrary, if the position The smaller the pixel value of each band (the darker), The larger it is (tends to 1).
[0034] After calculating the shadow segmentation index, the image is divided into shady slopes and sunny slopes using the following formula:
[0035] in, Indicates the slope type, is the shadow segmentation threshold.
[0036] This example calculates terrain feature constraints (shady slopes and sunny slopes) based on Sentinel-2 multispectral data. In this example, the The empirical parameters are 6, 10, and 10, corresponding to the near-infrared band and two short-wave infrared bands, respectively. The value is 0.096. The schematic diagram of all calculation results is as follows Figure 3 As shown in d.
[0037] S5. Train a generative adversarial network model based on the rasterized terrain elevation point data, the kriging interpolation results, the spatial interpolation knowledge constraints, the terrain feature constraints, and the ground truth, and determine the data overlap based on the training results; In an optional embodiment of the present invention, step S5 uses the generated rasterized terrain elevation point data, Kriging interpolation results, spatial interpolation knowledge constraints, terrain feature constraints and ground truth data to create a training data set, and cuts the original data into A subset of pixel sizes is randomly selected, and 80% is used for model training and 20% is used for model verification and edge cropping size determination.
[0038] Input the training data into the generative adversarial network for training, such as Figure 4 As shown in Figure 2, after the training is completed, the validation data is used to verify the model and determine the edge clipping size. The edge clipping size determination method is: after the validation set is input into the model to generate the DEM block, The pixel-sized DEM blocks are layered, with the outermost circle of pixels connected end to end forming a layer, and the layers are layered inwards pixel by pixel. The pixel-sized DEM block can be divided into 32 layers. The root mean square error of each layer is calculated to obtain an error line graph, and the cropping size is determined based on the error inflection point. In addition, the fusion range during weighted average fusion must be determined. A fusion range that is too large or too small will result in poor fusion results. It is usually set to 10%-30% of the image block pixel size. You can try multiple times to obtain the optimal fusion range size. In this example, the edge cropping size is 16 pixels, and the image size after cropping is Pixels, the selected fusion range size is 4 pixels.
[0039] The Generative Adversarial Network (G) consists of two main components: the generator (G) and the discriminator (D). The generator uses a U-Net architecture. Compared to the original encoder-decoder architecture, the U-Net's skip connection structure fuses feature maps from the encoder and decoder stages, preserving more spatial information and further improving the generator's performance. Unlike the generator, the discriminator uses a fully convolutional architecture. The generator's input data includes rasterized elevation data from spaceborne lidar terrain, shady and sunny slope data, spatial distance data, and kriging interpolation results from spaceborne lidar terrain elevation points. The generator no longer uses random noise as input to reduce the uncertainty of the generated results. The rasterized elevation data serves as the base input, while shady and sunny slopes and spatial distances serve as constraints, and the kriging interpolation results serve as background data. The discriminator's input data includes the generator's generated results, ground truth, shady and sunny slopes, spatial distances, and kriging interpolation results.
[0040] During the training of the model, the loss function is used to constrain and guide the training process of the generator. The first part of the loss function ( ) evaluates the gap between the generator's generated results and the local ground truth from the perspective of elevation value. The second part of the loss function ( ) is used to evaluate the overall similarity between the generator's generated results and the ground truth. Mathematical expectation and logarithmic operations are used to regularize the formula. The generator is trained by minimizing the loss function.
[0041] The generator is trained by minimizing the loss function, expressed as:
[0042]
[0043] in, The loss for the difference between the generator's generated results and the local ground truth, Generate results for the generator, is the ground truth, is the loss function of the generator, is the mathematical expectation, is the difference between the generator's generated result and the local ground truth, is the weight coefficient of the gap loss.
[0044] The task of the discriminator is to distinguish the generator's output from the ground truth as much as possible. The discriminator is trained by maximizing the loss function. It can be expressed as:
[0045] in, is the loss function of the discriminator.
[0046] It is worth noting that, whether it is the training or application stage of the model, for a wide range of application scenarios, it is usually necessary to crop the input data into multiple subsets for input to avoid memory overflow. Since the input and output images of the generative adversarial network are the same size, in order to obtain a complete digital elevation model, the output results need to be mosaicked. However, due to the cropping of the input data, contextual information such as terrain features and elevation ranges around the image will be lost, resulting in the peripheral accuracy of the output results being lower than the internal accuracy, especially in areas with complex terrain. Therefore, if the output results are directly spliced, the transition between adjacent results will be uneven and discontinuous. In response to the above situation, the generated results are post-processed. First, the edges of the single result are cropped. That is, if the original result image size is set to , the cropped size is , the cropping ratio can be estimated based on the accuracy of each layer from the periphery to the interior. In order to further make the mosaic result transition natural and smooth, the weighted average fusion method is used to fuse adjacent images, which is expressed as:
[0047] in, Indicates the fused elevation value, , Represents adjacent images (such as Figure 5 The elevation values of numbers 1 and 2) at the same coordinates, , Indicates the distance from the pixel to the edge in the fusion area, represents the sum of the two ( + ).
[0048] In summary, terrain feature constraints and spatial interpolation constraints are introduced into the generative adversarial network, and combined with edge clipping and weighted average fusion to realize the generation of high-resolution digital elevation model based on spaceborne lidar.
[0049] S6. Construct application input data based on data overlap, input the trained generative adversarial network model to generate a digital elevation model subset, and perform edge clipping and weighted average fusion on the digital elevation model subset to obtain the final digital elevation model.
[0050] In an optional embodiment of the present invention, step S6 reversely deduces the required overlap of the input data based on the determined edge cropping size and the fusion range, that is, the doubled cropping pixel size plus the fusion range size ( Pixels), application input data is generated based on the overlap. In this example, the overlap of the application input data is 56.25%.
[0051] The generated application input data is input into the trained generative adversarial network, and then edge clipping and weighted average fusion are applied. The final generated digital elevation model is as follows Figure 6 shown.
[0052] Compared with traditional elevation spatial interpolation methods, this paper uses generative adversarial networks to achieve the goal of producing high-resolution digital elevation models based on sparsely distributed spaceborne lidar altimetry data. It is an exploration and implementation of using deep learning methods and spaceborne lidar to generate high-quality digital terrain data.
[0053] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0054] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0056] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.
[0057] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.
Claims
1. A method for generating digital elevation models of space-borne laser radar based on generative adversarial networks, characterized in that: The following steps are involved: Extract terrain elevation point data based on spaceborne lidar data; Performing rasterization processing on the terrain elevation point data to obtain rasterized terrain elevation point data; Calculate Kriging interpolation results and spatial interpolation knowledge constraints based on rasterized terrain elevation point data; Compute terrain feature constraints from multispectral data; A generative adversarial network model is trained based on rasterized terrain elevation point data, kriging interpolation results, spatial interpolation knowledge constraints, terrain feature constraints, and ground truth, and the data overlap is determined based on the training results. The application input data is constructed according to the data overlap, and the trained generative adversarial network model is input to generate a digital elevation model subset. The digital elevation model subset is then edge cropped and weighted average fused to obtain the final digital elevation model.
2. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 1, wherein: The terrain elevation point data is rasterized to obtain rasterized terrain elevation point data, including: In the digital elevation model generation area, a regular grid of a set size is established, and the average value of the elevation point data falling in the grid is used as the grid value, otherwise the grid value is zero.
3. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 1, wherein: The knowledge constraints for calculating spatial interpolation based on rasterized terrain elevation point data include: Calculate the shortest spatial distance between the point to be interpolated and the known elevation point based on the rasterized terrain elevation point data.
4. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 1, wherein: Computing terrain feature constraints from multispectral data includes: The shadow segmentation index is calculated based on the near-infrared band data and the short-wave infrared band data, and then the spectral image is divided into shadow area and non-shadow area, that is, shady slope and sunny slope according to the set index threshold.
5. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 4, wherein: The shadow segmentation index is calculated as: in, is the shadow segmentation index, For band In position The normalized pixel intensity, For band The empirical parameters of is the total number of bands.
6. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 1, wherein: The generative adversarial network model includes a generator and a discriminator. The generator uses the U-Net network and the discriminator uses a fully convolutional network.
7. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 6, wherein: The input data of the generator includes rasterized terrain elevation point data, kriging interpolation results, spatial interpolation knowledge constraints, and terrain feature constraints.
8. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 7, wherein: The input data of the discriminator include the generator generation results, ground truth, Kriging interpolation results and spatial interpolation knowledge constraints and terrain feature constraints.
9. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 8, wherein: The generator is trained by minimizing the loss function, expressed as: in, The loss for the difference between the generator's generated results and the local ground truth, Generate results for the generator, is the ground truth, is the loss function of the generator, is the mathematical expectation, is the difference between the generator's generated result and the local ground truth, is the weight coefficient of the gap loss.
10. The method for generating a digital elevation model of a space-borne laser radar based on a generative adversarial network according to claim 9, wherein: The discriminator is trained by maximizing the loss function, which is expressed as: in, is the loss function of the discriminator.