A method for generating large-scale dynamic terrain based on multi-source structural data

Abstract

The present invention belongs to the technical field of terrain modeling, and specifically refers to a method for generating large-scale dynamic terrain based on multi-source structural data, the method including data sampling, pixel classification, band screening, spectral fusion, feature fusion and model generation. This scheme uses a dual-stream optimized U-Net model to process the data collected by optical sensors and synthetic aperture radar sensors respectively, so that each data stream can focus on the image features unique to each sensor, meeting the demand for realistic reproduction of large-scale natural landscapes; morphological operations are used to optimize the ground mask, firstly roughly classify the terrain area to be tested, and then a method combining patch segmentation and feature fusion is used to subdivide the crop types on the surface according to multispectral images and aperture radar data, thereby realizing the natural process of dynamic simulation of vegetation crops changing over time on the constructed digital terrain model.

Description

Technical Field

[0001] The invention relates to the technical field of terrain modeling, and in particular to a method for generating large-scale dynamic terrain based on multi-source structural data. Background Art

[0002] The large-scale dynamic terrain generation method based on multi-source structural data refers to a method that combines optical sensors and synthetic aperture radar sensors carried on satellites or aircraft to obtain multi-source structural data and generate large-scale fine terrain models. Game development, film and television production, geographic simulation and other fields all require high-precision large-scale terrain model generation technology. Large-scale dynamic terrain generation technology aims to efficiently create a terrain model with rich details and a wide range.

[0003] Among the existing approximate solutions, for example, CN113506370B, a method and device for constructing a three-dimensional geographic scene model based on stereo remote sensing images, focuses on establishing a three-dimensional terrain model using irregular triangulated network construction technology to ensure modeling accuracy and realism. However, it only considers the topographic texture of the terrain, but does not consider the type details of the covered vegetation landscape, resulting in a low degree of simulation fidelity for the terrain area covered with vegetation on the surface.

[0004] Traditional terrain generation technology is limited by the adverse effects of weather clouds on optical images and the high complexity of cloud removal algorithms. It is often difficult to cover large areas while maintaining high resolution, and it is difficult to achieve natural processes such as dynamic simulation of vegetation and crops changing over time. Summary of the invention

[0005] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a large-scale dynamic terrain generation method based on multi-source structural data; in view of the fact that the traditional terrain generation technology only considers the landform texture of the terrain, but does not consider the details of the types of covered vegetation landscape, and the simulation fidelity of the terrain area covered with vegetation on the surface is low, this solution uses a dual-stream optimized U-Net model to process the data collected by the optical sensor and the synthetic aperture radar sensor respectively, so that each data stream can focus on the image features unique to each sensor, thereby more effectively extracting spatial information to meet the demand for realistic reproduction of large-scale natural landscapes; in view of the fact that the traditional terrain generation technology is limited by the weather and cloud layers on the optical image Due to the adverse effects and high complexity of cloud removal algorithms, it is often difficult to cover large areas while maintaining high resolution, and it is difficult to realize the natural process of dynamic simulation of vegetation and crops changing over time. This scheme proposes a large-scale dynamic terrain generation method based on multi-source structural data, which screens and retains multi-source structural data of weather with cloud cover less than the cloud cover threshold during the window period, optimizes the ground mask using morphological operations, first roughly classifies the terrain area to be tested, and then uses a combination of patch segmentation and feature fusion to subdivide the crop types on the surface according to multispectral images and aperture radar data, thereby realizing the natural process of dynamic simulation of vegetation and crops changing over time on the constructed digital terrain model.

[0006] The technical solution adopted by the present invention is as follows: The present invention provides a method for generating large-scale dynamic terrain based on multi-source structural data, the method comprising the following steps:

[0007] Step S1: data sampling, specifically, using an optical sensor and a synthetic aperture radar sensor to respectively collect multispectral images and aperture radar data of the terrain area to be measured, recording the multispectral images and aperture radar data as multi-source structure data, presetting a cloud cover threshold, using a cloud detection algorithm to identify the cloud cover in the multi-source structure data, and screening and retaining the multi-source structure data of weather with a cloud cover less than the cloud cover threshold within a window period;

[0008] Step S2: pixel classification;

[0009] Step S3: band screening, specifically, screening and retaining the complementary spectral bands in the multispectral image and the aperture radar data, and using the multispectral image and the aperture radar data with complementary spectral bands as new multispectral image and aperture radar data; normalizing the multispectral image; filtering the aperture radar data, presetting a preset resolution and a preset geographic coordinate system, resampling the aperture radar data using the nearest neighbor method, unifying the resolution of the aperture radar data to the preset resolution, and reprojecting the aperture radar data to the preset geographic coordinate system;

[0010] Step S4: spectral fusion;

[0011] Step S5: feature fusion, using the K-nearest neighbor algorithm to segment the terrain area to be measured and roughly classify it into surface category cluster B, wherein the surface category cluster B specifically includes ground, vegetation and water body, and the surface crop type is finely classified for the terrain area to be measured where the surface category cluster B is vegetation;

[0012] Step S6: Model generation, specifically, using spatial interpolation technology to extract a digital terrain model from the aperture radar data, combining the surface category cluster B and the crop type, and drawing a fine terrain model of the terrain area to be measured based on the digital terrain model.

[0013] Furthermore, in step S2, the pixel classification specifically includes the following steps:

[0014] Step S21: Calculate the vegetation index of the terrain area to be measured;

[0015] Step S22: pixel clustering, specifically, using the spectral bands of the multispectral image as clustering features, using principal component analysis to reduce the dimension of the clustering features, constructing a Gaussian mixture model, inputting the reduced-dimensional clustering features into the Gaussian mixture model, and performing unsupervised clustering of the multispectral image with the Gaussian mixture model, grouping the pixels into surface category cluster A, and calculating the clustering probability that the pixels of the multispectral image belong to each surface category cluster A. The clustering probability calculation formula is as follows:

[0016] ;

[0017] In the formula, represents the joint conditional probability, Represents a given set of parameters, Indicates the number of pixels, represents the number of Gaussian distributions, Indicates The weights of the Gaussian components, Indicates The cluster feature vector of pixels, Indicates The probability density function of a Gaussian distribution is represents the mean, represents the covariance matrix;

[0018] Step S23: extracting a ground mask, specifically, extracting a ground mask from the surface category cluster A according to the vegetation index;

[0019] Step S24: optimizing the ground mask, specifically, considering pixels with a clustering probability less than λ as high uncertainty pixels, and removing the high uncertainty pixels from the ground mask;

[0020] Step S25: Remove sparsely distributed vegetation pixels to avoid large errors in areas with drastic terrain changes and low sampling density. Specifically, for areas with drastic terrain changes and low sampling density, skip the morphological operation; for other areas without drastic terrain changes and low sampling density, remove the sparsely distributed pixels in the ground mask that belong to the surface category cluster A with vegetation through the corrosion operation in the morphological operation.

[0021] Furthermore, in step S4, the spectrum fusion specifically includes the following steps:

[0022] Step S41: patch segmentation, specifically, using a sliding window method to segment all multispectral images and aperture radar data into non-overlapping patches of unequal sizes;

[0023] Step S42: patch fusion, specifically, for each patch, using a cloud detection algorithm, select the multispectral image and aperture radar data with the least cloud coverage, and fuse them separately to form a multi-temporal spectral complex and a multi-temporal radar complex respectively;

[0024] The multi-temporal spectral complex refers to an image obtained by fusing multispectral images acquired at different times for the same terrain area to be measured;

[0025] The multi-temporal radar complex refers to an image obtained by fusing aperture radar data acquired at different times in the same terrain area to be measured.

[0026] Furthermore, in step S5, the feature fusion specifically includes the following steps:

[0027] Step S51: optimizing a U-Net model, wherein the U-Net model is composed of a spectrum branch and a radar branch, each branch including a contraction path and an expansion path;

[0028] The contraction path is used to gradually reduce the size of the input data, extract deeper features, and double the number of feature maps. The contraction path includes three downsamplings, and each downsampling sequentially performs two consecutive 3D convolution operations, batch normalization, and nonlinear processing on the input data to obtain high-resolution features;

[0029] The expansion path is used to gradually restore the spatial resolution of the input data. The expansion path includes three upsamplings, each upsampling is performed through 3D inverse convolution, the weights of the high-resolution features in the contraction path are adjusted through the attention mechanism, and the high-resolution features in the contraction path are spliced ​​with the features in the expansion path using a jump connection structure to fuse the high-resolution features of the contraction path;

[0030] The spectral branch of the U-Net model processes multi-temporal spectral complexes, and the radar branch processes multi-temporal radar complexes;

[0031] Step S52: channel stitching, specifically, stitching the outputs of the two branches along the channel dimension to obtain a 3D feature map, the 3D feature map is reshaped into a two-dimensional array through two consecutive 3D convolutions, and the two-dimensional array is mapped to the crop type through 2D convolution.

[0032] The present invention provides a method for generating large-scale dynamic terrain based on multi-source structural data. The beneficial effects achieved by the present invention using the above scheme are as follows:

[0033] (1) Traditional terrain generation technology focuses on building a three-dimensional terrain model using irregular triangulated network construction technology, which only considers the topographic texture of the terrain and does not take into account the details of the types of vegetation landscape. As a result, the simulation fidelity of the terrain area covered with vegetation is low. This solution uses a two-stream optimized U-Net model to process the data collected by the optical sensor and the synthetic aperture radar sensor respectively, so that each data stream can focus on the image features unique to each sensor, thereby more effectively extracting spatial information to meet the demand for realistic reproduction of large-scale natural landscapes.

[0034] (2) Traditional terrain generation technology is limited by the adverse effects of weather clouds on optical images and the high complexity of cloud removal algorithms. It is often difficult to cover large areas while maintaining high resolution, and it is difficult to realize the natural process of dynamic simulation of vegetation and crops changing over time. This scheme proposes a large-scale dynamic terrain generation method based on multi-source structural data. The multi-source structural data of weather with cloud cover less than the cloud cover threshold during the window period are screened and retained. The ground mask is optimized using morphological operations. The terrain area to be tested is first roughly classified. Then, a combination of patch segmentation and feature fusion is used to subdivide the crop types on the surface according to multispectral images and aperture radar data, thereby realizing the natural process of dynamic simulation of vegetation and crops changing over time on the constructed digital terrain model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic flow chart of a method for generating large-scale dynamic terrain based on multi-source structural data provided by the present invention;

[0036] Figure 2 is a schematic diagram of step S2;

[0037] Figure 3 is a schematic diagram of step S4;

[0038] Figure 4 is a schematic diagram of step S5.

[0039] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0042] Example 1, see Figures 1 to 4 The present invention provides a method for generating large-scale dynamic terrain based on multi-source structural data, the method comprising the following steps:

[0043] Step S1: data sampling, specifically, using an optical sensor and a synthetic aperture radar sensor to respectively collect multispectral images and aperture radar data of the terrain area to be measured, recording the multispectral images and aperture radar data as multi-source structure data, presetting a cloud cover threshold, using a cloud detection algorithm to identify the cloud cover in the multi-source structure data, and screening and retaining the multi-source structure data of weather with a cloud cover less than the cloud cover threshold within a window period;

[0044] Step S2: pixel classification;

[0045] Step S3: band screening, specifically, screening and retaining the complementary spectral bands in the multispectral image and aperture radar data, and using the multispectral image and aperture radar data with complementary spectral bands as new multispectral image and aperture radar data;

[0046] Normalize multispectral images;

[0047] Performing filtering processing on the aperture radar data, presetting a preset resolution and a preset geographic coordinate system, resampling the aperture radar data using a nearest neighbor method, unifying the resolution of the aperture radar data to the preset resolution, and reprojecting the aperture radar data to the preset geographic coordinate system;

[0048] Step S4: spectral fusion;

[0049] Step S5: feature fusion, using the K-nearest neighbor algorithm to segment the terrain area to be measured and roughly classify it into surface category cluster B, wherein the surface category cluster B specifically includes ground, vegetation and water body, and the surface crop type is finely classified for the terrain area to be measured where the surface category cluster B is vegetation;

[0050] Step S6: Model generation, specifically, using spatial interpolation technology to extract a digital terrain model from the aperture radar data, combining the surface category cluster B and the crop type, and drawing a fine terrain model of the terrain area to be measured based on the digital terrain model.

[0051] Example 2, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S2, the pixel classification specifically includes the following steps:

[0052] Step S21: Calculate the vegetation index of the terrain area to be measured;

[0053] Step S22: pixel clustering, specifically, using the spectral bands of the multispectral image as clustering features, using principal component analysis to reduce the dimension of the clustering features, constructing a Gaussian mixture model, inputting the reduced-dimensional clustering features into the Gaussian mixture model, and performing unsupervised clustering of the multispectral image with the Gaussian mixture model, grouping the pixels into surface category cluster A, and calculating the clustering probability that the pixels of the multispectral image belong to each surface category cluster A. The clustering probability calculation formula is as follows:

[0054] ;

[0055] In the formula, represents the joint conditional probability, Represents a given set of parameters, Indicates the number of pixels, represents the number of Gaussian distributions, Indicates The weights of the Gaussian components, Indicates The cluster feature vector of pixels, Indicates The probability density function of a Gaussian distribution is represents the mean, represents the covariance matrix;

[0056] Step S23: extracting a ground mask, specifically, extracting a ground mask from the surface category cluster A according to the vegetation index;

[0057] Step S24: optimizing the ground mask, specifically, considering pixels with a clustering probability less than λ as high uncertainty pixels, and removing the high uncertainty pixels from the ground mask;

[0058] Step S25: Remove sparsely distributed vegetation pixels to avoid large errors in areas with drastic terrain changes and low sampling density. Specifically, for areas with drastic terrain changes and low sampling density, skip the morphological operation; for other areas without drastic terrain changes and low sampling density, remove the sparsely distributed pixels in the ground mask that belong to the surface category cluster A with vegetation through the corrosion operation in the morphological operation.

[0059] Example 3, see Figures 1 to 3 This embodiment is based on the above embodiment. In step S4, the spectrum fusion specifically includes the following steps:

[0060] Step S41: patch segmentation, specifically, using a sliding window method to segment all multispectral images and aperture radar data into non-overlapping patches of unequal sizes;

[0061] Step S42: patch fusion, specifically, for each patch, a cloud detection algorithm is used to select the multispectral image and aperture radar data with the least cloud coverage for fusion, respectively, to form a multi-temporal spectral complex and a multi-temporal radar complex;

[0062] The multi-temporal spectral complex refers to an image obtained by fusing multispectral images acquired at different times for the same terrain area to be measured;

[0063] The multi-temporal radar complex refers to an image obtained by fusing aperture radar data acquired at different times in the same terrain area to be measured.

[0064] Example 4, see Figures 1 to 4 This embodiment is based on the above embodiment. In step S5, the feature fusion specifically includes the following steps:

[0065] Step S51: Optimizing a U-Net model, wherein the U-Net model consists of two branches, each branch including a contraction path and an expansion path;

[0066] The contraction path is used to gradually reduce the size of the input data, extract deeper features, and double the number of feature maps. The contraction path includes three downsamplings, and each downsampling sequentially performs two consecutive 3D convolution operations, batch normalization, and nonlinear processing on the input data to obtain high-resolution features;

[0067] The expansion path is used to gradually restore the spatial resolution of the input data. The expansion path includes three upsamplings. Each upsampling is performed through 3D inverse convolution. A jump connection is used to splice the high-resolution features in the contraction path with the features in the expansion path to fuse the high-resolution features of the contraction path.

[0068] One branch of the U-Net model processes the multi-temporal spectral complex, and the other branch processes the multi-temporal radar complex;

[0069] Step S52: channel splicing, splicing the outputs of the two branches along the channel dimension to obtain a 3D feature map, the 3D feature map is reshaped into a two-dimensional array through two consecutive 3D convolutions, and the two-dimensional array is mapped to the crop type through 2D convolution.

[0070] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S3, the complementary spectral bands are specifically blue light, green light, red light, near infrared (NIR) and short wave infrared (SWIR1, SWIR2) bands.

[0071] Example 6, see Figure 1 This embodiment is based on the above embodiment. In step S3, the filtering process is specifically filtering using a gamma map algorithm with a window size of 3×3 pixels.

[0072] Embodiment 7, referring to the figure, this embodiment is based on the above embodiment. In step S3, the preset geographic coordinate system is specifically the WGS84 coordinate system.

[0073] Embodiment 8, see Figures 1 to 4 This embodiment is based on the above embodiment. In step S51, the kernel size of the 3D convolution operation is 3×3×3.

[0074] Embodiment 9, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S21, the vegetation index is specifically the normalized difference vegetation index (NDVI), the improved soil adjusted vegetation index (MSAVI) and the normalized integral water index (NDWI).

[0075] Embodiment 10, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S23, a ground mask is extracted from the surface category cluster according to the vegetation index. Specifically, the surface category cluster with the lowest reflected vegetation index and negative normalized integrated water index is identified as the ground mask.

[0076] Example 11, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S24, λ=0.8.

[0077] Example 12, see Figures 1 to 4 This embodiment is based on the above embodiment. In step S6, the spatial interpolation technology is the natural neighbor method of triangulation.

[0078] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0079] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0080] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for generating large-scale dynamic terrain based on multi-source structural data, characterized by: The method comprises the following steps: Step S1: data sampling, specifically, using an optical sensor and a synthetic aperture radar sensor to respectively collect multispectral images and aperture radar data of the terrain area to be measured, recording the multispectral images and aperture radar data as multi-source structure data, presetting a cloud cover threshold, using a cloud detection algorithm to identify the cloud cover in the multi-source structure data, and screening and retaining the multi-source structure data of weather with a cloud cover less than the cloud cover threshold within a window period; Step S2: pixel classification; Step S3: band screening, specifically, screening and retaining the complementary spectral bands in the multispectral image and the aperture radar data, and using the multispectral image and the aperture radar data with complementary spectral bands as new multispectral image and aperture radar data; normalizing the multispectral image; filtering the aperture radar data, presetting a preset resolution and a preset geographic coordinate system, resampling the aperture radar data using the nearest neighbor method, unifying the resolution of the aperture radar data to the preset resolution, and reprojecting the aperture radar data to the preset geographic coordinate system; Step S4: spectral fusion; Step S5: feature fusion, using the K-nearest neighbor algorithm to segment the terrain area to be measured and roughly classify it into surface category cluster B, wherein the surface category cluster B specifically includes ground, vegetation and water body, and the surface crop type is finely classified for the terrain area to be measured where the surface category cluster B is vegetation; Step S6: Model generation, specifically, using spatial interpolation technology to extract a digital terrain model from the aperture radar data, combining the surface category cluster B and the crop type, and drawing a fine terrain model of the terrain area to be measured based on the digital terrain model; In step S2, the pixel classification specifically includes the following steps: Step S21: Calculate the vegetation index of the terrain area to be measured; Step S22: pixel clustering, specifically, using the spectral bands of the multispectral image as clustering features, using principal component analysis to reduce the dimension of the clustering features, constructing a Gaussian mixture model, inputting the reduced-dimensional clustering features into the Gaussian mixture model, and performing unsupervised clustering of the multispectral image with the Gaussian mixture model, grouping the pixels into surface category cluster A, and calculating the clustering probability that the pixels of the multispectral image belong to each surface category cluster A. The clustering probability calculation formula is as follows: ； In the formula, represents the joint conditional probability, Represents a given set of parameters, Indicates the number of pixels, represents the number of Gaussian distributions, Indicates The weights of the Gaussian components, Indicates The cluster feature vector of pixels, Indicates The probability density function of a Gaussian distribution is represents the mean, represents the covariance matrix; Step S23: extracting a ground mask, specifically, extracting a ground mask from the surface category cluster A according to the vegetation index; Step S24: optimizing the ground mask, specifically, considering pixels with a clustering probability less than λ as high uncertainty pixels, and removing the high uncertainty pixels from the ground mask; Step S25: removing sparsely distributed vegetation pixels to avoid large errors in areas with dramatic terrain changes and low sampling density. Specifically, for areas with dramatic terrain changes and low sampling density, the morphological operation is skipped, and for other areas without dramatic terrain changes and low sampling density, the sparsely distributed pixels belonging to the surface category cluster A of vegetation in the ground mask are removed through the corrosion operation in the morphological operation; In step S4, the spectrum fusion specifically includes the following steps: Step S41: patch segmentation, specifically, using a sliding window method to segment all multispectral images and aperture radar data into non-overlapping patches of unequal sizes; Step S42: patch fusion, specifically, for each patch, using a cloud detection algorithm, select the multispectral image and aperture radar data with the least cloud coverage, and fuse them separately to form a multi-temporal spectral complex and a multi-temporal radar complex respectively; The multi-temporal spectral complex refers to an image obtained by fusing multispectral images acquired at different times for the same terrain area to be measured; The multi-temporal radar complex refers to an image obtained by fusing aperture radar data acquired at different times in the same terrain area to be measured.

2. The method for generating large-scale dynamic terrain based on multi-source structural data according to claim 1, characterized in that: In step S5, the feature fusion specifically includes the following steps: Step S51: optimizing a U-Net model, wherein the U-Net model is composed of a spectrum branch and a radar branch, each branch including a contraction path and an expansion path; The contraction path is used to gradually reduce the size of the input data, extract deeper features, and double the number of feature maps. The contraction path includes three downsamplings, and each downsampling sequentially performs two consecutive 3D convolution operations, batch normalization, and nonlinear processing on the input data to obtain high-resolution features; The expansion path is used to gradually restore the spatial resolution of the input data. The expansion path includes three upsamplings, each upsampling is performed through 3D inverse convolution, the weights of the high-resolution features in the contraction path are adjusted through the attention mechanism, and the high-resolution features in the contraction path are spliced ​​with the features in the expansion path using a jump connection structure to fuse the high-resolution features of the contraction path; The spectral branch of the U-Net model processes multi-temporal spectral complexes, and the radar branch processes multi-temporal radar complexes; Step S52: channel stitching, specifically, stitching the outputs of the two branches along the channel dimension to obtain a 3D feature map, the 3D feature map is reshaped into a two-dimensional array through two consecutive 3D convolutions, and the two-dimensional array is mapped to the crop type through 2D convolution.

Images