A method for constructing a building LOD1 model based on Gaofen-7 satellite images

Abstract

In view of the difficulty in constructing a large-scale LOD1 model of buildings and the high cost of extracting three-dimensional information of buildings, the present invention discloses a method for constructing a building LOD1 model based on Gaofen-7 satellite images, which is used to extract three-dimensional information of large-scale high-resolution buildings and complete the construction of the building LOD1 model. The main steps of the method are: step 1) extracting the roof of a building based on a deep learning method; step 2) constructing a three-dimensional point cloud of the city based on multi-view satellite images; step 3) filtering the point cloud of the three-dimensional point cloud of the city, and constructing a normalized digital surface model; step 4) superimposing the normalized digital surface model with the extraction result of the building roof, and performing post-processing operations on the superimposed result to complete the extraction of the building height; step 5) stretching the building roof according to the building height information to generate the building LOD1 model.

Description

Technical Field

[0001] The present invention relates to a method for constructing a building LOD1 model based on GF-7 multi-view satellite images. The method extracts high-resolution three-dimensional information of large-scale urban buildings according to the GF-7 multi-spectral images and multi-view panchromatic images, and constructs a building LOD1 model according to the extraction results of the three-dimensional information of the buildings. Background Art

[0002] In recent years, with the continuous development of domestic multi-view satellites, new ideas and methods have been provided for the construction of LOD1 models of urban buildings. As my country's first civilian sub-meter stereo mapping satellite, Gaofen-7 can provide multi-view and multi-spectral satellite images, mainly serving many application needs such as natural resource surveys, basic mapping, and urban three-dimensional building model generation. The building LOD1 model depicts the horizontal and vertical morphological characteristics and distribution patterns of the city, and can provide a reference for urban planning and green city construction. The construction of the building LOD1 model requires three-dimensional information of the building as a basis. Traditional three-dimensional building information extraction based on optical remote sensing images generally uses machine learning methods to extract building and building shadow information, and then uses building shadows to carry out building height estimation research, but this method is easily subject to many limitations such as building height, shadow effect, and viewing angle, and the accuracy of building height estimation depends on the shadow extraction effect. This patent uses a deep learning method to extract building roofs from the GF-7 multispectral satellite images, constructs a three-dimensional point cloud of the ground objects and a normalized digital surface model through multispectral images; combines the building boundary extraction results with the normalized digital surface model to generate a three-dimensional building information product, and finally obtains the building LOD1 model. Summary of the invention

[0003] The Gaofen-7 satellite is equipped with a 0.65m resolution rear-view panchromatic camera, a 2.6m resolution rear-view multispectral camera (camera inclination angle is -5°) and a 0.8m forward-view panchromatic camera (camera inclination angle is +26°); this patent uses a deep learning method to extract building roofs from the Gaofen-7 multispectral satellite images, constructs a three-dimensional point cloud of the ground objects based on the push-broom multispectral images, and then obtains a normalized digital surface model through operations such as point cloud filtering; the building boundary extraction result is superimposed on the normalized digital surface model to generate the three-dimensional information of the building, and finally the building LOD1 model is obtained.

[0004] Step 1) extracting building roofs based on deep learning methods;

[0005] Step 2) constructing a 3D point cloud of the city based on multi-view satellite images;

[0006] Step 3) using a morphological filtering algorithm to filter the urban 3D point cloud, and classifying the ground points and non-ground points in the urban 3D point cloud; using an inverse distance weighted interpolation method to construct a digital surface model, a digital elevation model, and a normalized digital surface model;

[0007] Step 4) superimposing the normalized digital surface model and the building roof extraction result, and performing post-processing operations on the superimposed result to generate a building height extraction result;

[0008] Step 5) The building roof is stretched according to the building height information to generate the building LOD1 model.

[0009] Further, the specific method of step 1) is:

[0010] To extract building roofs based on deep learning, we first perform full-color fusion of the Gaofen-7 multispectral image and the panchromatic image to obtain the full-color fusion result. Since buildings are complex and changeable, in order to improve the accuracy and efficiency of building extraction, our main work is as follows:

[0011] a) In terms of data sets, due to the lack of building training data sets for Gaofen-7, we first built our own data sets and selected representative areas for building annotation. Then two cropping methods were used for the annotated areas, random scaled cropping and non-overlapping cropping. Random scaled cropping randomly generates cropping seed points and randomly selects the cropping scale size. The cropping scale is kept between 0.5-1.5. For example, when the training image size is 512×512, the cropping size should be 256-768. Then scale the image to a 512×512 image, keep the number of crops the same as the number of non-overlapping crops, and then mix the two cropping methods as training data for the subsequent neural network model training process.

[0012] b) In terms of network structure, we modified the traditional encoding and decoding network framework in combination with the self-attention mechanism. The encoding layer uses ResNet34 as the backbone feature extraction network, and the jump connection part adds channel-dimensional and spatial-dimensional self-attention modules. In addition, to address the problem of excessive channel-dimensional attention calculation, a spatial pyramid pooling module is added to ensure the number of parameters and computing speed while improving training accuracy.

[0013] c) Training and reasoning strategies. The training process adopts a transfer learning strategy, using a public building dataset to complete the initial network weight training, and then uses a self-built dataset for training to complete the migration of the model from the source domain to the target domain, thereby maximizing the accuracy of building extraction. During the reasoning process, the entire image is divided into blocks to ensure that there is a certain degree of overlap between the small blocks. Finally, the training results are spliced, and the overlapping areas are determined by voting.

[0014] Further, the specific method of step 2) is:

[0015] Since the imaging method of multi-view satellite images is push-broom imaging, this imaging method determines that its epipolar lines are in the form of hyperbolas. Therefore, epipolar correction cannot be performed during stereo matching, and the two-dimensional matching problem cannot be reduced to a one-dimensional matching problem. Therefore, we choose to process the original image in blocks, and approximate the image in the block as a pinhole imaging model. When the block size is small enough, it can be ensured that the hyperbolic epipolar lines of push-broom imaging are approximately considered as the straight epipolar lines of the pinhole model. And we choose to use the feature points in each small area to perform affine correction in the small area to eliminate the errors of the satellite image RPC parameters and the errors caused by the above approximation to the greatest extent. After completing stereo matching, we obtained the disparity map of the multi-view satellite image, and converted the disparity map results into three-dimensional coordinates of the ground point through the satellite image RPC parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the deep learning building extraction model used in the present invention;

[0017] Figure 2 Schematic diagram of the attention mechanism adopted by the present invention;

[0018] Figure 3 Schematic diagram of the spatial pyramid pooling module used in the present invention;

[0019] Figure 4 The present invention is a process flow chart of the method. DETAILED DESCRIPTION

[0020] The following is a further explanation of the present invention "A method for constructing a building LOD1 model based on GF-7 satellite images" in conjunction with the accompanying drawings.

[0021] Step 1) Extract building roofs based on deep learning methods

[0022] Our network model is shown in the figure Figure 1 As shown in Figure 2, we modify the traditional encoding and decoding network framework by combining the self-attention mechanism. The encoding layer uses ResNet34 as the backbone feature extraction network, and the jump connection part adds channel dimension and space dimension self-attention modules. The attention module part is as follows Figure 2 As shown in the figure, in order to solve the problem of excessive channel-dimensional attention calculation, a spatial pyramid pooling module is added ( Figure 3 ) is used to improve the training accuracy while ensuring the number of parameters and computing speed.

[0023] a) Spatial Attention Module

[0024] We perform a 1×1 convolution operation on the input feature map of size C×H×W (C represents the number of channels in the feature map, H represents the height of the feature map, and W represents the width) to obtain the Query, Key, and Value branches. After the feature map enters the Q branch, the feature map of size C×H×W is flattened into a two-dimensional vector of size C×N, where N=H×W. The feature map Q is transposed to obtain a feature map Q' of size N×C. Similarly, the feature map enters branch K, and spatial pyramid pooling is performed on the feature map of size C×H×W to achieve dimensionality reduction. The spatial pyramid pooling operation is as follows: Figure 3 As shown in the figure. The spatial pyramid pooling part is to perform the maximum pooling of the input feature map with a window size of n×n to obtain a feature map of size C×n×n, which is used to represent the representative anchor sampling results of each region of the feature map. Then all feature maps after spatial pyramid pooling are flattened and cascaded to obtain a feature vector of length C×S, where S is determined by the size and number of the selected pooling windows. For example, in this article, pooling windows of sizes 1×1, 3×3, 6×6, and 8×8 are selected, then S is equal to

[0025]

[0026] After the feature map X enters the Query and Key branches, the feature vector Q' of size N×C and the feature vector K' of size C×S are matrix-multiplied to obtain the feature map QK'. The feature map QK' is normalized by SoftMax to obtain the attention map QK. The purpose of this process is to calculate the relationship between each pixel in the feature vector Q' and each pixel in K'. In this way, a feature map of size C×S is obtained, which represents the attention relationship between the Query pixel and the feature anchor pixel block in the Key, and represents the long-range dependency relationship between images.

[0027] The Value branch is similar to the Key branch. The feature map X is input into the Value branch to obtain a feature vector V' of size C×S. After transposition, the feature vector V' is matrix-multiplied with the feature map QK' to generate a feature map of size C×H×W. Then, the original feature map X is added to the feature map QKV element by element to generate the result of the spatial attention module.

[0028] B) Channel dimension attention module

[0029] In the building extraction process, each channel of the high-level features can be regarded as a response to the specific characteristics of the building, and different semantic features are interrelated. By extracting the interdependence relationship between channel-dimensional maps, we can emphasize the interdependent feature maps and improve the feature representation of specific semantics. Therefore, we use the channel attention module to model the channel-dimensional interdependence.

[0030] We compute the channel attention map from the original feature map X of size C×H×W. Specifically, we stretch the feature map X into a feature vector of size C×N (N=H×W). We then perform matrix multiplication of the feature vector and the transpose of the feature vector, and apply softmax normalization to obtain a feature attention map of size C×C.

[0031] The channel attention map represents the long-range dependencies between channels in the feature map. After obtaining the channel attention map, we perform a matrix multiplication operation on the input feature map X and the channel attention map, and reshape it to the size of the original feature map. Then, we multiply the result by a learnable scale factor α and sum it pixel by pixel with the original feature map X to obtain a feature map containing feature attention.

[0032] Step 2) constructing a 3D point cloud of the city based on multi-view satellite images;

[0033] Since the imaging mode of multi-view satellite images is push-broom imaging, this imaging mode determines that its epipolar lines are in the form of hyperbolas. In the stereo matching process, epipolar line correction cannot be performed, and the two-dimensional matching problem cannot be reduced to a one-dimensional matching problem. Therefore, the original image is divided into blocks, and the image in the block is approximately regarded as a pinhole imaging model. When the block size is small enough, it can be ensured that the epipolar lines in the form of hyperbolas of push-broom imaging are approximately regarded as the straight epipolar lines of the pinhole model. And we choose to use the matching results of feature points in each small area to perform affine correction in the small area to eliminate the errors of the satellite image RPC parameters and the errors caused by the above approximation to the greatest extent. After completing the stereo matching, we obtained the disparity map of the multi-view satellite image, and converted the disparity map results into three-dimensional coordinates of the ground point through the satellite image RPC parameters. And the outlier removal operation is performed on the generated point cloud. The outlier determination method is to determine whether there are other points in the neighborhood of a certain point.

[0034] Step 3) using a morphological filtering algorithm to filter the urban 3D point cloud, and classifying the ground points and non-ground points in the urban 3D point cloud; using an inverse distance weighted interpolation method to construct a digital surface model, a digital elevation model, and a normalized digital surface model;

[0035] Through experiments, it is found that since the point cloud generated in step 2 is relatively sparse compared to the lidar point cloud, the use of morphological filtering algorithm can obtain better experimental results.

[0036] Step 4) superimposing the normalized digital surface model and the building roof extraction result, and performing post-processing operations on the superimposed result to generate a building height extraction result;

[0037] Since there is a certain amount of noise in the normalized digital surface model, such as errors in the stereo matching process, outliers will appear in the roof point cloud, and they cannot be accurately removed by the post-processing criteria in step 2 during the point cloud generation process, we choose to filter the superimposed results, filter out the local maximum values ​​in the superimposed results, and select the maximum value in the roof area of ​​the building as the building height.

[0038] Step 5) The building roof is stretched according to the building height information to generate the building LOD1 model.

Claims

1. A method for constructing a building LOD1 model based on GF-7 multi-view satellite images, the method comprising the following steps: Step 1) Extract building roofs based on deep learning methods; propose an encoding and decoding network framework combined with a self-attention mechanism, and use ResNet34 as the backbone feature extraction network in the encoding layer; add channel dimension and spatial dimension self-attention modules to the jump connection part; and add a spatial pyramid pooling module to address the problem of excessive channel dimension attention calculation. Step 2) constructing a 3D point cloud of the city based on the GF-7 multi-view satellite image; for the push-broom imaging mode of the GF-7 multi-view satellite image, the block processing principle is adopted to divide the original image into multiple small areas, and affine correction is performed in the small areas using feature points; after completing the stereo matching of the small areas, a disparity map is generated, and the disparity map is converted into the 3D coordinates of the ground points based on the RPC parameters of the satellite image; Step 3) using a morphological filtering algorithm to filter the urban three-dimensional point cloud, and classifying the ground points and non-ground points in the three-dimensional point cloud; using an inverse distance weighted interpolation method to construct a digital surface model, a digital elevation model, and a normalized digital surface model; Step 4) superimposing the normalized digital surface model and the building roof extraction result, and performing a filtering and post-processing operation on the superimposed result to generate a building height extraction result; Step 5) The building roof is stretched according to the building height information to generate the building LOD1 model.

Images