Building foot point recognition and segmentation method based on two-stage 3D point cloud semantic segmentation

Abstract

The application discloses a building foot point recognition and segmentation method based on two-stage 3D point cloud semantic segmentation, and comprises the following steps: acquiring three-dimensional point cloud data of a target scene; performing point cloud ground filtering processing by using cloth simulation ground filtering (CSF) algorithm, separating ground point cloud from object point cloud in the original point cloud, and removing vegetation points by combining r radius search judgment neighborhood points and echo times after acquiring non-ground points; setting a threshold value to quickly pay attention to Euclidean clustering, obtaining point cloud clusters to which each point belongs, and obtaining high-quality candidate point cloud clusters; preprocessing the point cloud clusters, training an improved PointNet++ semantic segmentation neural network model by using a three-dimensional point cloud training set; inputting the to-be-tested point cloud into the trained improved PointNet++ semantic segmentation neural network model, and obtaining a segmentation result of building foot point point cloud. The application effectively solves the problems of current building reconstruction, such as many noise points, low efficiency of traditional algorithms, low precision of deep network structure, and training cost.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional point cloud data processing, and particularly relates to a building foot point recognition and segmentation method based on two-stage 3D point cloud semantic segmentation. BACKGROUND

[0002] Three-dimensional reconstruction of buildings is an important research topic in the fields of computer vision, photogrammetry and remote sensing. With the rapid development of Light Detection and Ranging (LIDAR) systems, it is possible to quickly and accurately obtain large-scale urban three-dimensional point clouds, which greatly facilitates the reconstruction of three-dimensional buildings. Three-dimensional point cloud is the most common and basic three-dimensional model in spatial information expression. Three-dimensional point cloud is mainly used to represent data in three-dimensional space. Compared with two-dimensional planar graphics, three-dimensional point cloud has depth direction information and can naturally decouple the target and background in the depth direction. In addition, for three-dimensional visual measurement, three-dimensional point cloud is the main means of support. By extracting the target three-dimensional point cloud, the actual physical geometric information of the target in space can be obtained.

[0003] 3D point cloud segmentation is the process of classifying point clouds into different regions. Points in the same region have similar attributes. Due to high redundancy, uneven sampling density and lack of explicit structure in point cloud data, 3D point cloud segmentation is a challenging task. Point cloud segmentation helps analyze scenes in various applications, such as locating and identifying objects, classifying and function extraction.

[0004] Before the rise of deep learning technology, semantic segmentation of three-dimensional point clouds mainly relied on manually designed features. Since 2006, the rapid development of deep learning has enabled it to automatically learn features based on large amounts of training data, making the semantic segmentation technology of 3D point clouds rapidly develop.

[0005] In recent years, with the emergence of large-scale data sets, the reduction of computer hardware costs and the improvement of GPU parallel computing capabilities, deep neural networks (DCNNs) have been more widely used. Unlike traditional handcrafted features, DCNNs can automatically learn rich feature representations from data, so they perform well in many computer vision problems such as semantic segmentation.

[0006] Fully Convolutional Networks (FCNs) in DCNNs is a kind of DCNNs, which is particularly outstanding in feature extraction. For the scene semantic segmentation task, the global context information between different class labels affects its accurate positioning. However, FCNs do not have the ability to model the context relationship between different class labels due to its fully convolutional characteristics; the semantic segmentation results obtained by the convolution kernel with a large receptive field are relatively rough; at the same time, the hierarchical features generated by the pooling layer also lose part of the positioning information, further reducing the possibility of outputting fine-grained semantic segmentation results.

[0007] PointNet model uses MaxPooling as a symmetric function to handle the unordered nature of point cloud models, and uses two T-networks to process the rotational invariance of the model. The disadvantage of this model is that only one MaxPooling layer is used to integrate single-point features, and the network lacks the ability to extract local information from the model.

[0008] SE only considers channel attention and ignores spatial attention. BAM and CBAM consider both channel attention and spatial attention, but still have two most important shortcomings: (1) no capture of spatial information at different scales to enrich the feature space. (2) Spatial attention only considers local region information, and cannot establish long-distance dependence. Subsequent PyConv, Res2Net and HS-ResNet are used to solve these two shortcomings of CBAM, but the amount of calculation is too large.

[0009] In addition, most of the current 3D Lidar point cloud segmentation methods are to project 3D point cloud to 2D space, and then use 2D convolutional neural network for segmentation processing, including two methods based on distance image and BEV image. But these methods will inevitably lose accurate 3D geometric information in the projection process from 3D to 2D, which will affect the final accuracy.

[0010] The front building reconstruction has the problems of many noise points, low efficiency of traditional algorithm, low precision of deep network structure and high training cost. Due to the noise of the obtained point cloud data, the existing three-dimensional point cloud segmentation method is not accurate and unstable. SUMMARY

[0011] To solve the above problems, the application makes full use of the advantages of point cloud, combines the efficiency of traditional geometric method and the robustness of deep learning method, and proposes a two-stage 3D point cloud semantic segmentation framework combining building geometric prior knowledge and attention mechanism to identify and segment the building foot point, effectively solving the problems of current building reconstruction noise, low efficiency of traditional algorithm, low precision of deep network structure and training cost.

[0012] The specific scheme is as follows:

[0013] The building foot point identification and segmentation method based on the two-stage 3D point cloud semantic segmentation comprises the following steps:

[0014] 1) Obtain three-dimensional point cloud data of a target scene;

[0015] 2) Use cloth simulation filter (CSF) algorithm to perform point cloud ground filtering, separate ground point cloud from ground object point cloud in the original point cloud, and remove vegetation points by combining r radius search judgment neighborhood points and echo times after obtaining non-ground points;

[0016] 3) Set threshold fast attention Euclidean clustering, cluster point clouds without ground into several parts, obtain the point cloud cluster to which each point belongs, and optimize the candidate point cloud cluster to obtain high-quality candidate point cloud cluster in a very short time;

[0017] 4) Preprocess the point cloud cluster, train the improved PointNet++ semantic segmentation neural network model using a three-dimensional point cloud training set, the training sample is a target three-dimensional point cloud data set, and the point cloud segmentation network is trained to obtain accurate semantic labels for each point, and the label is a real semantic category;

[0018] 5) Input the to-be-tested point cloud into the trained improved PointNet++ semantic segmentation neural network model to obtain the segmentation result of the building foot point point cloud.

[0019] As a further improvement of the application, the pyramid feature attention network (PFA) is added when performing Euclidean clustering in step 3), which can model long-distance regional correlation, establish spatial topological relationship, extract global features and point features of point cloud, and focus attention on feature channels with large information amount, suppress unimportant channel features, and greatly reduce the calculation amount and time consumption of subsequent point cloud processing.

[0020] As a further improvement of the application, the improved PointNet++ semantic segmentation model constructed in step 4) is a semantic segmentation model based on a graph stochastic attention mechanism (GSAT), wherein the graph stochastic attention mechanism is added to the PointNet++ semantic segmentation model, which prevents information irrelevant to the label by injecting randomness in the learning of attention, and selects a subgraph related to the label by reducing randomness, so as to achieve guaranteed explainability and generalization ability, thereby improving the semantic segmentation precision.

[0021] The application has the following beneficial effects:

[0022] (1) A two-stage 3D point cloud semantic segmentation framework fusing building geometric prior knowledge and attention mechanism is proposed to identify and segment building foot points.

[0023] (2) A threshold fast attention mechanism Euclidean clustering is set to obtain high-quality candidate point cloud clusters in a very short time, greatly reducing the calculation amount and time consumption of subsequent point cloud processing.

[0024] (3) PointNet++ is used as the backbone, and a graph stochastic attention mechanism is added to the PointNet++ semantic segmentation model to achieve guaranteed explainability and generalization ability, thereby improving the semantic segmentation precision. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 It is a schematic diagram of the two-stage 3D point cloud semantic segmentation method.

[0026] Figure 2 It is a pyramid feature attention network structure diagram.

[0027] Figure 3 It is an improved PointNet++ network architecture diagram.

[0028] Figure 4 It is a graph stochastic attention GSAT network structure diagram. DETAILED DESCRIPTION

[0029] The application will be further illustrated in conjunction with the drawings and specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the application and not to limit the scope of the application.

[0030] As shown in the figure, the application discloses a building foot point identification and segmentation method based on two-stage 3D point cloud semantic segmentation, which solves the problems of current building reconstruction noise, low efficiency of traditional algorithms, low precision of deep network structure, and training cost. The method comprises the following steps:

[0031] 1) Acquire 3D point cloud data of the target scene;

[0032] 2) The Cloth Simulation Filter (CSF) algorithm is used to perform point cloud ground filtering to separate the ground point cloud from the ground feature point cloud from the original point cloud; after obtaining non-ground points, the neighboring points are judged by the radius of r and the echo count to remove vegetation points.

[0033] 3) Set a threshold for fast attention Euclidean clustering to group point clouds without ground into several parts, obtain the point cloud cluster to which each point belongs, optimize the candidate point cloud clusters, and obtain high-quality candidate point cloud clusters in a very short time.

[0034] 4) Preprocess the point cloud clusters, train the improved PointNet++ semantic segmentation neural network model using the 3D point cloud training set, the training samples are the target 3D point cloud dataset, train the point cloud segmentation network to obtain the accurate semantic label of each point, and the label is the real semantic category;

[0035] 5) Input the point cloud to be tested into the trained improved PointNet++ semantic segmentation neural network model to obtain the segmentation result of the building foot point cloud.

[0036] In this embodiment, when performing Euclidean clustering in step 3), a Pyramid Feature Attention Network (PFA) is added. This network can model long-distance regional correlations and establish spatial topological relationships. It is used to extract global and point features of the point cloud and focuses attention on feature channels with high information content, suppressing unimportant channel features, which greatly reduces the computational load and time consumption of subsequent point cloud processing.

[0037] In this embodiment, the improved PointNet++ semantic segmentation model constructed in step 4) is a semantic segmentation model based on the Graph Stochastic Attention (GSAT) mechanism. On the basis of the PointNet++ semantic segmentation model, the GSAT mechanism is added. It prevents information unrelated to the label by injecting randomness into the learning of attention, and uses the reduction of randomness to select subgraphs related to the label, so as to achieve guaranteed interpretability and generalization ability, thereby improving the semantic segmentation accuracy.

[0038] The technical means disclosed in the present application are not limited to the technical means disclosed in the above embodiments, and include technical solutions composed of any combination of the above technical features. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered within the scope of protection of the present application.

Claims

1. A building foot point recognition and segmentation method based on two-stage 3D point cloud semantic segmentation, characterized in that, The method comprises the following steps: 1) obtaining three-dimensional point cloud data of a target scene; 2) using cloth simulation ground filtering (CSF) algorithm to perform ground filtering on the point cloud data, separating ground point cloud from non-ground point cloud; after obtaining the non-ground point cloud, combining r radius search to judge the neighborhood points and the echo times to remove the vegetation points; 3) setting a threshold to perform fast attention Euclidean clustering, clustering the point cloud without ground into several parts to obtain the point cloud cluster to which each point belongs, optimizing the candidate point cloud cluster to obtain the candidate point cloud cluster; 4) preprocessing the point cloud cluster, training an improved PointNet++ semantic segmentation neural network model using a three-dimensional point cloud training set, the training sample being a target three-dimensional point cloud data set, training the point cloud segmentation network to obtain an accurate semantic label for each point, the label being a real semantic category; 5) inputting the to-be-tested point cloud into the trained improved PointNet++ semantic segmentation neural network model to obtain a segmentation result of the building foot point cloud.

2. The building footpoint recognition and segmentation method based on two-stage 3D point cloud semantic segmentation according to claim 1, characterized in that, In the step 3), a pyramid feature attention (PFA) network is added to the Euclidean clustering to model the regional correlation of a long distance, establish a spatial topological relationship, extract global features and point features of the point cloud, and focus attention on the feature channels with large information amount and suppress the channel features that are not important, thereby reducing the calculation amount and time consumption of subsequent point cloud processing.

3. The building footpoint recognition and segmentation method based on two-stage 3D point cloud semantic segmentation according to claim 1, characterized in that, In the step 4), the improved PointNet++ semantic segmentation model is a graph stochastic attention (GSAT) based semantic segmentation model, which adds a graph stochastic attention mechanism to the PointNet++ semantic segmentation model, injects randomness into the learning of attention to prevent information irrelevant to the label, and uses the reduction of randomness to select subgraphs related to the label, so as to achieve guaranteed explainability and generalization ability, thereby improving the semantic segmentation accuracy.

Images