A point cloud target classification method based on an improved PointNet++ neural network

Abstract

The application discloses a point cloud target classification method based on an improved PointNet++ neural network, and comprises the following steps: step 1, preprocessing point cloud data, filtering out noise points and outliers by using a statistical filter, and retaining point cloud clusters containing only a region of interest; step 2, using a clustering segmentation algorithm on the region of interest point cloud clusters obtained in the above step to segment single target point clouds independent of each other; and step 3, inputting the point cloud in step 2 into an improved PointNet++ neural network model for classification, wherein the improved network uses a multilayer perceptron to extract features of the input point cloud by adding a feature extraction layer, continuously extracts local features through two set sampling layers, uses a distance feature function in the set sampling layer to strengthen the features of near points and weaken the interference of far points, finally aggregates all feature information through a maximum pooling layer to obtain global features, and outputs a point cloud classification result.

Description

TECHNICAL FIELD

[0001] The application relates to the field of point cloud target recognition and classification, in particular to a point cloud target classification method based on an improved PointNet++ neural network. BACKGROUND

[0002] In recent years, methods for target recognition and classification based on images have become mature, but images are susceptible to light, and the recognition result will fluctuate greatly due to changes in light conditions. Point clouds are not affected by light and have strong robustness, but they have three characteristics: disorder, sparsity and limited information.

[0003] PointNet++ is a network structure applied to target classification proposed in recent years, which can directly take point clouds as input and directly output the class label of the entire input, and can well process point cloud data in three-dimensional space. However, PointNet++ directly taking point clouds as input does not extract local features in a deeper level, ignores all feature information of the original point cloud, and the effect is not very good for multi-classification problems with multiple instances, which affects the accuracy of multi-classification. SUMMARY

[0004] In order to solve the problems in the background art, the application provides a point cloud target classification method based on an improved PointNet++ neural network. The application first pre-processes the original point cloud, and adds a feature extraction layer on the basis of the original network structure to extract all features of the initially input points. The distance feature function is used to strengthen the features of nearby points and weaken the interference of distant points, so as to improve the problem that the original network does not capture local features well, and effectively improve the classification accuracy.

[0005] The technical scheme of the application is a point cloud target classification method based on an improved PointNet++ neural network, which comprises the following steps:

[0006] Step 1: Pre-process the point cloud data, filter out noise points and outliers by using a statistical filter, and retain point cloud clusters containing only the region of interest;

[0007] Step 2: Use a clustering segmentation algorithm to segment the point cloud clusters in the region of interest obtained in the above step into single target point clouds independent of each other;

[0008] Step 3, input the point cloud in step 2 into the improved PointNet++ neural network model for classification, and the improved network uses a multilayer perceptron to extract features from the input point cloud by adding a feature extraction layer, then continuously extracts local features through two set sampling layers, uses a distance feature function in the set sampling layer to strengthen the features of nearby points and weaken the interference of distant points, finally aggregates all feature information through a max-pooling layer to obtain global features, and outputs the point cloud classification result.

[0009] Further, the step 1 specifically comprises:

[0010] Step 1.1, setting a point cloud set Calculate the average distance d of the point to the nearest neighbor point by K nearest neighbor search i ;

[0011] Step 1.2, calculating the mean value of the point set and the standard deviation Where d i is the average distance of each point to its nearest point, and n is the number of points in the point set;

[0012] Step 1.3, compare all points with the field distance, if the distance is greater than μ+λσ (λ is the multiple of standard deviation) is marked as an outlier, and is removed, otherwise is retained.

[0013] Further, the specific process of the step 2 is:

[0014] Step 2.1, based on the point cloud group data obtained in step 2, use the density-based clustering DBSCAN method with noise for clustering segmentation, set the field ε and the field density threshold MinPoints;

[0015] Step 2.2, scan all data points, if the number of points in a certain data point R radius range is ≧ MinPoints, then it is included in the core point list, and the points directly density of it form the corresponding temporary clustering cluster;

[0016] Step 2.3, for each temporary clustering cluster, check whether the point in it is a core point, if so, merge the temporary clustering cluster corresponding to the point with the current temporary clustering cluster to obtain a new temporary clustering cluster;

[0017] Step 2.4, traverse all core points in the ε field of the core point, repeat the operation of step 2.3, until each point in the current temporary clustering cluster is either not in the core point list or its density directly reaches the point in the temporary clustering cluster, the temporary clustering cluster is upgraded to a clustering cluster;

[0018] Step 2.5, continue to perform the same merging operation on the remaining temporary clustering clusters until all temporary clustering clusters are processed, and finally obtain single target point clouds independent of each other.

[0019] Further, the specific process of step 3 is as follows:

[0020] Step 3.1, based on the input point cloud data, a feature extraction layer is added to the original PointNet++ neural network, and all features of the input point cloud are extracted and retained by using a multilayer perceptron (MLP);

[0021] Step 3.2, the input point set N is down-sampled by the sampling layer FPS (farthest point sampling) to the subset, and the selected sampling points constitute the center points of each local region of the grouping layer;

[0022] Step 3.3, the grouping layer groups the points obtained by the sampling layer, sets the center point as the center of the circle and artificially sets the radius r, and constructs the local region set by KNN neighbor search of the surrounding neighbor points;

[0023] Step 3.4, the input of the PointNet layer is set as the local region set, the local region pattern is encoded into a feature vector to obtain the local feature;

[0024] Step 3.5, a weighting function is used in the sampling layer and the grouping layer to strengthen the near-point feature and weaken the far-point interference, and the weighting function is added to the feature of the point;

[0025] The weighting function formula is as follows:

[0026] Wherein, alpha is the weighting function parameter, when the function parameter value increases or decreases, the feature of the point closer to the center point will increase or decrease, respectively;

[0027] Step 3.6, the operations of steps 3.2, 3.3 and 3.4 are repeated twice in turn, the feature extraction is continuously performed, and finally the global feature is obtained by aggregating all feature information through the maximum pooling layer, and the point cloud classification result is output.

[0028] The beneficial effects of the present application are:

[0029] The PointNet++ network before improvement does not capture local features in a deeper level, and the effect is not good for multi-classification problems with multiple instances. The original point cloud is preprocessed first to avoid the interference of noise points and outliers, and the structure of the PointNet++ network model is improved, a feature extraction layer is added on the basis of the original model to extract all features of the initially input points, the features of the initially input points ignored by the original network are made up, the features are extracted through two set sampling layers, the distance feature function is used to strengthen the features of nearby points and weaken the interference of distant points, finally all feature information is aggregated through a maximum pooling layer to obtain global features for classification, and classification scores are output to realize accurate classification of point clouds. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The overall flowchart of the application is shown in the figure.

[0031] Figure 2 The improved PointNet++ network model structure diagram of the application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application.

[0033] Figure 1 The overall flowchart of the point cloud recognition and classification method based on the improved PointNet neural network is shown in the figure, and the specific implementation steps are as follows:

[0034] Step 1, use a statistical filter to remove environmental noise points and outliers, mark discrete noise points by calculating field distance and remove them, and retain only point cloud groups of interest, and the specific steps are as follows:

[0035] 1) Set the point cloud set For each point, find its nearest neighbor point through K nearest neighbor search, calculate the average distance d of the point to all nearest neighbor points i , wherein the value of K controls the field size, if the field size is too large, the noise points cannot be effectively removed, and if the field size is too small, a large number of non-noise points will be removed;

[0036] 2) Calculate the mean value mu and the standard deviation sigma of the point set;

[0037]

[0038] 3) Compare all points with the field distance, if the distance is greater than mu+lambda sigma, mark it as an outlier and remove it, otherwise retain it, wherein mu+lambda sigma is a specified threshold, and lambda is a multiple of the standard deviation.

[0039] Step 2, the point cloud group of the region of interest obtained by the above step is segmented into single target point clouds independent of each other using a clustering segmentation algorithm, and the main process of the clustering segmentation algorithm is as follows:

[0040] 1) Set the field ε and the field density threshold MinPoints;

[0041] 2) Scan all data points, and if the number of points within a certain data point R radius is >= MinPoints, the data point is included in the core point list, and the points directly reaching the density of the data point form a corresponding temporary clustering cluster;

[0042] 3) For each temporary clustering cluster, check whether the points in the temporary clustering cluster are core points, and if so, merge the temporary clustering cluster corresponding to the point with the current temporary clustering cluster to obtain a new temporary clustering cluster;

[0043] 4) Repeat step 3) for all core points in the ε field of the core point until each point in the temporary clustering cluster is either not in the core point list or the points directly reaching the density of the point are all in the temporary clustering cluster, and the temporary clustering cluster is upgraded to a clustering cluster;

[0044] 5) Continue the same merging operation for the remaining temporary clustering clusters until all temporary clustering clusters are processed to form multiple single target point clouds separated from each other.

[0045] Step 3, input the point cloud processed in the above step into the improved PointNet++ neural network model for classification, and the improved network uses a multilayer perceptron to extract features of the input point cloud by adding a feature extraction layer, then uses two set sampling layers to continuously extract local features, uses a distance feature function to strengthen near-point features and weaken far-point interference, and finally aggregates all feature information through a max-pooling layer to obtain global features and outputs the point cloud classification result.

[0046] Figure 2 To improve the PointNet++ network model structure diagram, the specific process of using the improved PointNet++ model to extract deeper local features from the point cloud is as follows:

[0047] 1) Based on the input point cloud data, a feature extraction layer is added to the original PointNet++ neural network, and a MLP multilayer perceptron is used to extract all features of the input point cloud and retain them;

[0048] 2) Sampling layer: farthest point sampling (FPS) is used to downsample the point set, and the selected sampling points constitute the center points of each local region in the grouping layer;

[0049] 3) Grouping layer: grouping the selected points in the sampling layer, finding the neighboring points around the center point to construct the local region set;

[0050] 4) Constructing local regions by using the ball query method, setting the search ball radius r and the number of points k in the center point field, finding the neighboring points within the set radius r around the center point with the upper limit of k, thereby constructing the local region set;

[0051] 5) PointNet layer: inputting the local region set, converting the input coordinates to local coordinates relative to the center by subtracting the coordinates of the center points in the local region, thereby generating new feature dimensions in the PointNet network to obtain the point relationship in the local region, and encoding the local region pattern into a feature vector to obtain the local feature;

[0052] In the sampling layer and the grouping layer, the distance feature function is used simultaneously, and the weighted function formula is as follows:

[0053]

[0054] Wherein, alpha is the weighted function parameter (this design alpha = 1), when the function parameter value increases or decreases, the feature of the point closer to the center point will increase or decrease respectively, that is, when the distance of the point to the center point is close, the feature of the near point will be strengthened, and the interference of the far point will be weakened, the importance of the neighborhood point feature is changed, and the neighborhood is more specific and has a distinction;

[0055] 5) Repeat steps 2), 3), 4) in turn twice, and continuously extract features;

[0056] Finally, the maximum pooling layer is used as a symmetric function to aggregate all feature information to generate global features, and then the set shape combination of local and global features is classified to output the point cloud classification result.

[0057] The application carries out some experiments on the ModelNet40 data set, and compares with other classification methods, and the results are shown in Table 1, it can be seen that the classification accuracy of the network is improved and has certain advantages, and it is a good and effective point cloud classification method.

[0058] Table 1: Classification results on ModelNet40 data set

[0059] Method Input Accuracy (%) MVCNN Point 87.2 PointNet Point 90.6 PointNet++ Point 91.9 Ours Point 92.4

[0060] To sum up, on the basis of the original PointNet++ neural network, the application increases the feature extraction layer to use the multi-layer perceptron for feature extraction on the original input point cloud, then continuously extracts local features through two set sampling layers, simultaneously designs a distance feature function to strengthen the near point feature and weaken the far point interference, finally aggregates all feature information through the maximum pooling layer to generate global features, and outputs the point cloud classification result.

[0061] The above merely describes preferred embodiments of the present application, but does not constitute any form of limitation on the present application, and those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A point cloud object classification method based on an improved PointNet++ neural network, characterized in that, The method comprises the following steps: Step 1, preprocessing point cloud data, using a statistical filter to filter out noise points and outliers, and retaining only point cloud groups containing the region of interest; Step 2, using a clustering segmentation algorithm on the point cloud groups of the region of interest obtained in the above step to segment single target point clouds independent of each other; Step 3, inputting the point cloud in step 2 into an improved PointNet++ neural network model for classification, the improved network uses a multilayer perceptron to extract features of the input point cloud by adding a feature extraction layer, then continuously extracts local features through two set sampling layers, and uses a distance feature function in the set sampling layer to strengthen the features of nearby points and weaken the interference of distant points, finally aggregates all feature information through a max-pooling layer to obtain global features and outputs the point cloud classification result; The specific process of step 3 is as follows: Step 3.1, based on the input point cloud data, a feature extraction layer is added to the original PointNet++ neural network, and a multilayer perceptron MLP is used to extract all features of the input point cloud and retain them; Step 3.2, inputting the point set N into the sampling layer FPS farthest point sampling to downsample the subset, and the selected sampling points constitute the center points of each local region in the grouping layer; Step 3.3, the grouping layer groups the points obtained in the sampling layer, sets a radius r with the center point as the center, and constructs a local region set by KNN neighbor search around the nearby points; Step 3.4, setting the input of the PointNet layer as the local region set, encoding the local region pattern into a feature vector to obtain local features; Step 3.5, a weighting function is used in the sampling layer and the grouping layer to strengthen the features of nearby points and weaken the interference of distant points, and the weighting function formula is as follows: ; where a is a weighting function parameter, and as the function parameter value increases or decreases, the features of points closer to the center point are increased or decreased, respectively; Step 3.6, steps 3.2, 3.3 and 3.4 are repeated twice in turn, and the features are continuously extracted, finally all feature information is aggregated through a max-pooling layer to obtain global features, and the point cloud classification result is output; The specific process of step 1 is as follows: Step 1.1, Set of setpoint clouds , Compute the average distance d of a point to its nearest neighbors by K- nearest neighbor search i ; Step 1.2, calculating the mean of the point set and the standard deviation where d i is the average distance of each point to its nearest point, and n is the number of points in the point set; Step 1.3, compare the distance of all points with the field distance, if the distance is greater than mu+lambda sigma, mark it as an outlier and remove it, otherwise retain it, and lambda is the multiple of standard deviation.

2. The point cloud object classification method based on the improved PointNet++ neural network according to claim 1, characterized in that: The specific process of step 2 is as follows: Step 2.1, based on the point cloud group data obtained in step 2, using the density-based clustering method DBSCAN with noise for clustering segmentation, setting the field epsilon and the field density threshold MinPoints; Step 2.2, scanning all data points, if the number of points in a certain data point R radius range >= MinPoints, then it is included in the core point list, and the points directly connected to it form a corresponding temporary clustering cluster; Step 2.3, for each temporary clustering cluster, check whether the points in it are core points, if so, merge the temporary clustering cluster corresponding to the point with the current temporary clustering cluster to obtain a new temporary clustering cluster; Step 2.4, traverse all core points in the epsilon field of the core point, repeat the operation of step 2.3, until each point in the current temporary clustering cluster is either not in the core point list or its density directly connected points are already in the temporary clustering cluster, and the temporary clustering cluster is upgraded to a clustering cluster; Step 2.4, traverse all core points in the epsilon field of the core point, repeat the operation of step 2.3, until each point in the current temporary clustering cluster is either not in the core point list or its density directly connected points are already in the temporary clustering cluster, and the temporary clustering cluster is upgraded to a clustering cluster; Step 2.5, continue the same merging operation on the remaining temporary cluster clusters until all temporary cluster clusters are processed, and finally obtain single target point clouds independent of each other.

Images