A 3D point cloud classification method integrating persistent homology

Abstract

The present invention belongs to the field of computer vision technology, and specifically relates to a three-dimensional point cloud classification method integrating persistent homology. In order to solve the problem of lack of characterization of point cloud topological features in current three-dimensional point cloud classification technology, the present invention further introduces the persistent homology method in algebraic topology to classify point clouds on the basis of the existing neural network-based point cloud classification technology PointNet++. First, a point cloud witness simplicial complex topological structure is constructed, and the persistent homology topological features of the point cloud are quantified from two aspects: Betti number and persistence graph. Secondly, a loss function based on persistent homology is defined, and the network model for three-dimensional point cloud classification is trained and learned accordingly to obtain various parameters of the neural network model. Finally, the trained convolutional neural network is used to perform the classification task of three-dimensional point clouds. The test results show that the present invention significantly improves the accuracy of point cloud classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a three-dimensional point cloud classification method integrating persistent homology, which is suitable for classifying three-dimensional point clouds. Background Art

[0002] With the popularity of 3D scanning equipment such as LiDAR and depth cameras, the collection of 3D point cloud data is becoming more and more convenient. How to distinguish these point cloud data is a key problem to be solved in the fields of autonomous driving, intelligent robot navigation, scene reconstruction, virtual reality, etc. 3D point cloud classification refers to classifying point clouds into different point cloud sets. The same point cloud set has similar or identical attributes, and marking the classified point cloud sets with labels to obtain the category of each point cloud set. Traditionally, machine learning methods are generally used for point cloud classification. With the rapid development of deep learning technology in recent years, point cloud classification technology based on deep learning has gradually emerged.

[0003] At present, the three-dimensional point cloud classification method is mainly based on the geometric structure of the point cloud for classification. The description of the point only includes its three-dimensional coordinates and normal vector and other geometric information, but lacks the description of the topological properties of the point cloud, which makes the point cloud classification basis incomplete. Therefore, the classification performance of the point cloud classification research combined with the essential topological properties of the point cloud still has a lot of room for improvement. In recent years, a topological data analysis method based on the persistent homology theory has emerged internationally, which is suitable for quantifying the topological features of objects of arbitrary shapes. The present invention integrates the topological invariance features based on the persistent homology theory on the basis of the existing point cloud geometric features, so as to fully characterize the three-dimensional point cloud structure and improve the three-dimensional point cloud classification capabilities. Summary of the invention

[0004] In view of the problem that existing three-dimensional point cloud classification methods lack point cloud topological features and are difficult to improve in accuracy, the present invention proposes a three-dimensional point cloud classification method that integrates persistent homology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A 3D point cloud classification method integrating persistent homology comprises the following steps:

[0007] Step 1: Establish a 3D point cloud data set, divide it into a training data set and a test data set, and complete the 3D point cloud data preprocessing;

[0008] Step 2, construct the witness simplicial complex topological structure of the three-dimensional point cloud through the witness simplicial complex;

[0009] Step 3, calculate the persistent homology features in the witness simplicial complex topology of the three-dimensional point cloud;

[0010] Step 4, define a loss function of a convolutional neural network based on persistent homology;

[0011] Step 5: training a convolutional neural network model for the 3D point cloud classification task;

[0012] Step 6: Use the convolutional neural network model trained in step 5, input the 3D point cloud test data set, perform the 3D point cloud classification task, and output the category label of the object to which each point cloud belongs.

[0013] Furthermore, in step 1, a three-dimensional point cloud data set is established and divided into a training data set and a test data set, and three-dimensional point cloud data preprocessing is completed at the same time. The specific steps are as follows:

[0014] Step 1.1, three-dimensionally measure objects in the real world by three-dimensional scanning to obtain their point cloud representation. Each point cloud P is a point set representation of an object, in which any point defines the three-dimensional spatial coordinates (x, y, z) of the point. Through a large number of three-dimensional scanning measurements, three-dimensional point clouds of many objects are obtained to form a three-dimensional point cloud data set;

[0015] Step 1.2: For the training data set, give the object category label c corresponding to each point cloud P;

[0016] Step 1.3, use the meshlabserver plug-in in meshlab to batch downsample the 3D point cloud data, reduce the number of points by reducing the number of faces, and set the topological features to be retained before downsampling.

[0017] Furthermore, in step 2, the witness simplicial complex topology structure is constructed by the witness simplicial complex, and the specific steps are:

[0018] Each point in the point cloud P is surrounded by a sphere, and the diameter λ of the sphere gradually increases from 0. When the distance between any two points is less than the diameter λ at that moment, the two points are connected together. In this process, simplicial complexes are used to fill the entire changing structure, and finally a series of nested simplicial complexes are formed. As the diameter λ gradually increases, the witness complex constructed by the original point cloud P is larger than all previous witness complexes, that is, Where 0=λ0<λ1<λ2<…<λ n .

[0019] Furthermore, in step 3, the persistent homology feature is calculated in the witness simplicial complex topological structure of the three-dimensional point cloud, including calculating the persistence graph and the Betti number. The specific steps are as follows:

[0020] Step 3.1, calculate the persistence graph of point cloud P, and use the persistence graph H k(P) is used to characterize the topological feature descriptor of the point cloud P in the kth dimension;

[0021] Step 3.2, calculate the Betti number of the object category corresponding to the point cloud. Given a point cloud P and its corresponding object category label c, according to the homology group, its Betti number b in the k-th dimension space can be known k , the k-th Betti number is the rank of the k-th homology group, indicating the number of holes in the object in the k-th dimension.

[0022] Furthermore, in step 4, a loss function of a convolutional neural network based on persistent homology is defined, and the specific steps are:

[0023] Step 4.1: sort all feature points in the persistence graph according to the persistence duration (y k -x k ) are sorted from largest to smallest, so, Indicates the feature point with the i-th persistence time in the k-th dimension of the persistence graph. and Respectively represent the generation time and disappearance time of the feature point;

[0024] Step 4.2, define the loss function of point cloud P in the kth dimension as:

[0025]

[0026] Step 4.3, for a given three-dimensional point cloud P, define its persistent homology loss function in all topological space dimensions. The function formula is defined as follows:

[0027]

[0028] Step 4.4, define the loss function of the entire neural network. The loss function of the neural network consists of two parts: the cross entropy loss function and the persistent homology loss function, namely:

[0029]

[0030] Among them, X represents the training data set of 3D point cloud, ω represents the parameters of neural network, α represents the weight ratio of persistent homology loss function, and is the cross entropy loss function, which is used to measure the difference information between two probability distributions.

[0031] Furthermore, the convolutional neural network model for the three-dimensional point cloud classification task is trained in step 5, and the specific steps are as follows:

[0032] Step 5.1, select PointNet++ neural network as the point cloud classifier, use the farthest point sampling method to sample the original point cloud data during training, use ball query to generate local areas, and in addition, the network applies different scale grouping layers, and then extracts the features of each scale according to the points, and finally connects the features of different scales in series to form multi-scale features;

[0033] Step 5.2: Input the training data set X of the 3D point cloud into the Point Net++ neural network for training. By minimizing the value of the neural network loss function, determine the neural network parameter ω and the weight ratio λ of the persistent homology loss function, and complete the convolutional neural network training for the 3D point cloud classification task.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] (1) In terms of point cloud structure representation, it not only includes traditional geometric features, but also integrates point cloud topological features, thus improving the structured description of point clouds.

[0036] (2 A loss function of a convolutional neural network based on persistent homology is defined, the neural network structure is trained from the perspective of global constraints, and the network parameters are further optimized.

[0037] (3) Compared with the existing point cloud classification methods PointNet, PointNet++, Point CNN, and SpiderCNN, the point cloud classification accuracy of the present invention is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flow chart of the method of the present invention;

[0039] Figure 2 Point cloud data examples;

[0040] Figure 3 Schematic diagram of the witness simplicial complex topology structure of the point cloud;

[0041] Figure 4 Schematic diagram of the point cloud persistence graph. DETAILED DESCRIPTION

[0042] Example 1

[0043] like Figure 1 As shown, a 3D point cloud classification method integrating persistent homology in this embodiment includes the following steps:

[0044] Step 1: Create a 3D point cloud data set, divide it into a training data set and a test data set, and complete the 3D point cloud data preprocessing. The specific steps are as follows:

[0045] Step 1.1, three-dimensionally measure the objects in the real world by three-dimensional scanning to obtain the representation of their point clouds, such as Figure 2 As shown. Each point cloud P is a point set representation of an object, where any point defines the three-dimensional spatial coordinates (x, y, z) of the point. Through a large number of three-dimensional scanning measurements, three-dimensional point clouds of many objects are obtained to form a three-dimensional point cloud data set;

[0046] Step 1.2: For the training data set, give the object category label c corresponding to each point cloud P;

[0047] Step 1.3, use the meshlabserver plugin in meshlab to batch downsample the 3D point cloud data, reduce the number of points by reducing the number of faces, and set the topological features to be preserved before downsampling. This not only preserves the topological features inside the 3D model, but also reduces the density of the model and shortens the subsequent calculation time.

[0048] Step 2: Construct the witness simplicial complex topological structure of the 3D point cloud through the witness simplicial complex. The specific steps are as follows:

[0049] The three-dimensional point cloud itself is a collection of discrete points without structured relationships. Its topological structure must be established before quantifying its topological features. In the topological feature analysis based on persistent homology, the topological structure to be constructed is usually in the form of a simplicial complex. The present invention uses witness simplicial complexes to construct the topological structure of the three-dimensional point cloud model. Each point in the point cloud P is surrounded by a sphere, and the diameter λ of the sphere gradually increases from 0. When the distance between any two points is less than the diameter λ at that moment (i.e., the filter value λ), the two points are connected together. In this process, simplicial complexes are used to fill the entire changing structure, and finally a series of nested simplicial complexes are formed. Figure 3 As the diameter λ (i.e., the filter value λ) gradually increases, the witness complex constructed by the original point cloud P is larger than all previous witness complexes, i.e. Where 0=λ0<λ1<λ2<…<λ n .

[0050] Step 3: Calculate the persistent homology features in the witness simplicial complex topological structure of the 3D point cloud. The specific steps are as follows:

[0051] Step 3.1, calculate the persistence graph of the point cloud P. The persistence homology method can analyze the appearance and disappearance of holes in various dimensions within a wide range of filtering scales. k (P) is used to characterize the topological feature descriptor of the point cloud P in the kth dimension. Figure 4An example of a persistence graph of a point cloud P in three different dimensions is given. The horizontal axis of the persistence graph represents the time when the hole is generated (that is, the filtering value λ at this time), and the vertical axis represents the time when the hole disappears (that is, the filtering value λ at this time). Therefore, the closer the topological feature point is to the upper left corner on the persistence graph, the more stable it is.

[0052] Step 3.2, calculate the Betti number of the object category corresponding to the point cloud. A common way to quantify the topological structure of an object is to observe the rank of the homology group of the object, that is, the Betti number. The k-th dimension Betti number is the rank of the k-th homology group, which represents the number of holes in the object in the k-th dimension. Given a point cloud P and its corresponding object category label c, its Betti number b in the k-th dimension space can be known according to the homology group. k For example, the Betti numbers corresponding to the tire class are: b0=1, b1=2.

[0053] Step 4: Define a loss function of a convolutional neural network based on persistent homology. The specific steps are:

[0054] Step 4.1: sort all feature points in the persistence graph according to the persistence duration (y k -x k ) are sorted from largest to smallest, so, Indicates the feature point with the i-th persistence time in the k-th dimension of the persistence graph. and Respectively represent the generation time and disappearance time of the feature point;

[0055] Step 4.2, in the training data set of the neural network, since the classification label c of the three-dimensional point cloud P is a known condition, it can be seen from step 3.2 that the persistent homology feature (i.e., Betti number) of the point cloud P in the kth dimension is also known, i.e., b k The loss function of the point cloud P in the kth dimension is defined as:

[0056]

[0057] Step 4.3, for a given three-dimensional point cloud P, define its persistent homology loss function in all topological space dimensions. The function formula is defined as follows:

[0058]

[0059] Since the topological dimension in the three-dimensional point cloud space is 0 to 2, the value of k is also 0 to 2.

[0060] Step 4.4, define the loss function of the entire neural network. The loss function of the neural network consists of two parts: the cross entropy loss function and the persistent homology loss function, namely:

[0061]

[0062] Among them, X represents the training data set of 3D point cloud, ω represents the parameters of neural network, α represents the weight ratio of persistent homology loss function, and is the cross entropy loss function, which is used to measure the difference information between two probability distributions.

[0063] Step 5: Train the convolutional neural network model for the 3D point cloud classification task. The specific steps are as follows:

[0064] Step 5.1, select PointNet++ neural network as the point cloud classifier, use the farthest point sampling method to sample the original point cloud data during training, use ball query to generate local areas, and in addition, the network applies different scale grouping layers, and then extracts the features of each scale according to the points, and finally connects the features of different scales in series to form multi-scale features;

[0065] The structure of the Point Net++ neural network is shown in Table 1, where N represents the number of point cloud categories in the training dataset.

[0066] Table 1 Neural network structure

[0067]

[0068] Step 5.2: Input the 3D point cloud training data set X into the Point Net++ neural network for training.

[0069] Given the classification label c of each point cloud P in the dataset, determine the Betti number b of each dimension in its topological space through step 3.2. k .

[0070] For each point cloud P, calculate its persistence map H according to step 3.1 k (P).

[0071] Calculate the persistent homology loss function according to step 4.3

[0072] Calculate the neural network loss function according to step 4.4 By minimizing the value of the neural network loss function, determining the neural network parameter ω and the weight ratio α of the persistent homology loss function, the convolutional neural network training for the three-dimensional point cloud classification task is completed.

[0073] Step 6: Use the convolutional neural network model trained in step 5, input the 3D point cloud test data set, perform the 3D point cloud classification task, and output the category label of the object to which each point cloud belongs.

[0074] In order to demonstrate the advantages of the present invention in the accuracy of point cloud classification, a three-dimensional point cloud classification test was conducted on the dataset on ModelNet40, and compared with the existing excellent point cloud classification methods PointNet, PointNet++, Point CNN, and Spider CNN, and the classification results are shown in Table 2. The results show that the accuracy of the present invention is the best both in terms of the average classification of each category and in terms of overall classification.

[0075] Table 2 Comparison of 3D point cloud classification accuracy

[0076] Point cloud classification method Average classification accuracy (%) Overall classification accuracy (%) PointNet 84.7 89.2 PointNet++ 88.7 91.9 Point CNN 89.4 92.2 Spider CNN 89.8 92.4 The present invention 91.2 93.8

Claims

1. A 3D point cloud classification method integrating persistent homology, characterized in that: The following steps are involved: Step 1: Establish a 3D point cloud data set, divide it into a training data set and a test data set, and complete the 3D point cloud data preprocessing; Step 2, construct the witness simplicial complex topological structure of the three-dimensional point cloud through the witness simplicial complex; Step 3, calculate the persistent homology features in the witness simplicial complex topology of the three-dimensional point cloud; Step 4, define a loss function of a convolutional neural network based on persistent homology; Step 5: training a convolutional neural network model for the 3D point cloud classification task; Step 6: Use the convolutional neural network model trained in step 5, input the 3D point cloud test data set, perform the 3D point cloud classification task, and output the category label of the object to which each point cloud belongs; In step 4, a loss function of a convolutional neural network based on persistent homology is defined, and the specific steps are: Step 4.1: sort all feature points in the persistence graph according to the persistence duration (y k -x k ) are sorted from largest to smallest, so, Indicates the feature point with the i-th persistence time in the k-th dimension of the persistence graph. and Respectively represent the generation time and disappearance time of the feature point; Step 4.2, define the loss function of point cloud P in the kth dimension as: Step 4.3, for a given three-dimensional point cloud P, define its persistent homology loss function in all topological space dimensions. The function formula is defined as follows: Step 4.4, define the loss function of the entire neural network. The loss function of the neural network consists of two parts: the cross entropy loss function and the persistent homology loss function, namely: Among them, X represents the training data set of 3D point cloud, ω represents the parameters of neural network, α represents the weight ratio of persistent homology loss function, and is the cross entropy loss function, which is used to measure the difference information between two probability distributions.

2. A 3D point cloud classification method integrating persistent homology according to claim 1, characterized in that: In step 1, a 3D point cloud data set is established and divided into a training data set and a test data set. At the same time, the 3D point cloud data preprocessing is completed. The specific steps are as follows: Step 1.1, three-dimensionally measure objects in the real world by three-dimensional scanning to obtain their point cloud representation. Each point cloud P is a point set representation of an object, in which any point defines the three-dimensional spatial coordinates (x, y, z) of the point. Through a large number of three-dimensional scanning measurements, three-dimensional point clouds of many objects are obtained to form a three-dimensional point cloud data set; Step 1.2: For the training data set, give the object category label c corresponding to each point cloud P; Step 1.3, use the meshlabserver plug-in in meshlab to batch downsample the 3D point cloud data, reduce the number of points by reducing the number of faces, and set the topological features to be retained before downsampling.

3. A 3D point cloud classification method integrating persistent homology according to claim 2, characterized in that: In step 2, the witness simplicial complex topology structure is constructed by the witness simplicial complex. The specific steps are: Each point in the point cloud P is surrounded by a sphere, and the diameter λ of the sphere gradually increases from 0. When the distance between any two points is less than the diameter λ at that moment, the two points are connected together. In this process, simplicial complexes are used to fill the entire changing structure, and finally a series of nested simplicial complexes are formed. As the diameter λ gradually increases, the witness complex constructed by the original point cloud P is larger than all previous witness complexes, that is, Where 0=λ0<λ1<λ2<…<λ n .

4. A 3D point cloud classification method integrating persistent homology according to claim 2, characterized in that: In step 3, the persistent homology feature is calculated in the witness simplicial complex topological structure of the three-dimensional point cloud. It includes two parts: calculating the persistence graph and the Betty number. The specific steps are: Step 3.1, calculate the persistence graph of point cloud P, and use the persistence graph H k (P) is used to characterize the topological feature descriptor of the point cloud P in the kth dimension; Step 3.2, calculate the Betti number of the object category corresponding to the point cloud. Given a point cloud P and its corresponding object category label c, according to the homology group, its Betti number b in the k-th dimension space can be known k , the k-th Betti number is the rank of the k-th homology group, indicating the number of holes in the object in the k-th dimension.

5. The method for 3D point cloud classification integrating persistent homology according to claim 1, characterized in that: The convolutional neural network model for the three-dimensional point cloud classification task in step 5 is trained in the following specific steps: Step 5.1, select PointNet++ neural network as the point cloud classifier, use the farthest point sampling method to sample the original point cloud data during training, use ball query to generate local areas, and in addition, the network applies different scale grouping layers, and then extracts the features of each scale according to the points, and finally connects the features of different scales in series to form multi-scale features; Step 5.2: Input the training data set X of the 3D point cloud into the PointNet++ neural network for training. By minimizing the value of the neural network loss function, determine the neural network parameter ω and the weight ratio λ of the persistent homology loss function, and complete the convolutional neural network training for the 3D point cloud classification task.

Images