A method for automatic classification of 3D point clouds based on graph convolutional neural networks

Abstract

A three-dimensional point cloud classification method based on a graph convolutional neural network is used to automatically classify and identify complex objects during the process of a service robot grasping a three-dimensional object, so as to facilitate grasping by determining the grasping position based on the category information. It belongs to the field of three-dimensional perception of computer vision and robot navigation technology. It includes the following steps: (1) preprocessing the three-dimensional point cloud; (2) constructing a point cloud classification network; (3) training the constructed network; (4) using the trained network to perform classification testing. The advantages of the present invention are: (1) using a graph convolutional neural network to learn local features, thereby enhancing the network's ability to learn local topological structure information; (2) learning global features of the input point cloud, thereby enhancing the network's understanding of contextual semantics. The local features and global features are aggregated, and the classification score of each point cloud is output using a fully connected layer, which has higher classification accuracy than the existing network.

Description

Technical Field

[0001] The present invention designs a three-dimensional point cloud automatic classification method based on a graph convolutional neural network. By constructing a deep learning network based on graph convolution, the service robot can automatically classify and identify complex objects in the process of grasping three-dimensional objects, so as to facilitate grasping by determining the grasping position according to the category information. The present invention relates to the field of three-dimensional perception of computer vision and robot navigation technology, and in particular to a method for automatic classification and recognition of three-dimensional point clouds. Background Art

[0002] With the development of theories such as computer vision and artificial intelligence, 3D point clouds are widely used and play an important role in the fields of autonomous driving, robot perception and navigation, and virtual / augmented reality. Especially in the field of robot perception, if the robot can detect objects and obtain the classification categories of different objects through deep learning training on the indoor 3D point cloud data scanned by its own 3D sensor, it will not only help the robot to select appropriate navigation strategies to avoid obstacles, but also help service robots to correctly identify and pick up / deliver objects required by users under the user's instructions, thereby greatly improving the quality of life of elderly people with limited mobility and people with disabilities. However, due to the characteristics of point clouds such as large data volume, irregular shape, and uneven density, the automatic classification of 3D point clouds has always been a very challenging problem.

[0003] Due to the irregularity and disorder of 3D point clouds, traditional convolutional neural networks are difficult to process directly. Therefore, early research based on deep learning is to process 3D point clouds into a form suitable for standard spatial convolution, such as conventional voxel grids or multi-views. However, data format conversion usually causes information loss, which ultimately leads to a decrease in 3D classification performance. In addition, the use of voxel grids or multi-view methods will incur expensive computational overhead, greatly increasing the spatial complexity and time complexity of the algorithm. Therefore, such methods are not suitable for automatic classification of large data volumes and large scene point cloud data. It was not until 2017 that Qi et al. proposed a point cloud neural network PointNet that directly classifies unstructured raw point clouds (Qi, CR; Su, H.; Mo, K.; Guibas, LJ Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017, 134 (1): 77-85.). Its basic idea is to learn the features of each point through a shared multi-layer perception (Multi-Layer-Perception, abbreviated as MLP) network, and then aggregate the features of all individual points into a global point cloud representation, while using a symmetric function (such as maximum pooling) to ensure that the order of the point cloud remains unchanged, and finally output a one-dimensional matrix through a fully connected layer after the aggregated features. Each component of the matrix corresponds to the classification score of the point cloud classified into that category, and the category with the highest score is taken as the category of the input point cloud. However, the network ignores the geometric relationship between points, so it cannot obtain local fine-grained feature information composed of adjacent points, and the classification accuracy still needs to be improved.

[0004] The present invention proposes an end-to-end graph neural network that captures local fine-grained feature information while taking into account contextual semantics for the automatic classification task of point clouds, helping service robots to correctly identify object categories and avoid obstacles in the process of grasping objects. By utilizing the characteristic of graph convolution that can maintain the permutation invariance of point clouds, a graph convolutional neural network is constructed to capture the local fine-grained features of point clouds, thereby making up for the shortcomings of existing methods in local point cloud feature extraction. Through the development and integration of modules such as data acquisition, data preprocessing, local feature extraction, and global feature extraction, a three-dimensional point cloud classification method based on graph convolutional neural network is proposed, which improves the classification accuracy by enhancing the network's ability to extract fine-grained features in local areas. Summary of the invention

[0005] The present invention proposes a three-dimensional point cloud classification method based on a graph convolutional neural network, and designs a graph convolutional neural network to process point clouds collected in a metric space, so as to help service robots better identify object categories and avoid obstacles. Specifically, a preprocessing module is first used to implement data enhancement, and then a local feature extraction module and a global feature extraction module are constructed to respectively obtain local features with local fine-grained information and global features with contextual semantic information, and finally, the local features and global features are combined through an aggregation function to represent the final features of the point cloud. The aggregated features are output as a one-dimensional matrix through a fully connected layer, and each component of the matrix corresponds to the classification score of the point cloud classified into that category, and the category with the highest score is taken as the category of the input point cloud. The present invention solves the problem of low classification accuracy in the process of three-dimensional point cloud processing, and realizes the function of the service robot to accurately identify the category of objects in the process of grasping objects.

[0006] The present invention adopts the following technical solutions and implementation steps:

[0007] A three-dimensional point cloud automatic classification method based on graph convolutional neural network, the method comprising:

[0008] Step 1: Point cloud data preprocessing;

[0009] Step 2: Construct a graph convolutional neural network for automatic classification of 3D point clouds;

[0010] Step 3: Train the constructed network;

[0011] Step 4: Use the trained point cloud automatic classification network to perform classification tests.

[0012] The data preprocessing specifically includes the following operations:

[0013] The data preprocessing refers to the preprocessing of point cloud data acquired by 3D laser radar, scanner and other equipment or existing public data sets. The specific process is: normalizing the collected point cloud data to the unit sphere, and randomly selecting a number of points as its sampling samples; in addition, in order to improve the generalization ability of the network, the present invention adopts random rotation, random scaling, random jitter and adding Gaussian noise to achieve data amplification.

[0014] The graph convolutional neural network for automatically classifying three-dimensional point clouds specifically includes the following modules:

[0015] Module 1: Space Transformation Matrix Module

[0016] The spatial transformation matrix module refers to a D×D transformation matrix that is most conducive to network classification by learning the pose information of the point cloud itself. Its input is a preprocessed three-dimensional point cloud B×n×D (where B, n, and D represent the batch size, number of point clouds, and feature dimensions of the input point cloud, respectively), and the output is a point cloud obtained after spatial transformation. The parameters involved in the spatial transformation matrix module are obtained through training and learning. This module uses convolution operations to downsample the input point cloud three times, and then uses the maximum pooling layer and the fully connected layer to upsample the input features, and finally adds weights and biases. The specific calculation steps of the spatial transformation matrix are as follows:

[0017] T=ω·γ(g(h(x1),h(x2)...h(x n )))+b(1)

[0018] Where: {x1,x2,...,x n} represents a three-dimensional point cloud, γ and h represent multi-layer perceptrons, g represents a symmetric function, ω and b represent weights and biases, respectively. After the above operations, the output of the network is a three-dimensional point cloud {x′1,x′2,...,x′ n}.

[0019] Module 2: Local feature extraction module

[0020] The local feature extraction module is a graph neural network used to construct local features for points in the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a local feature that aggregates the features of K neighboring points. The specific operation steps are as follows: Use a directed graph G = (V, E) to represent the point cloud. The vertices and edges in the directed graph are represented by V = {1, 2, ..., n} and The local information of each point in the point cloud and its neighboring points can be represented by its K-nearest neighbor graph (KNN graph for short), then the entire point cloud can be represented by graph G, and the local features are calculated by constructing a local neighborhood graph and applying graph convolution operations on the edges connecting adjacent point pairs, which is defined as follows:

[0021] e ij =ReLU(θ·(x j ′-x i ′)+φ·x i ′) (2)

[0022] Where: x i ′ is a point in the current 3D point cloud, x j ′ represents point x i ′’s neighboring points, θ and φ represent weights, ReLU represents the activation function, e ijRepresents local features. Finally, we define the local graph convolution operation by applying a symmetric function (taking the maximum value) to all the associated edges of each vertex, so the output of the local graph convolution of the i-th vertex is:

[0023]

[0024] Where: i represents the center point x i ′, j represents the center point x i ′’s K nearest neighbors, ε represents the subscript of x i The subscript set of the corresponding points of the local point cloud of the KNN graph centered at ′. After the above operation, the output size of the network is B×n×K×D, where K represents the number of neighboring points.

[0025] Module 3: Global feature extraction module

[0026] The global feature extraction module is composed of a KNN graph and a 3D convolution layer, and is used to obtain the global features of the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a global feature representing the entire point cloud. The process of extracting global features by the global feature extraction module is as follows: first, the input point cloud is divided into n groups using the KNN graph, and then two layers of three-dimensional convolution operations are used to map them into n×64-dimensional features, and then two layers of two-dimensional convolution functions are used to map the obtained n×64-dimensional features to n×128 dimensions, and then the features obtained by each layer of convolution operations are merged, and finally the global features are obtained by two-dimensional convolution operations. The output size of the network is B×n×D.

[0027] Module 4: Related layer module

[0028] The related layer module refers to a module that uses an aggregation function to merge local features and global features to obtain local fine-grained features and point cloud features with contextual semantic information.

[0029] Module 5: Output Module

[0030] The output module is mainly composed of a fully connected layer, which is used to integrate the feature information with category distinction in the convolution layer. In order to improve the performance of the network, each neuron in the fully connected layer uses the ReLU activation function. The definition of the fully connected layer is as follows:

[0031] z=ReLU(W·m+c) (4)

[0032] Where ReLU is the activation function, m is the input of the layer, W is the weight matrix of the layer, c is the bias of the layer, and z is the output of the layer.

[0033] The training network model specifically includes the following steps:

[0034] Step 1: The present invention normalizes the point cloud data obtained by the service robot using 3D laser radar, scanner and other equipment or each point cloud data in the existing public data set into a unit sphere, and then randomly extracts 1024 points as its sampling samples. In each iterative training of the point cloud data, the arrangement order of the training data is randomly disrupted. At the same time, in order to increase the amount of training data and improve the generalization ability of the network, the present invention adopts random rotation, random scaling in the scale range of [0.8, 1.2], random jitter in the interval [-0.1, 0.1] and adding [0.01, 0.05] Gaussian noise to achieve data amplification.

[0035] The preprocessed training data set and the corresponding classification benchmark value are used as input, and the constructed 3D point cloud automatic classification network is trained by forward propagation to learn the parameters of the network. The learning parameters are initialized before training, where the initial value of the learning rate is set to [0, 0.1], the momentum of batch normalization is [0, 1], the maximum number of iterations is [200, 300], and the batch size is [8, 32].

[0036] Step 2: Calculate the loss cost function value L based on cross entropy y ′:

[0037]

[0038] Where: y i Refers to the i-th value in the predicted label, y i ′ refers to the i-th value in the actual label. The batch gradient descent method is used to solve the global optimal solution of the entire training sample according to the loss function. At the same time, the Adam optimizer is used to update the learning parameters such as the weight and bias of the model. The Adam optimizer allows each parameter to obtain an adaptive learning rate to achieve a dual improvement in optimization quality and speed.

[0039] Step 3: Repeat steps 1 and 2, and continuously iterate the training of network model parameters to obtain the optimal 3D point cloud automatic classification network model. In order to prevent overfitting, the Dropout mechanism is used in the training process, so that the model randomly sets the weights of some nodes in the network to zero during the training process, and does not participate in the training, to avoid overfitting during the training process.

[0040] Finally, the trained network is used to perform 3D point cloud classification tests. Unlike the training phase, the test phase only requires loading the trained network model and does not require retraining the network model.

[0041] Beneficial effects:

[0042] The present invention provides a three-dimensional point cloud automatic classification method based on a graph convolutional neural network, which is convenient for a service robot to automatically classify and identify complex objects when grasping three-dimensional objects. Through the designed space transformation matrix module, local feature extraction module, global feature extraction module, related layer module and output module, the automatic classification task of three-dimensional point clouds can be implemented end-to-end. The three-dimensional point cloud automatic classification network of the present invention not only includes local features but also integrates contextual semantic information in the feature extraction process, thereby improving the classification accuracy of the network. Experimental data show that the network model obtained through deep learning training can correctly identify 2261 three-dimensional point clouds among the given 2468 three-dimensional point cloud test data by applying the present invention, with an accuracy rate of up to 91.6%, which is 2.4% higher than PoinetNet. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the method for automatic classification of three-dimensional point clouds based on graph convolutional neural network provided by the present invention.

[0044] Figure 2 It is a network structure diagram of the automatic classification method of three-dimensional point cloud based on graph convolutional neural network provided by the present invention.

[0045] Figure 3 It is a network structure diagram of the local feature extraction module in the three-dimensional point cloud automatic classification method based on graph convolutional neural network provided by the present invention.

[0046] Figure 4 It is the point cloud data in the ModelNet40 data set provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The purpose of the present invention is to provide a three-dimensional point cloud automatic classification method based on a graph convolutional neural network, which is used for automatic classification and recognition of complex objects in the process of a service robot grasping a three-dimensional object, and is convenient for grasping the grasping position according to the category information. The network training can be completed end-to-end without any post-processing process. The three-dimensional point cloud automatic classification method of the graph convolutional neural network of the present invention can not only extract local fine-grained features but also integrate contextual semantic information, significantly improving the classification accuracy.

[0048] The present invention will be described in detail below in conjunction with the accompanying drawings. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0049] Figure 1 It is a flow chart of a method for automatic classification of three-dimensional point clouds based on graph convolutional neural network provided by the present invention; Figure 2 It is a network structure diagram of the automatic classification method of three-dimensional point cloud based on graph convolutional neural network provided by the present invention. Figure 3It is a network structure diagram of the local feature extraction module in the three-dimensional point cloud automatic classification method based on graph convolutional neural network provided by the present invention. Figure 4 It is the point cloud data in the ModelNet40 data set provided in an embodiment of the present invention.

[0050] The three-dimensional point cloud automatic classification method based on graph convolutional neural network provided by the present invention specifically includes:

[0051] Step 1: Data Preprocessing

[0052] The data preprocessing described in the embodiment refers to preprocessing the public data set ModelNet40. Figure 4 For each point cloud model, 1024 points are randomly selected as the sampling samples, and the coordinates of each point cloud model are pre-normalized to the unit ball interval [-1, 1]. Data augmentation is achieved by random rotation, random scaling in the scale range [0.8, 1.2], random jitter in the interval [-0.1, 0.1], and adding [0.01, 0.05] Gaussian noise.

[0053] Step 2: Build a 3D point cloud automatic classification network

[0054] The graph convolutional neural network for automatic classification of three-dimensional point clouds constructed by the present invention specifically includes a space transformation matrix module, a local feature extraction module, a global feature extraction module, a correlation layer module and an output module.

[0055] The spatial transformation matrix module refers to a 3×3 transformation matrix that is most conducive to network classification by learning the pose information of the point cloud itself. Its input is a preprocessed three-dimensional point cloud B×n×D (where B, n, and D represent the batch size, number of point clouds, and feature dimensions of the input point cloud, respectively), and the output is a point cloud obtained after spatial transformation. The parameters involved in the spatial transformation matrix module are obtained through training and learning. The module consists of a convolutional neural network with a convolution kernel of 1×1, a step size of 1, an output size of 64, 128, and 1024, a maximum pooling layer, and a fully connected layer with an output size of 512 and 128. Except for the last layer, all layers include a ReLU activation function and a BN layer. The final output size of the network is a 16×3×3 three-dimensional point cloud. The specific calculation of the spatial transformation matrix is ​​shown in formula (1).

[0056] The local feature extraction module is a graph neural network used to construct local features for points in the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a local feature that aggregates the features of 20 neighboring points. The specific operation steps are as follows: Use a directed graph G = (V, E) to represent the point cloud. The vertices and edges in the directed graph are represented by V = {1, 2, ..., n} and Represented by. The local information of each point in the point cloud and its 20 neighboring points can be represented by a KNN graph, and the entire point cloud can be represented by a graph G. The 3D point cloud is mapped to 64 dimensions by constructing a local neighborhood graph and applying a graph convolution operation on the edges connecting adjacent point pairs, as shown in formula (2). Finally, the local graph convolution operation is defined by applying a symmetric function (taking the maximum value) to all associated edges of each vertex, as shown in formula (3). In the local feature extraction process, the local feature extraction module is iterated four times, where the output of the previous module is used as the input of the next module, and the 3D point cloud is mapped to 64 dimensions, 64 dimensions, 64 dimensions, and 128 dimensions in turn. Finally, the size of the 3D point cloud representing the local features is 16×1024×128, as shown in Figure 2 shown.

[0057] The global feature extraction module consists of a KNN graph and a 3D convolution layer, and is used to obtain the global features of the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a global feature representing the entire point cloud. The process of extracting global features by the global feature extraction module is as follows: First, the input point cloud is divided into 1024 groups using the KNN graph, and each group consists of a center point and 4 adjacent points. Secondly, two layers of three-dimensional convolution operations are used to map them into n×64-dimensional features, and the sizes of the convolution kernels are [1,1,3] and [1,1,1], respectively, with a step size of 1. Then, two layers of two-dimensional convolution functions are used to map the obtained n×64-dimensional features to n×128 dimensions, and then the features obtained by each layer of convolution operations are merged. Finally, the global features are obtained by a convolution operation with a convolution kernel of [1,1] and a step size of 1. The output size of the network is 16×1024×128, where 16, 1024, and 128 represent the batch size of the input point cloud, the number of point clouds, and the feature dimension, respectively.

[0058] The relevant layer module uses an aggregation function to merge global features and local features, and outputs a one-dimensional matrix of size 1×40 through an output module composed of fully connected layers with output sizes of 512, 256, and 40 respectively (40 represents 40 different categories of objects in the ModelNet40 dataset). Each component of the matrix corresponds to the classification score of the point cloud classified into that category, and the category with the highest score is taken as the category of the input point cloud.

[0059] Step 3: Train the network model:

[0060] The hardware environment of the present invention is Intel i9-7940x processor, 16GB memory, and GPU is TITAN XP; the software environment is Ubuntu 16.04 operating system, CUDA 9.0, CUDNN 7.3, TensorFlow 1.9 and Python 3.5.

[0061] In the embodiment, each three-dimensional point cloud model in the ModelNet40 data set is normalized to the unit sphere interval [-1,1], and then 1024 points are randomly selected as its sampling samples. In each iterative training of the point cloud data, the arrangement order of the training data is randomly disrupted. At the same time, in order to increase the amount of training data and improve the generalization ability of the network, the method in step 1 is used to preprocess the data.

[0062] During the training process, the pre-processed point cloud data with a batch size of 16 and the corresponding classification benchmark values ​​are first used as input, and the constructed 3D point cloud automatic classification network is trained through forward propagation to learn the parameters of the network. The initial value of the learning rate is set to 0.001, the momentum of batch normalization is 0.9, the maximum number of iterations is 250, and the batch size is 16. Then, based on the cross entropy loss function (5), the adaptive moment estimation method (Adaptive Moment Estimation, referred to as Adam) is used for back propagation training to update the model's weights, bias terms and other learning parameters. Adam allows each parameter to obtain an adaptive learning rate during the model training optimization process to achieve a dual improvement in optimization quality and speed. Finally, the network model parameters are iterated 250 times to obtain the optimal 3D point cloud automatic classification network model. In order to prevent overfitting, the Dropout mechanism is used during the training process, so that the model randomly sets the weights of some nodes of the network to zero during the training process, and does not participate in the training, to avoid overfitting during the training process.

[0063] Step 4: Use the trained point cloud automatic classification network to perform classification tests.

[0064] The trained automatic classification network is used to perform 3D point cloud classification tests. Unlike the training phase, the test phase only requires loading the trained network model and does not require retraining the network model.

[0065] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technology can understand and think of changes and substitutions within the technical scope disclosed by the present invention, which should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A three-dimensional point cloud automatic classification method based on graph convolutional neural network, characterized in that The following steps are involved: Step 1: construct a data set and perform preprocessing, using a 3D laser radar scanner to obtain point cloud data, the data set consists of L categories, including I training data and J test data; The specific process of data preprocessing is as follows: normalize the collected point cloud data to the unit sphere, randomly select several points as sampling samples; use random rotation, random scaling, random jitter and add Gaussian noise to achieve data amplification; Step 2: constructing a point cloud automatic classification network, wherein the point cloud automatic classification network includes a space transformation matrix module, a local feature extraction module, a global feature extraction module, a correlation layer module, and an output module; The spatial transformation matrix module refers to a D×D transformation matrix that is most conducive to network classification by learning the pose information of the point cloud itself. Its input is a preprocessed three-dimensional point cloud B×n×D, where B, n, and D represent the batch size, number of point clouds, and feature dimensions of the input point cloud, respectively. The output is a point cloud obtained after spatial transformation. The parameters involved in the spatial transformation matrix module are obtained through training and learning; the module uses convolution operations to downsample the input point cloud three times, and then uses the maximum pooling layer and the fully connected layer to upsample the input features, and finally adds weights and biases; after the above operations, the output size of the network is B×D×D; The local feature extraction module is a graph neural network used to construct local features for points in the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a local feature that aggregates the features of K neighboring points. The specific operation steps are as follows: a directed graph G = (V, E) is used to represent the point cloud. The vertices and edges in the directed graph are represented by V = {1, 2, ..., n} and Representation; the local information of each point in the point cloud and its neighboring points is represented by its K nearest neighbor graph, then the entire point cloud is represented by a graph G, and local features are calculated by constructing a local neighborhood graph and applying a graph convolution operation on the edges connecting adjacent point pairs. After the above operation, the output size of the network is B×N×K×D, where K represents the number of neighboring points; The global feature extraction module is composed of a KNN graph and a 3D convolution layer, and is used to obtain the global features of the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a global feature representing the entire point cloud. The process of extracting global features by the global feature extraction module is as follows: first, the KNN graph is used to divide the input point cloud into n groups, and then each group of point clouds is subjected to feature extraction through a 3D convolution layer, and finally, the global features are obtained through a maximum pooling operation. The output size of the network is B×N×D. The related layer module refers to merging local features and global features to obtain local fine-grained features and point cloud features with contextual semantic information; The output module is mainly composed of a fully connected layer, which is used to integrate the feature information with category distinction in the convolution layer; each neuron in the fully connected layer adopts the ReLU activation function; Step 3: Model training: normalize the point cloud data obtained by the service robot using a 3D laser radar or scanner device or each point cloud data in the existing public data set to the unit sphere, and then randomly select a number of points as its sampling samples. In each iterative training of the point cloud data, the order of the training data is randomly disrupted, and data augmentation is achieved by random rotation, random scaling within the scale range of [0.8, 1.2], random jitter within the interval [-0.1, 0.1], and adding [0.01, 0.05] Gaussian noise; The preprocessed training data set and the corresponding classification benchmark value are used as input, and the constructed 3D point cloud automatic classification network is trained through forward propagation to learn the parameters of the network. The learning parameters are initialized before training, where the initial value of the learning rate is set to [0, 0.1], the momentum of batch normalization is [0, 1], the maximum number of iterations is [200, 300], and the batch size is [8, 32]. Then, the adaptive moment estimation method, referred to as Adam, is used for back propagation training based on the cross entropy loss function to update the weights and bias terms of the model. Adam allows each parameter to obtain an adaptive learning rate during the model training and optimization process to achieve a dual improvement in optimization quality and speed; finally, the network model parameters are continuously iterated to obtain the optimal 3D point cloud automatic classification network model; the Dropout mechanism is used during the training process, so that the model randomly sets the weights of some nodes in the network to zero during the training process, and does not participate in the training, to avoid overfitting during the training process; Step 4: Use the trained classification network to perform 3D point cloud classification test.

2. The automatic classification method of three-dimensional point cloud according to claim 1, characterized in that: The preprocessing described in step 1 refers to: for each point cloud model, 1024 points are randomly selected as its sampling samples, and the coordinates of each point cloud model are pre-normalized to the unit sphere interval [-1,1], and data amplification is achieved by random rotation, random scaling in the scale range of [0.8,1.2], random jitter in the interval [-0.1,0.1], and adding [0.01,0.05] Gaussian noise.

3. The automatic classification method of three-dimensional point cloud according to claim 1, characterized in that: The specific calculation steps of the space transformation matrix are as follows: T=ω·γ(g(h(x1),h(x2)...h(x n )))+b(1) Where: {x1,x2,...,x n } represents a three-dimensional point cloud, γ and h represent multilayer perceptrons, g represents a symmetric function, ω and b represent weights and biases, respectively; after the above operations, the output of the network is a three-dimensional point cloud {x′1,x′2,...,x′ n }, where B and D represent the batch size and feature dimension of the input point cloud, respectively.

4. The automatic classification method of three-dimensional point cloud according to claim 1, characterized in that: The definition of local feature extraction described in step 1 is as follows: e ij =ReLU(θ·(x′ j -x′ i )+φ·x′ i ) (2) Where: x′ i is a point in the 3D point cloud currently being focused on, x′ j Represents point x′ i The neighboring points of θ and φ represent weights, ReLU represents the activation function, and e ij Represents local features; Finally, the local graph convolution operation is defined by applying a symmetric function (taking the maximum value) to all associated edges of each vertex, so the output of the local graph convolution of the i-th vertex is: Where: i represents the center point x′ i The subscript j represents the center point x′ i The subscript of the K nearest neighbors of x′ i The subscript set of corresponding points in the local point cloud of the KNN graph centered on .

5. The automatic classification method of three-dimensional point cloud according to claim 1, characterized in that: The global feature extraction module is composed of a KNN graph and a 3D convolution layer, and is used to obtain the global features of the point cloud. Its input is a three-dimensional point cloud that has undergone spatial transformation, and its output is a global feature representing the entire point cloud. The process of extracting global features by the global feature extraction module is as follows: first, the input point cloud is divided into n groups using the KNN graph, and then two layers of three-dimensional convolution operations are used to map them into n×64-dimensional features, and then two layers of two-dimensional convolution functions are used to map the obtained n×64-dimensional features to n×128 dimensions, and then the features obtained by each layer of convolution operations are merged, and finally the global features are obtained through a two-dimensional convolution operation. After the above operations, the output size of the network is B×N×D.

6. The automatic classification method of three-dimensional point cloud according to claim 1, characterized in that: The aggregation function is used to merge local features and global features in the relevant layer module.

Images