A point cloud feature extraction method based on edge convolution network, clustering network and attention mechanism

Abstract

The application discloses a point cloud feature extraction method based on an edge convolution network, a clustering network and an attention mechanism, and comprises the following steps: firstly, constructing a point cloud feature extraction network based on the edge convolution network, the clustering network and the attention mechanism; and then, performing feature extraction; the point cloud feature extraction network firstly extracts local features of each point by using an edge convolution layer, then an attention module is used on the output of the edge convolution layer to learn the weight of each point, and the features of the points are aggregated according to the weight to obtain the global features of the current point cloud, then the clustering layer is used to cluster the points updated after the edge convolution to reduce the number of points; through the above process repeatedly, finally, after the input point cloud is clustered into one point, the features of the final input point cloud are obtained by fusing the global features of multiple levels and the features of the points obtained through the final clustering. The point cloud features extracted by the application can effectively retain the global information and local information of the point cloud, and can improve the point cloud classification precision.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms. Background Technology

[0002] Point cloud data is a dense set of points collected directly or indirectly through certain measurement methods, conforming to measurement rules and capable of characterizing the surface characteristics of a target. It is the third type of spatial data after vectors and images, providing the most direct and effective way to represent the three-dimensional real world. The ultimate manifestation of computer vision is three-dimensional vision, and the expression of three-dimensional vision is point clouds. Point cloud processing plays a crucial role in the entire field of three-dimensional vision, involving almost all related areas, such as autonomous driving perception and localization, SLAM, 3D scene reconstruction, AR / VR, SFM, pose estimation, 3D recognition, structured light, stereo vision, 3D measurement, and visual guidance. Point cloud data has the following characteristics: 1) Disorderliness: Changes in the order of point clouds do not affect the objects represented. 2) Unstructured: Point clouds are points distributed in space and cannot be structured like pixels in an image.

[0003] Based on the above two points, point cloud data cannot be directly used for learning and classification using convolutional neural networks to classify image or other structured data. However, as more and more researchers dedicate themselves to point cloud data classification research, several methods for transforming point cloud data have been proposed. Multi-view methods project an unstructured point cloud onto a 2D image by projecting the point cloud shape onto multiple views and extracting view functions, then fusing these functions for accurate shape classification, such as generating a grid rendering using a virtual camera. Voxel-based methods convert the point cloud into a 3D mesh and then apply a 3D convolutional neural network, such as rasterizing the point cloud using voxels before processing it with a convolutional network. These transformation methods, which convert point cloud data into different formats before processing, result in information loss in the point cloud data and unnecessarily increase the computational load. Therefore, how to directly utilize convolutional networks to process raw point cloud data is increasingly attracting the attention of researchers.

[0004] PointNet utilizes convolutional networks to process raw point cloud data, directly using the point cloud as input to achieve permutation invariance of symmetric functions. Specifically, PointNet uses several MLP layers to independently learn point-state features and uses max-pooling layers to extract global features. Deep sets achieve permutation invariance by summing all representations and applying nonlinear transformations. Since each point in PointNet learns features independently, it cannot capture local structural information between points. Therefore, researchers proposed a hierarchical network, PointNet++, to capture fine geometric structures from the neighborhood of each point. The ensemble abstraction layer is the core of the PointNet++ hierarchical structure, consisting of three layers: a sampling layer, a grouping layer, and a PointNet-based learning layer. By stacking multiple ensemble abstraction layers, PointNet++ learns features from local geometric structures and abstracts local features layer by layer.

[0005] The DGCNN network proposes a novel operation, EdgeConv, which can operate directly on the point cloud. EdgeConv captures local geometric information while ensuring permutation invariance. The entire network represents the point cloud data using EdgeConv by continuously stacking the representation results of each EdgeConv layer. By continuously recalculating the nearest neighbors of each point in the feature space of each layer, the local structure in the feature space is made different from the local structure in the input space. The graph structure processed by each layer is a dynamically defined local region based on a certain distance metric.

[0006] However, current deep learning-based point cloud feature extraction and classification algorithms often suffer from two main problems: (1) The algorithm often takes the entire point cloud data as input, which results in a large computational load. Due to the improved data accuracy of existing lidar and other technologies, the data volume of a point cloud target is often quite large. If all data points of the point cloud target are taken as input, the computational load of the entire algorithm will be quite large. Furthermore, some networks need to downsample the point cloud to a fixed small size, and this sampling process will discard the global information of the point cloud data.

[0007] (2) The algorithm often uses multiple layers of convolution to continuously operate and finally extract the deep features of each point. These features often aggregate a wider range of global information, which makes the local feature information of each point submerged. Summary of the Invention

[0008] To address the problems of high computational cost and insufficient feature extraction in existing point cloud feature extraction and classification algorithms, this invention proposes a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms. This method utilizes edge convolution operations to extract deep-level features of each point, employs clustering to continuously aggregate points and reduce computational cost, and utilizes attention mechanisms to obtain global features of the point cloud at each level. Finally, after clustering the point cloud into a single point, the global features from different levels output by each attention module and the final point's features are fused to obtain target point cloud features that effectively preserve both local and global information of the point cloud data. This improves the accuracy of point cloud data classification and reduces computational cost.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: This invention proposes a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms, comprising: Construct a point cloud feature extraction network based on edge convolutional network, clustering network and attention mechanism; Feature extraction is performed on point cloud data based on the constructed point cloud feature extraction network.

[0010] Furthermore, before performing feature extraction on the point cloud data based on the constructed point cloud feature extraction network, the following steps are also included: The point cloud feature extraction network is trained based on the point cloud dataset used for classification.

[0011] Furthermore, after extracting features from the point cloud data based on the constructed point cloud feature extraction network, the process also includes: Classification is performed based on the features extracted from each point cloud data.

[0012] Furthermore, the point cloud feature extraction network includes multiple edge convolutional layers, multiple clustering layers, and multiple attention modules, and the number of edge convolutional layers, clustering layers, and attention modules is the same.

[0013] Furthermore, during the convolutional computation of point cloud data, after each edge convolutional layer, an attention module is used to learn the different weights of each point in the point cloud, and then the features of each point are aggregated according to the weights to obtain the global features of the point cloud at the current level.

[0014] Furthermore, during the convolution calculation of point cloud data, after each edge convolution layer, a clustering layer is used to cluster the points of the target object into a fixed number of points. This clustering reduces the number of points and the amount of computation.

[0015] Furthermore, the point cloud feature extraction network repeatedly uses edge convolutional layers, clustering layers, and attention modules. It continuously uses edge convolutional layers to extract deep local features of each point in the point cloud, uses clustering layers to continuously reduce the number of points, and continuously uses attention modules to output global features of the point cloud at different levels until the point cloud is finally clustered into 1 point.

[0016] Furthermore, after clustering the point cloud targets into a single point using edge convolutional layers, clustering layers, and attention modules, feature concatenation is used to fuse the global point cloud features from multiple levels of output with the features of the final single point output, resulting in the final feature of the point cloud target. This feature integrates global point cloud features from multiple granularities.

[0017] Furthermore, in order to reduce the computational cost of the attention mechanism, a 1D convolution operation is used to project each point of the point cloud into a scalar. Then, a softmax operation is used to obtain the weights of each point. Finally, the weights are multiplied by the feature vectors of each point to obtain the global features of the point cloud at the current level.

[0018] Furthermore, to avoid the high computational cost of traditional clustering algorithms, a projection matrix is ​​used to learn the probability that each point belongs to a new point in the latent space. Then, a softmax operation is used to normalize the probability that each point belongs to the same point in the new space. Finally, the normalized probabilities are multiplied with the features of each point in the original point cloud and aggregated to obtain the features of each point in the latent space.

[0019] Furthermore, in this method, when point cloud data passes through the edge convolutional layers in the point cloud feature extraction network, it can retain the local information between points. After the point cloud data is processed by each edge convolutional layer, it passes through a clustering layer to cluster the points of the point cloud into a fixed number of points, thereby reducing the number of points. After each edge convolutional layer, an attention module is set to learn the weights of each point, and then the global features of the point cloud at the current level are obtained by aggregating the weights. After the number of points in the point cloud is clustered into 1 by the clustering layer, the global features of the point cloud output from different attention modules and the features of the final point are fused by feature concatenation to obtain the final feature vector representing the entire point cloud.

[0020] Compared with the prior art, the present invention has the following beneficial effects: This invention establishes a point cloud feature extraction network structure based on edge convolution, clustering, and attention mechanisms. It uses edge convolution to extract deep features of each point, uses clustering to continuously aggregate points to reduce computation, and uses attention mechanisms to obtain global features of the point cloud at each level. Finally, after clustering the point cloud into a single point, it fuses the global features at different levels output by each attention module with the features of the final point to obtain the features of the target point cloud that effectively preserves both local and global information of the point cloud data, thereby improving the classification accuracy of point cloud data and reducing computation. Attached Figure Description

[0021] Figure 1 This is one of the flowcharts of a point cloud feature extraction method based on edge convolutional network, clustering network and attention mechanism according to an embodiment of the present invention; Figure 2 This is a second flowchart of a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to an embodiment of the present invention. Figure 3 This is the third flowchart of a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the point cloud feature extraction network constructed according to an embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments: like Figure 1 As shown, a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms includes: Construct a point cloud feature extraction network based on edge convolutional network, clustering network and attention mechanism; Feature extraction is performed on point cloud data based on the constructed point cloud feature extraction network.

[0023] Furthermore, such as Figure 2 As shown, before performing feature extraction on point cloud data based on the constructed point cloud feature extraction network, the following steps are also included: The point cloud feature extraction network is trained based on the point cloud dataset used for classification.

[0024] Furthermore, such as Figure 3 As shown, after extracting features from the point cloud data based on the constructed point cloud feature extraction network, the process also includes: Classification is performed based on the features extracted from each point cloud data.

[0025] Furthermore, such as Figure 4As shown, the point cloud feature extraction network includes multiple edge convolutional layers, multiple clustering layers, and multiple attention modules, and the number of edge convolutional layers, clustering layers, and attention modules is the same.

[0026] Furthermore, during the convolutional computation of point cloud data, after each edge convolutional layer, an attention module is used to learn the different weights of each point in the point cloud, and then the features of each point are aggregated according to the weights to obtain the global features of the point cloud at the current level.

[0027] Furthermore, during the convolution calculation of point cloud data, after each edge convolution layer, a clustering layer is used to cluster the points of the target object into a fixed number of points. This clustering reduces the number of points and the amount of computation.

[0028] Furthermore, the point cloud feature extraction network repeatedly uses edge convolutional layers, clustering layers, and attention modules. It continuously uses edge convolutional layers to extract deep local features of each point in the point cloud, uses clustering layers to continuously reduce the number of points, and continuously uses attention modules to output global features of the point cloud at different levels until the point cloud is finally clustered into 1 point.

[0029] Furthermore, after clustering the point cloud targets into a single point using edge convolutional layers, clustering layers, and attention modules, feature concatenation is used to fuse the global point cloud features from multiple levels of output with the features of the final single point output, resulting in the final feature of the point cloud target. This feature integrates global point cloud features from multiple granularities.

[0030] Furthermore, in order to reduce the computational cost of the attention mechanism, a 1D convolution operation is used to project each point of the point cloud into a scalar. Then, a softmax operation is used to obtain the weights of each point. Finally, the weights are multiplied by the feature vectors of each point to obtain the global features of the point cloud at the current level.

[0031] Furthermore, to avoid the high computational cost of traditional clustering algorithms, a projection matrix is ​​used to learn the probability that each point belongs to a new point in the latent space. Then, a softmax operation is used to normalize the probability that each point belongs to the same point in the new space. Finally, the normalized probabilities are multiplied with the features of each point in the original point cloud and aggregated to obtain the features of each point in the latent space.

[0032] As one possible implementation method, the following specific example is provided: Step S1: Construct a point cloud feature extraction network based on edge convolutional network, clustering network and attention mechanism; the point cloud feature extraction network includes multiple edge convolutional layers, multiple clustering layers and multiple attention modules, and the number of edge convolutional layers, clustering layers and attention modules is the same.

[0033] Step S2: Train the point cloud feature extraction network based on the point cloud dataset used for classification. As one possible implementation, the constructed point cloud feature extraction network is trained using the ModelNet40 dataset. The ModelNet40 dataset is a point cloud dataset for classification, containing 40 categories and a total of 12,311 point cloud data points. Based on preprocessing, training and validation sets are set up: the training set has 9,843 point cloud data points, and the validation set has 2,468 point cloud data points. Each point cloud data point contains 10,000 points.

[0034] Step S3: Input the point cloud data in the training set into the point cloud feature extraction network in batches, and use the edge convolutional layer to extract the features of the point cloud data. Specifically, the edge convolutional layer uses the KNN algorithm to obtain each K nearest neighbor points, and then, based on the edge relationship between each point and its nearest neighbors, it fuses and updates the features of each point to obtain a new feature vector for each point in the point cloud.

[0035] Step S4: Using the feature vector of the point cloud data obtained in step S3 as input, the clustering layer is used to aggregate the feature information of the points into a fixed number of points in the latent space. The features of each new point are aggregated with the features of the input points, thus reducing the number of points.

[0036] Step S5: Add an attention module after each edge convolutional layer. The feature vector of the point cloud data obtained in step S4 is used as input. The main function of the attention module is to calculate the weight of each point and obtain the new feature vector of the point cloud based on the weight between points.

[0037] Step S6: Repeatedly use the edge convolutional layer, clustering layer and attention module to continuously extract the deep features of each point, reduce the number of points, and output the global features of the point cloud at multiple levels until the number of points in the point cloud is clustered into 1 through the clustering layer.

[0038] Step S7: Finally, the point cloud feature extraction network fuses the global point cloud features output by each attention module with the point cloud features obtained by the last clustering layer, and uses this as the final output of the point cloud features.

[0039] Step S8: Utilize the point cloud feature vectors output in Step S7 for learning, continuously backpropagating to update parameters. The final network is obtained through learning and validated on the validation set. Experiments show that this effectively improves the classification accuracy of point cloud data and significantly reduces computational load.

[0040] In summary, this invention proposes a point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms. This method can minimize information loss while reducing the number of points through clustering. Furthermore, the point cloud feature vectors obtained through edge convolutional networks and attention mechanisms can preserve both local and global information of the point cloud, thereby effectively improving the final point cloud classification accuracy.

[0041] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms, characterized in that, include: Construct a point cloud feature extraction network based on edge convolutional network, clustering network and attention mechanism; The point cloud feature extraction network comprises multiple layers connected in sequence. Each layer includes: an edge convolutional layer for extracting local features of each point in the point cloud; a clustering layer for clustering the points in the point cloud into a fixed number of points; and an attention module for learning the weights of each point and aggregating them into global features of the current layer. The point cloud feature extraction network is trained based on the point cloud dataset used for classification; Feature extraction is performed on point cloud data based on a pre-trained point cloud feature extraction network; Classification is performed based on the features extracted from each point cloud data; When point cloud data passes through the edge convolutional layers in the point cloud feature extraction network, it can retain the local information between points. After each edge convolutional layer, the point cloud data goes through a clustering layer to cluster the points in the point cloud into a fixed number of points. Clustering reduces the number of points. After each edge convolutional layer, an attention module is set to learn the weights of each point. Then, the global features of the point cloud at the current level are obtained by aggregating the weights. After the number of points in the point cloud is clustered into 1 by the clustering layer, the global features of the point cloud output from different attention modules and the features of the final point are fused by feature concatenation to obtain the final feature vector representing the entire point cloud.

2. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 1, characterized in that, The point cloud feature extraction network includes multiple edge convolutional layers, multiple clustering layers, and multiple attention modules, and the number of edge convolutional layers, clustering layers, and attention modules is the same.

3. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 2, characterized in that, After each edge convolutional layer, an attention module learns different weights for each point in the point cloud, and then aggregates the features of each point according to the weights to obtain the global features of the point cloud at the current level.

4. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 2, characterized in that, After each edge convolutional layer, a clustering layer is used to cluster the points of the target object into a fixed number of points.

5. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 2, characterized in that, The point cloud feature extraction network repeatedly uses edge convolutional layers, clustering layers, and attention modules. It continuously uses edge convolutional layers to extract deep local features of each point in the point cloud, uses clustering layers to continuously reduce the number of points, and continuously uses attention modules to output global features of the point cloud at different levels until the point cloud is finally clustered into 1 point.

6. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 5, characterized in that, After clustering point cloud targets into a single point by applying edge convolutional layers, clustering layers, and attention modules, the feature concatenation method is used to fuse the global features of the point cloud from multiple levels and the features of the single point output to obtain the final features of the point cloud targets.

7. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 1, characterized in that, A 1D convolution operation is used to project each point of the point cloud into a scalar. Then, a softmax operation is used to obtain the weights of each point. Finally, the weights are multiplied by the feature vectors of each point to obtain the global features of the point cloud at the current level.

8. The point cloud feature extraction method based on edge convolutional networks, clustering networks, and attention mechanisms according to claim 1, characterized in that, Using the projection matrix, the probability that each point belongs to a new point in the latent space is learned. Then, the softmax operation is used to normalize the probability that each point belongs to the same point in the new space. Finally, the normalized probabilities are multiplied with the features of each point in the original point cloud and aggregated to obtain the features of each point in the latent space.

Images