3D point cloud data classification method based on hierarchical graph convolutional network and spatial attention mechanism

Abstract

The present invention discloses a three-dimensional point cloud data classification method based on a hierarchical graph convolution network and a spatial attention mechanism, which belongs to the field of three-dimensional point cloud classification and includes a hierarchical graph convolution module, a spatial attention module, and a similarity comparison module. Adjacent points are obtained based on multiple fields of view and different density sparsity, and new feature vectors are obtained by combining the point cloud features themselves in a graph convolution manner, and the spatial attention mechanism is injected to obtain the feature vector of the new node; the receptive field is expanded, and the neighboring nodes similar to the new node are merged according to the structural similarity on the basis of the new node to obtain multi-level super-node information, reduce the redundancy of local information and strengthen local features. The normal vector features are calculated and added to the global features of the point cloud to improve the overall feature information of the point cloud. The present invention collects multi-level neighbor node features and combines spatial attention to achieve the purpose of strengthening local features, which can further improve the robustness of the three-dimensional point cloud processing and classification network.

Description

Technical Field

[0001] The present invention mainly relates to the field of deep learning, and in particular to a three-dimensional point cloud data classification method based on a hierarchical graph convolutional network and a spatial attention mechanism. Background Art

[0002] There are many methods for classifying 3D point clouds. In the early work, 3D point cloud data was transformed through multi-view and voxelization methods to indirectly learn features. With the development of 3D radar technology, 3D point cloud data has gradually become the main representation method of 3D data due to its easy processing and high precision. The data format of 3D point cloud generally contains 3D position coordinates (x, y, z) and a vector attribute, such as color, reflection intensity, etc. [1] et al. used point cloud data as direct input for feature learning for the first time, which reduced a lot of early data preprocessing work. At the same time, compared with multi-view, voxelization and other methods, it reduced the problem of feature loss in data conversion. 3D point cloud can be applied to unmanned vehicle projects. By autonomously collecting and producing high-precision maps to record complete 3D road information, it can achieve vehicle positioning with centimeter-level accuracy. Based on the advantages of 3D point cloud data, 3D point cloud data is also often used for target recognition. [2] , indoor navigation [3] and other fields.

[0003] Among the point cloud classification methods based on perceptron and feature pooling, PointNet++ [4] There are multiple scale operations in the point cloud, which will cause information redundancy in local areas with little information. To solve this problem, KOMARIICHEVA et al. [5] A circular convolution (A-CNN) was proposed. This new convolution operator can better capture the local geometric features of each point by specifying the ring structure and direction in the calculation, but it does not fully consider the sparsity of point cloud data. In recent years, deep learning has been widely used in the field of graph neural networks. Graph neural networks based on deep learning can process unstructured data and have made breakthroughs in point cloud classification and segmentation. [6] The network for data enhancement based on point cloud shape rotation does not take into account the rotation invariance of point clouds. To solve this problem, CHEN et al. [7] A deep hierarchical clustering network is proposed, which uses hierarchical clustering to explore the geometric structure of point clouds in a hierarchical tree and improves the robustness of rotation transformation.

[0004] This paper optimizes the feature extraction method for point cloud data with varying sparsity levels in circular convolution. Drawing on the idea of ​​hierarchical clustering, this paper injects an improved spatial attention mechanism for classification task training. This improves the ability to extract local features from point clouds and enhances the robustness of point cloud classification.

[0005] References:

[0006] [1] C.R. Qi, H. Su, K. Mo, and L.J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 77 - 85, read.

[0007] [2] Z. Ren and E.B. Sudderth. 3D object detection with latent support

[0008] surfaces. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.

[0009] [3] Lu, W., et al. “L3 - Net: Towards Learning Based LiDAR Localization for Autonomous Driving.” 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR) IEEE, 2020.

[0010] [4] QI C R, YI L, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space[EB / OL].[2022 - 04 - 07].https: / / arxiv.org / pdf / 1706.02413.pdf.

[0011] [5]KOMARICHEV A, ZICHUN Z, JING HA-cnn: annularly convolutional neural networks on point clouds[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2019.

[0012] [6] JOSEPH R, MOR AZ, RON K.Momen^et: flavor the moments in learning to classify shapes[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision Workshops, 2019.

[0013] [7]CHEN C, LI G, XU R, et al. Clusternet: deep hierarchical clusternetwork with rigorous rotation-invariant representation for pointcloud analysis[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2019. Summary of the Invention

[0014] The technical problem to be solved by the present invention is how to enhance the local features of three-dimensional point cloud classification and increase the accuracy of three-dimensional point cloud classification.

[0015] In order to solve the above problems, the present invention is achieved and solved by the following technical means:

[0016] A three-dimensional point cloud data classification method based on a hierarchical graph convolutional network and a spatial attention mechanism includes a hierarchical graph convolution module, a spatial attention module, and a similarity comparison module. The hierarchical graph convolution module selects K neighboring points of a query point based on multiple views and varying degrees of density and sparsity, and combines these with the query point's own point cloud features via graph convolution to generate a new feature vector. The spatial attention module improves upon the DAnet operation steps, removing the reshape operation that affects shape features and introducing dynamic scaling feature parameters to correlate shapes of distant points. These parameters are then multiplied by the spatial attention matrix to obtain enhanced feature parameters, which are the new node feature vectors. The similarity comparison module performs a KNN query on a new node to obtain its neighboring nodes. Based on the new node, similar neighboring nodes are merged based on structural similarity to obtain multi-level supernode information, reducing information redundancy among similar nodes and enhancing local features. The supernode's normal vector is calculated, and the local features of the supernode with the normal vector features are fused with the global features with the normal vector input to obtain the final point cloud classification result.

[0017] The hierarchical graph convolution module further includes screening K neighboring points with multiple fields of view and different density sparsity. For any given point cloud node, each node has a different degree, and the neighboring points found may be contained in local areas of different sizes, resulting in information redundancy and different densities at different locations. To obtain the feature information of the K neighboring points, the following operations are performed:

[0018] Based on the following definition: The definition of the graph is as follows, G = (V, C), where V is the set of point cloud nodes and C is the set of edges. Each node j corresponds to its feature x j , we can use the matrix X N*T Indicates that C is an N×K adjacency matrix, where N represents the number of nodes, T is the dimension of the node feature vector, and K represents the number of neighboring points.

[0019] For a point cloud node Q = {X, Y, Z}, where {X, Y, Z} represents the three-dimensional coordinates of node Q, two different receptive fields, M1 and M2, are provided. r1 and r2 represent the radii of the different receptive fields, where r1 < r2 and M1 < M2. A KNN algorithm is used to query the neighboring points of point cloud node Q and project them onto the tangent plane of point cloud node Q. Adjacent points within the neighborhood are extracted based on the ratio of the neighborhood point densities of the different receptive fields, and K neighboring points are found.

[0020] Taking the M1 receptive field with a radius of r1 as an example to illustrate the search process of K adjacent points, first calculate the volume ratio of different receptive fields, the calculation formula is Where b1 represents the volume ratio of the receptive field sphere of M1 with radius r1, and b2 represents the volume ratio of the receptive field sphere of M1 with radius r1 to the middle ring of the receptive field of M2 with radius r2. Then calculate the normal vector of the point cloud, the calculation formula is: n=b1×K. i represents the i-th adjacent point, Represents the average coordinate value of the adjacent points around node Q, Nor is the normal vector, and ||n|| represents the norm of n.

[0021] Then calculate the projection vector of the tangent plane of these n adjacent points in the field of view of M1 at point Q. The calculation formula is: Y j =X j -((QX j )·Nor)·Nor, where Y j is the projection vector of the adjacent node j on the tangent plane.

[0022] Select a projection vector Y i Connect Q to get a reference Z = Y i -Q, calculate the angle of the remaining projection vector relative to the reference. The angle calculation formula is where Y j Indicates that except for the projection vector Y i The other projection vector, θ j Indicates that except for the projection vector Y i The angle of other projection vectors. j The range is (0, 2π), the entire range is divided into two regions (0, π) and (π, 2π), and the adjacent points X are randomly selected in these two regions. i , until the number of adjacent points obtained is m, where n represents the number of adjacent points in the field of view of M1.

[0023] The obtained adjacent point vectors can be concatenated to form an array [X1, X2, ...X m ]. Perform the same steps as above for the receptive field sphere of M1 with radius r1 and the middle ring volume of the receptive field of M2 with radius r2, and get the adjacent point vector array [X m+1 , X m+2 ,...X m+s ], where s represents the number of adjacent points obtained from the volume of the intermediate ring of the receptive field sphere M1 with radius r1 and the receptive field M2 with radius r2. By integrating the two adjacent point vector arrays, we can obtain the P adjacent point vector information of the point cloud Q. This method takes into account the density distribution of points and avoids the redundancy of adjacent point information.

[0024] Another operation is graph convolution to extract local features and generate new nodes. The obtained P neighbor point vectors contain local information of the point cloud object. The P neighbor point vector matrices are defined as a set C, where the expression of C is C = {X j : j = 1, 2, 3 ... P}. Where X j Represents the eigenvector of neighboring point j. Based on the graph convolution method, the Laplace matrix is ​​introduced and normalized, and the formula is H l+1 =σ(LH l W l ), H 0 =X, Where σ is a nonlinear activation function, the ReLU activation function is used in the experiment, Wl is the weight parameter matrix of the lth layer, D is its own degree matrix, is the interpolation of the degree matrix and the adjacent point matrix, X is the first layer input, and finally H is obtained l+1 That is the feature vector of the l+1th layer of graph convolution.

[0025] Injecting a spatial attention module on the basis of the new node is characterized by including the following steps:

[0026] Two independent convolution operations are used to generate two new features S and F. In DANet, after obtaining the two new features, a reshape operation is performed, and then the two new features S and F are multiplied.

[0027] Then, we use softmax to get the spatial attention weights, but the reshape operation will lose the spatial feature information of the point cloud. The spatial attention module designed in this paper removes the reshape operation in DANet and directly multiplies the new features S and F to get the weighted attention matrix U, which is then normalized by softmax. The calculation is: Where N represents the number of points, S i , F j Represents the features of point i and point j, U ij Indicates the similarity between point i and point j. The larger the value, the higher the similarity.

[0028] Multiply the feature H by the spatial attention matrix U and the scaling parameter σ, and finally add it to the feature H to obtain the enhanced output feature W, which is the new node feature. σ is a learnable scaling parameter with an initial value of 0 and a maximum learnable value of no more than 1. Setting the learnable scaling parameter can enhance the connection between spatial features of distant previous points, which is conducive to extracting spatial structural features. The enhanced new node feature W is calculated as follows: Among them H j , H iRepresents the features of point j and point i. The introduction of spatial attention mechanism can establish the global structural relationship between points and extract the long-distance and deep semantic information between points.

[0029] Further similarity merging includes the following steps:

[0030] Based on the new node W, a KNN query is performed to find the adjacent point matrix information C, and the local information is further merged through the similarity comparison module. The steps for designing the similarity comparison module are as follows:

[0031] Step 1: Compare the similarity of the distance between adjacent points. First, select an adjacent node C of the new node W. i , for C i Perform kNN query to obtain the adjacency matrix M, calculate the Euclidean distance between node W and each node in the adjacency matrix C, and calculate C i The sum of the Euclidean distances of each node in the adjacency matrix M is obtained by dividing the two distance sums to obtain S1.

[0032] Step 2: Weight and degree similarity comparison is divided into two categories. One is completely identical neighbor nodes, which can directly get S1=1; the other requires comparing the spatial structure of local features and merging nodes according to the similarity of the structure. The similarity comparison formula is: in Indicates W, C i The weight of the edges, Represents nodes W, C i degree.

[0033] Step 3: Assign different weights to the obtained S1 and S2 and add them together. The calculation formula is S = α·S1+β·S2. When s is greater than a certain set value, merge W and C. i The two nodes are merged into a new supernode. The supernode information obtained after merging can sense the information of nodes that are farther away and has richer local feature information.

[0034] Next, feature fusion is performed, the normal vector of the super node is calculated, the global feature is calculated through MLP, the local features of the super node, the normal vector, and the global feature are fused for training, and finally the classification result of the point cloud is obtained.

[0035] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0036] 1. Based on the irregular characteristics of 3D point cloud data, a multi-layered graph convolution module is proposed to provide different receptive fields. This takes into account the differences in sparsity of different 3D point cloud data. Even for point cloud data with large differences in sparsity, relatively complete local features can be more accurately found based on the receptive field ratio and sparsity. Providing normal vector information on this basis can further enrich local feature information. Through this improvement, point cloud features can be learned more efficiently and accurately.

[0037] 2. Based on an improved spatial attention mechanism and similarity comparison merging method. By improving the DAnet operation steps and dynamically introducing scaling parameters to obtain a spatial attention map, we can consider the spatial structural connections between points at different distances. On this basis, we introduce a similarity comparison merging mechanism to merge nodes with high similarity, reduce the impact of duplicate points, and improve the ability to obtain local features.

[0038] 3. The proposed algorithm performs 3D point cloud classification on the ModelNet10 and ModelNet40 datasets, achieving significantly improved classification accuracy compared to previous algorithms. See Tables 1 and 2 for details. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is the overall process architecture diagram of the present invention.

[0040] Figure 2 Schematic diagram of multi-layer graph convolution neighbor point selection in the present invention.

[0041] Figure 3 Schematic diagram of the spatial attention mechanism module in the present invention.

[0042] Figure 4 Schematic diagram of the similarity comparison module in the present invention.

[0043] Figure 5 Schematic diagram of PointNet network architecture. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] The present invention proposes a three-dimensional point cloud data classification method based on a hierarchical graph convolutional network and a spatial attention mechanism. Through a hierarchical graph convolution module, a spatial attention module, and a similarity comparison module, the purpose of reducing information redundancy and enhancing local features is achieved.

[0046] Step 1: Perform a hierarchical graph convolution module operation. This operation includes screening K neighboring points of different density and sparsity based on the multi-view of the query point, and combining the point cloud features of the query point itself in a graph convolution manner to obtain a new feature vector.

[0047] Based on the following definition: The definition of the graph is as follows, G = (V, C), where V is the set of point cloud nodes and C is the set of edges. Each node j corresponds to its feature x j , we can use the matrix X N*T Indicates that C is an N×K adjacency matrix, where N represents the number of nodes, T is the dimension of the node feature vector, and K represents the number of neighboring points.

[0048] For any given point cloud node Q, Q∈V, take node Q as the center node, calculate the distance from other point cloud nodes to node Q, and find the farthest distance d max and the closest distance d min Randomly select two nodes, the two selected nodes must satisfy πd min 3 =απ(d max 3 -d min 3 ), 0<α<2. If the above conditions are not met, first fix one point and continue to select another point until the above conditions are met. The distances between the two selected nodes and node Q are r1 and r2. This ensures that the difference between the two different receptive fields M1 and M2 is relatively small.

[0049] Perform a KNN neighboring point query on node Q within the receptive fields M1 and M2, projecting the points onto the tangent plane of the point cloud node Q. Based on the ratio of the neighborhood point density in different receptive fields, K neighboring points are extracted. In practice, K is set to 15, 20, 25, or 30, respectively.

[0050] 1. According to the volume ratio of receptive field M1 and receptive field M2, calculate the density ratio of different receptive field points. The calculation formula is: b1 represents the volume ratio of the receptive field sphere of M1 with radius r1, and b2 represents the volume ratio of the receptive field sphere of M1 with radius r1 and the middle ring of the receptive field M2 with radius r2.

[0051] 2. Calculate the normal vector of node Q, perform plane fitting on the adjacent points, and use the least squares method to obtain the fitted plane. The vector of the fitted plane is the normal vector. The calculation formula is: where x i represents the i-th adjacent point, Represents the average coordinate value of the adjacent points around node Q, Nor is the normal vector, and ||n|| represents the norm of n.

[0052] 3. Calculate the projection vector of the tangent plane of these n adjacent points in the field of view of M1 at point Q. Projection processing is performed here to project the three-dimensional space data into two-dimensional space data. The density of points in different receptive fields in the three-dimensional space can be estimated based on the number of points in the plane. The calculation formula is: Y j =X j -((QX j )·Nor)·Nor, where Y j is the projection vector of the adjacent node j on the tangent plane.

[0053] 4. Select a projection vector Y i Connect Q to get a reference Z = Y i -Q, calculate the angle of the remaining projection vector relative to the reference. The angle calculation formula is where Y j Indicates that except for the projection vector Y i The other projection vector, θ j Indicates that except for the projection vector Y i The angle of other projection vectors. j The range is (0, 2π), and the entire range is divided into two regions (0, π) and (π, 2π). Adjacent points Xi are randomly selected in these two regions in turn until the number of adjacent points obtained is m, where n represents the number of adjacent points in the field of view of M1.

[0054] 5. Concatenate the adjacent point vectors obtained from the receptive field M1 and express them as an array [X1, X2, ...X m ].

[0055] 6. For the receptive field M2, execute the above steps 3, 4, and 5 to obtain the adjacent vector array [X m+1 , X m+2 ,...X m+s ], where s represents the number of adjacent points obtained in the volume of the middle ring of the receptive field sphere M1 with radius r1 and the receptive field M2 with radius r2.

[0056] 7. Concatenate the adjacent point vector arrays obtained in steps 5 and 6 to obtain [X1, X2, ...X m ,...X m+s ]. The m+s adjacent points obtained by this method take into account the density distribution of points and avoid the redundancy of adjacent point information. Note that P=m+s.

[0057] 8. The P neighbors obtained in step 7 contain the local information of node Q. Define the P neighbor point vector matrix as set C, where the expression of C is C = {X j: j = 1, 2, 3 ... P}. Where X j Represents the eigenvector of neighboring point j.

[0058] 9. For the vector matrix of adjacent points, normalize the operation. Here we use the mainstream Laplace matrix normalization method. The calculation formula is Where D is its own degree matrix, For the interpolation of the degree matrix and the adjacency matrix, for a single node operation, normalization is to divide it by the degree of its node, so that the value of each adjacent edge information is normalized.

[0059] 10. Using graph convolution, introduce the Laplace matrix and normalize it. The formula is H l+1 =σ(LH l W l ), H 0 =X, Where σ is a nonlinear activation function, the ReLU activation function is used in the experiment, W l is the weight parameter matrix of the lth layer, D is its own degree matrix, is the interpolation of the degree matrix and the adjacent point matrix, X is the first layer input, and finally H is obtained l+1 That is the feature vector of the l+1th layer of graph convolution.

[0060] Step 2: Inject the spatial attention module based on the new node. The spatial features of the point cloud can better reflect the category features of the point cloud. For the feature H obtained in step 1, two independent convolution operations are used to generate two new features S and F. In DANet, after obtaining the two new features, a reshape operation will be performed. Then the two new features S and F are multiplied together, and then the spatial attention weights are obtained through softmax. However, the reshape operation will lose the spatial feature information of the point cloud. The spatial attention module designed this time deletes the reshape operation in DANet and directly multiplies the new features S and F to obtain the weighted attention matrix U. It is then normalized through softmax and calculated as: Where N represents the number of points, S i , F j Represents the features of point i and point j, U ij Indicates the similarity between point i and point j. The larger the value, the higher the similarity.

[0061] Multiply the feature H by the spatial attention matrix U and the scaling parameter σ, and finally add it to the feature H to obtain the enhanced output feature W, which is the new node feature. σ is a learnable scaling parameter with an initial value of 0 and a maximum learnable value of no more than 1. Setting the learnable scaling parameter can enhance the connection between spatial features of distant previous points, which is conducive to extracting spatial structural features. The enhanced new node feature W is calculated as follows: Among them H j , H i Represents the features of point j and point i. The introduction of a spatial attention mechanism establishes global structural relationships between points, extracting long-range and deep semantic information between them. Set σ to 0 initially and increase by 0.1 until it reaches 0.5 and then by 0.01. The introduction of a spatial attention mechanism establishes global structural relationships between points, extracting long-range and deep semantic information between them.

[0062] Step 3: Introduce the similarity comparison module to further increase the receptive field range and enhance local features.

[0063] Based on the new node W, a KNN query is performed to find the adjacent point matrix information C, and the local information is further merged through the similarity comparison module. The steps for designing the similarity comparison module are as follows:

[0064] 1. Comparison of the similarity of adjacent point distances. First, select an adjacent node C of the new node W. i , for C i Perform kNN query to obtain the adjacency matrix M, calculate the Euclidean distance between node W and each node in the adjacency matrix C, and calculate C i The sum of the Euclidean distances of each node in the adjacency matrix M is obtained by dividing the two distance sums to obtain S1.

[0065] 2. Weight and degree similarity comparison is divided into two categories. One is completely identical neighbor nodes, which can directly obtain S1 = 1; the other requires comparing the spatial structure of local features and merging nodes based on structural similarity. The similarity comparison formula is: in Indicates W, C i The weight of the edges, Represents nodes W, C i degree.

[0066] 3. Assign different weights to the obtained S1 and S2 and add them together. The calculation formula is S = α·S1+β·S2. When s is greater than a certain set value, merge W and C. iThe two nodes are merged to form a new supernode. The supernode information obtained after merging can sense the information of nodes that are farther away and has richer local feature information. For simplicity and convenience, the weight and degree similarity are considered to have higher priority. Set α to 0.3 and β to 0.7. When S is greater than 0.8, the two nodes are considered similar. Then for W, C i After normalization, the average value of the node information is taken to form a new supernode. The convolution operation is performed to obtain the features of the supernode, and finally the n×128 is obtained through the convolution operation.

[0067] Step 4: Combine the enhanced local features with the global features to perform point cloud classification.

[0068] In the PointNet network, a transformation matrix is ​​first predicted, the transformation is performed, and then an MLP is used to embed each point. The transformation matrix is ​​then predicted again in the feature space, the transformation is performed, and the embedding is performed again. Finally, maxpooling is used to obtain global features. This only solves the most basic problem of satisfying the properties of point clouds and only captures global information.

[0069] This design improves some operations of the pointNet network. In terms of input data format, the normal vector information of the point is first calculated. According to the method in step 1, the adjacent points are fitted with a plane. The least squares method is used to obtain the fitted plane, and the fitted plane vector is used to obtain the normal vector feature. Secondly, the twice predicted transformation matrix and transformation operation are removed, and three MLP operations are directly performed on the point cloud. The number of sampling points is set to n=256, 512, 1024, 2048, and the batch_size is 8, 16, 32, and 64 respectively. The initial input point cloud information is 256×4, 512×4, 1024×4, and 2048×4 respectively. After the MLP operation of 64, 128, and 1024 dimensions, the global features of n×1024 are obtained. The local features of the super nodes are connected to obtain the feature matrix of n×1152. Finally, the fused features are classified through the fully connected layer.

[0070] Experiments were conducted on an Ubuntu 18.04 system, using an NVIDIA GeForce RTX 3060 12GB GPU and CUDA version 10.1. The number of neighboring nodes, K, was set to 10, 15, 20, and 25. Learning rate decay was applied at iterations 30, 70, 100, and 120, with decay rates set to 0.1, 0.1, 0.2, and 0.2, respectively. The initial learning rate was 0.001. To prevent overfitting, batch normalization and a dropout layer with a parameter of 0.3 were added to each fully connected layer.

[0071] The following table shows the three-dimensional point cloud classification results obtained by the present invention and other methods.

[0072] Table 1. ModelNet40 classification results

[0073] Model Input format Average precision Overall accuracy PointNet (x,y,z) 86.2 89.2 PointNet++ (x,y,z) 89.4 90.7 PointCNN (x,y,z) 88.1 92.1 A-CNN (x,y,z,n) 88.8 92.3 This invention (x,y,z,n) 90.2 92.9

[0074] Table 2. ModelNet10 classification results

[0075] Model Input format Average precision Overall accuracy KD-Net (x,y,z) 93.5 94.0 PCNN (x,y,z) 94.0 94.7 PointASNL (x,y,z) 94.4 95.2 A-CNN (x,y,z,n) 94.3 94.9 This invention (x,y,z,n) 94.8 95.5

[0076] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, those skilled in the art may make various specific modifications based on the teachings of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A 3D point cloud data classification method based on a hierarchical graph convolutional network and a spatial attention mechanism, characterized by: It includes a hierarchical graph convolution module, a spatial attention module, and a similarity comparison module. The hierarchical graph convolution module includes the screening of K neighboring points based on multiple views of the query point and different densities and sparsities, and combines the point cloud features of the query point itself with the graph convolution method to obtain a new feature vector. The spatial attention module includes improving the DAnet operation steps, removing the reshape operation that affects the shape features, introducing dynamic scaling feature parameters to associate the shapes of distant points, and multiplying them with the spatial attention matrix to obtain enhanced feature parameters, which are the new node feature vectors. The similarity comparison module includes performing KNN query on the new node to obtain the neighbor nodes of the new node. Based on the new node, the neighbor nodes similar to the new node are merged according to the structural similarity to obtain multi-level super node information, reducing the information redundancy of similar nodes and strengthening local features. Calculate the normal vector of the super node, fuse the local features of the super node with the normal vector feature and the global features with the normal vector information input, and finally obtain the classification result of the point cloud; Step 1: Filter K neighboring points with multiple views and different densities and sparsities of the query point: For any given point cloud node, the degree of each node is different, and the neighboring points to be searched may be contained in local areas of different sizes, resulting in information redundancy and different densities of points at different locations. In order to obtain the feature information of K neighboring points, the following operations are performed: The definition of the graph is as follows, G = (V, C), where V is the set of point cloud nodes and C is the set of edges; each node j corresponds to its feature x j , using the matrix X N*T Indicates that C is an N×K adjacency matrix, where N represents the number of nodes, T is the dimension of the node feature vector, and K represents the number of neighboring points; For the point cloud node Q = {X, Y, Z}, where {X, Y, Z} represent the three-dimensional coordinate values of the node Q respectively, two different receptive fields M1 and M2 are provided, and r1 and r2 represent the radii of different receptive fields, where r1 < r2 and M1 < M2; the neighboring points of the point cloud node Q are queried through KNN and projected onto the tangent plane of the point cloud node Q, and the adjacent points within the neighborhood are extracted according to the ratio of the neighborhood point densities of different receptive fields, and K neighboring points are found; the process of finding K adjacent points is illustrated with the receptive field M1 with a radius of r1. First, calculate the volume ratio of different receptive fields. The calculation formula is where b1 represents the volume ratio of the receptive field sphere M1 with a radius of r1, and b2 represents the volume ratio of the middle ring between the receptive field sphere M1 with a radius of r1 and the receptive field M2 with a radius of r2; then calculate the normal vector of the point cloud. The calculation formula is: n = b1 × K; where x i represents the i-th adjacent point, represents the average coordinate value of the adjacent points around the node Q, Nor is the normal vector, and ‖n‖ represents the norm of n; Then calculate the projection vector of the tangent plane of the n adjacent points in the field of view of M1 at the node Q; the calculation formula is: Y i =X i -((QX i )·Nor)·Nor, where Y i is the projection vector of the adjacent node i on the tangent plane, Q = {X, Y, Z} represents the three-dimensional coordinate values ​​of node Q respectively; Step 2: Convolution of adjacent point graph to extract feature operations.

2. The three-dimensional point cloud data classification method based on hierarchical graph convolutional network and spatial attention mechanism according to claim 1 is characterized in that: The following steps are involved: Step 1 also includes: selecting a projection vector Y i Connect Q to get a reference Z = Y i -Q, calculate the angle of the remaining projection vector relative to the reference. The angle calculation formula is where Y j Indicates that except for the projection vector Y i The other projection vector, θ j Indicates that except for the projection vector Y i The angle of other projection vectors; at this time θ j The range is (0,2π), the entire range is divided into two regions (0,π) (π,2π), and the adjacent points X are randomly selected in these two regions. i , until the number of adjacent points obtained is m, where n represents the number of adjacent points in the visual field of M1; The obtained adjacent point vectors are concatenated and represented as an array [X1, X2, ...X m ]; Perform the same steps as above for the receptive field sphere of M1 with radius r1 and the middle ring volume of the receptive field of M2 with radius r2 to obtain the adjacent point vector array [X m+1 ,X m+2 ,...X m+s ], where s represents the number of adjacent points obtained in the receptive field sphere of M1 with radius r1 and the intermediate ring volume of the receptive field M2 with radius r2; integrating the two adjacent point vector arrays can obtain the P adjacent point vector information of the point cloud Q; Step 2 also includes: obtaining the P neighboring point vectors mentioned in step 1 containing local information of the point cloud object, defining the P neighboring point vector matrix as a set C, where the expression of C is C={X j :j=1,2,3...P};where X j Represents the eigenvector of neighboring point j; based on the graph convolution method, the Laplace matrix is ​​introduced and normalized, the formula is H l+1 =σ(LH l W l ), H 0 =X, Where σ is a nonlinear activation function, the ReLU activation function is used in the experiment, W l is the weight parameter matrix of the lth layer, D is its own degree matrix, is the interpolation of the degree matrix and the adjacent point matrix, X is the first layer input, and finally H is obtained l+1 That is the feature vector of the l+1th layer of graph convolution.

3. The three-dimensional point cloud data classification method based on hierarchical graph convolutional network and spatial attention mechanism according to claim 2 is characterized in that: The spatial attention module consists of the following steps: For the obtained feature H, two independent convolution operations are used to generate two new features S and F. In DANet, after obtaining the two new features, a reshape operation is performed, and then the two new features S and F are multiplied together, and then the spatial attention weight is obtained through softmax. However, the reshape operation will lose the spatial feature information of the point cloud. The spatial attention module designed in this paper deletes the reshape operation in DANet and directly multiplies the new features S and F to obtain the weighted attention matrix U. Then it is normalized through softmax and the calculation is: Where N represents the number of points, S i ,F j Represents the features of point i and point j, U ij Indicates the similarity between point i and point j. The larger the value, the higher the similarity. Multiply the feature H by the spatial attention matrix U and the scaling parameter σ, and finally add it to the feature H to obtain the enhanced output feature W, which is the new node feature. σ is a learnable scaling parameter with an initial value of 0, and its maximum learnable value is set to no more than 1. Setting the learnable scaling parameter enhances the connection between the spatial features of the previous points at a longer distance, which is conducive to extracting spatial structural features. The enhanced new node feature W is calculated as follows: Among them H j , H i Represent the features of point j and point i; introduce the spatial attention mechanism to establish the global structural relationship between points and extract the long-distance and deep semantic information between points.

Images