Three-dimensional point cloud classification method based on multi-geometry double-edge attention network

Abstract

The application provides a three-dimensional point cloud classification method based on a multi-geometry double-edge attention network. The network is based on an attention mechanism and comprises a multi-geometry enhancement module, a double-edge attention module and a channel attention module. The attention mechanism enables the network to pay more attention to the more important part of the input data, thereby improving the accuracy of three-dimensional point cloud classification. The multi-geometry enhancement module considers the Euclidean space feature and the geometric feature, obtains the original point cloud feature in different directions, and obtains more geometric clues. The double-edge attention module extracts more meaningful information from the low-level geometric input and appropriately weights the point features to aggregate the point features. The double-edge attention module effectively learns local and global features through two different branches, and finally connects the two branches to aggregate local and global context elements. The channel attention module extracts meaningful information in the final output and appropriately weights the point features to aggregate the point features for the three-dimensional point cloud classification task.

Description

Technical Field

[0001] This invention belongs to the field of point cloud data analysis technology, specifically relating to a three-dimensional point cloud classification method based on a multi-geometric dual-edge attention network. Background Technology

[0002] The acquisition of point clouds has become more convenient due to the development of 3D sensing technology. Generally, 3D point clouds can be collected in natural environments using physical touch or non-contact measurement methods such as LiDAR, RGB-D cameras, and scanners. Point clouds have been widely used in fields such as high-precision large-scale digital elevation models (DEMs), 3D building modeling, land cover classification, object change monitoring, forest resource surveys, biomass estimation, infrastructure mapping, coastal zone surveying, mine surveying, disaster assessment, wetland mapping, urban planning, 3D reconstruction, autonomous driving, and robotics. Point cloud classification is a fundamental task in point cloud data processing. For example, in satellite remote sensing, accurate monitoring requires accurate identification of object categories. In autonomous driving, vehicles need to analyze various objects (e.g., pedestrians and cars) and their motion states in outdoor scenes before making reliable decisions. For robotic grasping tasks, reconstructing and analyzing the 3D objects of the surrounding environment is a prerequisite for navigation and manipulation. Due to its significant research value, point cloud data classification has attracted widespread research interest from numerous scholars.

[0003] Deep learning has become increasingly popular for analyzing point cloud data, such as classification and segmentation, due to its large-scale learning capabilities. Reliable point cloud analysis under deep learning mode has received widespread attention. Unlike 2D images, point clouds are irregular, unstructured, unevenly distributed, and have a large amount of data, making them difficult to process and analyze using conventional convolutional methods. How to effectively learn representational features from unorganized point clouds remains a challenging problem. Currently, people have directly replicated the remarkable success of convolutional neural networks (CNNs) onto point clouds. Many works have explored different methods to improve 3D point cloud processing. For example, the pioneering works PointNet[1] and PointNet++[2] directly process unordered point clouds and achieve effective end-to-end point feature learning. These works have achieved good results, but there are still difficulties in point cloud processing. It is impossible to establish effective local geometric feature correspondences and cannot make good use of the information between points, which leads to low accuracy in point cloud classification and segmentation.

[0004] Since the introduction of PointNet[1], deep learning of 3D point clouds has progressed rapidly. Due to the unique properties of 3D point clouds in N-dimensional space, the network needs to keep their reordering invariant. PointNet[1] and Deep Set[3] first pointed out that this is a potential research problem. They use permutation invariant functions and permutation isovariant functions to process points, mapping the data to symmetric functions to obtain the required data representation. Inspired by PointNet, subsequent research has advanced the PointNet idea and applied it to various fields. Graph Convolutional Neural Network (GCN)[4] is a neural network that extends CNN to graphs. GCN is suitable for processing irregular data structures such as point clouds. In the Graph Neural Network DGCNN[5], the neighborhood points are regarded as a local graph, and the graph is dynamically updated after each layer of the network. Local information is also added to the architecture. This type of network can dynamically view local regions in space, rather than the global coordinates of the entire point cloud. This means that the network only focuses on the local feature mining of the point cloud and ignores the global features.

[0005] In addition, existing 3D point cloud classification methods based on direct point clouds still have some unresolved issues: 1) In addition to 3D coordinates, can more geometric cues be provided for CNN-based feature learning? 2) How can the network automatically learn better representations from abstract high-level feature spaces? 3) To solve the difficulty of feature learning caused by disordered point clouds, many works preprocess 3D point cloud data into 2D images or 3D representations, such as multi-view projection [6] and voxelization [7-9]. However, this inevitably leads to a waste of time and space. Projection methods cause information loss, and determining the optimal number of views is also a problem that needs to be considered. Although voxel methods can directly apply 3D convolution for feature extraction, the increase in resolution will bring greater computational costs. These methods generally have low generalization ability, high computational complexity, and serious loss of inherent geometric information during the conversion process. Therefore, methods that use the original point cloud as input for end-to-end learning have received increasing attention.

[0006] The above references are:

[0007] [1]Qi CR,Su H,Mo K,et al.Pointnet:Deep learning on point sets for 3dclassification and segmentation[C] / / Proceedings of the IEEE conference oncomputer vision and pattern recognition.2017:652-660.

[0008] [2]Qi C R,Yi L,Su H,et al.Pointnet++:Deep hierarchical featurelearning on point sets in a metric space[J].arXiv preprint arXiv:1706.02413,2017.

[0009] [3]Zaheer M,Kottur S,Ravanbakhsh S,et al.Deep sets[J].arXiv preprintarXiv:1703.06114,2017.

[0010] [4]Chiang W L,Liu X,Si S,et al.Cluster-gcn:An efficient algorithm fortraining deep and large graph convolutional networks[C] / / Proceedings of the25th ACM SIGKDD International Conference on Knowledge Discovery&DataMining.2019:257-266.

[0011] [5]Wang Y,Sun Y,Liu Z,et al.Dynamic graph cnn for learning on pointclouds[J].Acm Transactions On Graphics(tog),2019,38(5):1-12.

[0012] [6]Su H,Maji S,Kalogerakis E,et al.Multi-view convolutional neuralnetworks for 3d shape recognition[C] / / Proceedings of the IEEE internationalconference on computer vision.2015:945-953.

[0013] [7]Maturana D,Scherer S.Voxnet:A 3d convolutional neural network for real-time object recognition[C] / / 2015 IEEE / RSJ International Conference onIntelligent Robots and Systems(IROS).IEEE,2015:922-928.

[0014] [8] Klokov, R.; Lempitsky, V. Escape from cells: Deep kd-networks for therecognition of 3d point cloud models. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp.863–872.

[0015] [9] Riegler, G.; Osman, UA; Geiger, A. Octnet: Learning deep 3d representations at high resolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26July2017; pp.3577–3586. Summary of the Invention

[0016] To address the aforementioned problems, this invention provides an end-to-end point cloud analysis method that is more efficient and accurate. The technical solution adopted in this invention is as follows:

[0017] This invention provides a 3D point cloud classification method based on a multi-geometric dual-edge attention network, characterized by the following steps: Step S1, constructing a multi-geometric dual-edge attention network based on an attention mechanism. This network has a multi-geometric enhancement module, repeatedly stacked dual-edge attention modules, and a channel attention module. The attention mechanism includes local attention and global attention. Step S2, inputting the unordered point set of the original point cloud into the multi-geometric dual-edge attention network, and obtaining point cloud features of the original point cloud in different directions through the multi-geometric enhancement module, thereby generating multi-dimensional low-level geometric vectors. Step S3, the dual-edge attention module has an edge feature branch and a density edge attention branch. These two branches generate multi-dimensional low-level geometric vectors. The first step involves taking low-level geometric vectors as input and extracting global and local point features from the original point cloud based on local attention. The extracted features are then connected using skip connections to obtain the final 3D point cloud feature vector. The second step involves the channel attention module taking multiple low-level geometric vectors as input and generating a channel attention mask based on global attention. This channel attention mask is used to scale the 3D point cloud feature vector, further enhancing its feature representation capability. The third step involves the multi-geometric dual-edge attention network using max pooling and average pooling to collect representative feature vectors from the scaled features. The collected feature vectors are then merged element-wise through an MLP layer to obtain the output value, which is used to determine the category of the original point cloud.

[0018] The 3D point cloud classification method based on multi-geometric dual-edge attention network provided by the present invention can also have the following technical features: the multi-geometric enhancement module divides the 3D point cloud space into Euclidean space and feature value space, and obtains similar local structural features in the feature value space based on the local structural features in the Euclidean space, thereby extracting the multi-geometric features of the original point cloud.

[0019] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention may also have the following technical features, wherein step S2 includes the following sub-steps: Step S2-1, based on the 3D coordinates of any point in the Euclidean space of point cloud X, the K-Nearest Neighbor search algorithm is used to search for any point P in point cloud X. i The two nearest neighbor points P i1 and P i2 , respectively represented as Step S2-2, based on two neighboring points P i1 and P i2 With point P i Form a triangular face, thus obtaining the outline edge of the triangle: E1 = P i1 -P i E2 = P i2 -P i Point Pi The normal vector: n = E1 × E2, and the distance between these two points:

[0020] l Eu =|E i |=|P ij -P i |;

[0021] Step S2-3, let M = (E1, E2, ..., E i E k )=(P i1 -P i ,P i1 -P i ,...,P ii -P i ,P ik -P i The three-dimensional structure tensor in the feature space is defined as: T = M × M T Step S2-4: Decompose the three-dimensional structure tensor T into T = RΛR T Where R is a rotation matrix, and Λ is a diagonal and positive definite matrix, called the eigenvector and eigenvalue matrix, respectively; Step S2-5, for each point P i Using three-dimensional structure tensors Point P i Eigenvalues ​​(1≤i≤N, λ) i ∈R 3 ), where the positive eigenvalue λ i ∈R 3 It is ordered, and λ 1 ≥λ 2 ≥λ 3 ≥0; Step S2-6, use the L2 norm to calculate the distance between different points in the feature space. :

[0022]

[0023] Finally, the multi-geometry enhancement module expands the 3D coordinates of each point into a multi-dimensional low-level geometric vector:

[0024]

[0025] In the formula,

[0026] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention also has the following technical features: the local attention of the dual-edge attention module uses an Edgeconv network to aggregate local features to capture local point cloud information, and the KNN algorithm is used to obtain the center point P. iThe neighborhood graph is then used to share parameters through MLP mapping in the feature space, and finally a symmetric function is applied to encode the corresponding region information into the feature of each point.

[0027] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention also has the following technical features, wherein the process of extracting local features from the edge feature branches is as follows: The point features P... i and edge vector P j The feature differences between them are concatenated to form neighborhood information P. j -P i Then, the neighborhood information P j -P i Features P i Aggregation is performed as P i With P j Edge features between e ij :

[0028] e ij =h(P i ,P j -P i )

[0029] In the formula, h() is the aggregation operation, and P i Edge features G i for:

[0030] Edge features G are analyzed based on a multilayer perceptron (MLP), batch normalization layer, and activation function. i Encode:

[0031] M(G i ) = AF(BN(conv 1×1 (G i )))

[0032] In the formula, M represents a multilayer perceptron (MLP), AF is the activation function, BN is batch normalization, conv is convolution, and its subscript indicates the filter size.

[0033] Based on local attention, the encoded edge features G i Perform global point feature extraction:

[0034] f e =La(Max(M(G) i )))

[0035] In the formula, La represents local attention, and Max() represents the max pooling operation. This max pooling operation is a symmetric function that obtains the maximum value on each feature channel through the max-pooling operator. Combining it with local attention La makes it easier to capture the most representative features.

[0036] The final output of the edge feature branch is:

[0037] f E =concat(f e1 ,f e2 ,f e3 ,f e4 )

[0038] In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

[0039] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention also has the following technical features: the density edge attention branch uses density convolution to adaptively aggregate the local information of all neighbors, thereby obtaining more fine-grained features from the edge convolution. The feature extraction process is as follows: the density convolution is used as a 1×k convolution on the local region, the weights of k neighbors in the region are learned, and finally the weighted sum of the features of the k neighbors is aggregated as a detailed local context, as shown below:

[0040] D(G i ) = AF(BN(conv 1×k (G i )))

[0041] In the formula, D represents density convolution. Compared to 1×1 convolution, the kernel size of density convolution is 1×k, where k is the size of k in the k-nearest neighbor algorithm. The squeeze operation removes dimensions with only one channel, and local attention is applied. The output feature is represented as:

[0042] f d =La(squeeze(D(G) i )))

[0043] The final output of the density edge attention branch is:

[0044] f D =concat(f d1 ,f d2 ,f d3 ,f d4 )

[0045] In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

[0046] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention also has the following technical features, wherein the process of obtaining the channel attention mask in step S4 is as follows:

[0047] The channel attention module will output the multi-geometric features, i.e., multi-dimensional low-level geometric vectors, from the multi-geometric enhancement module. As input, two 2D convolutional layers with a kernel size of 1×1 are used to reduce the dimensionality of the geometric feature, and a sigmoid function is used to generate the channel attention mask Ga, as follows:

[0048]

[0049] Ga = Sigmoid(S)

[0050] In the formula, ABC(·)=AF(BN(conv) 1×1 (·))), AF uses the LeakyReLU function.

[0051] Scaling using channel attention masks:

[0052] f g =concat(f E ,f D )·Ga.

[0053] The 3D point cloud classification method based on a multi-geometric dual-edge attention network provided by this invention also has the following technical features, wherein step S5 is represented as follows:

[0054] fo ut =MLP(concat(AMP(f) g ),AAP(f g )))

[0055] In the formula, AMP represents Adaptive Max Pooling, and AAP represents Adaptive Average Pooling.

[0056] Invention Function and Effect

[0057] According to the present invention, a 3D point cloud classification method based on a multi-geometric dual-edge attention network is constructed using an attention mechanism. This network has a multi-geometric enhancement module, a dual-edge attention module, and a channel attention module. First, the multi-geometric enhancement module considers not only Euclidean space features but also geometric features, acquiring original point cloud features from different directions to obtain sufficiently rich information to generate more geometric cues. Second, the present invention proposes a dual-edge attention module to extract more meaningful information from low-level geometric input and appropriately weight it to aggregate point features. This dual-edge attention module effectively learns local and global features using two different branches, and finally connects the two branches to aggregate local and global contextual elements. Furthermore, the designed repeatedly stacked dual-edge attention module can be repeated multiple times, and the extracted features are connected in a skip connection manner, thereby enhancing the network's capabilities and improving the accuracy of point cloud classification. Simultaneously, a channel attention module is used to further enhance the architecture, extracting meaningful information from the final output and appropriately weighting and aggregating point features for 3D point cloud classification tasks. Furthermore, because the multi-geometric dual-edge attention network employs an attention mechanism, it allows the network to focus on important regions over a larger area, enabling the network to better focus on the more important parts of the input data for prediction, thereby further improving the accuracy of 3D point cloud classification. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the 3D point cloud classification method based on a multi-geometric dual-edge attention network in an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of the structure of the multi-geometric dual-edge attention network in an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram illustrating the visualization effect of multiple geometry enhancement modules enhancing geometric features in an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the dual-edge attention module in an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the structure of the local attention mechanism in an embodiment of the present invention; and

[0063] Figure 6 This is a schematic diagram of the global attention mechanism in an embodiment of the present invention. Detailed Implementation

[0064] This invention proposes an end-to-end 3D point cloud classification network: Multi-Geometric Dual-Edge Attention Network (MgdaNet). This model framework includes a multi-geometric feature enhancement module and a dual-edge attention module. It uses an attention mechanism to learn global and local point cloud features from more features to achieve more efficient and accurate point cloud analysis.

[0065] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following describes in detail the three-dimensional point cloud classification method of the multi-geometric dual-edge attention network of this invention with reference to embodiments and accompanying drawings.

[0066] <Example>

[0067] Figure 1 This is a flowchart illustrating the three-dimensional point cloud classification method based on a multi-geometric dual-edge attention network in an embodiment of the present invention.

[0068] like Figure 1 As shown, the 3D point cloud classification method based on multi-geometric dual-edge attention network includes the following specific steps:

[0069] Step S1: Construct a multi-geometry double-edge attention network based on the attention mechanism. This network has a multi-geometry enhancement module (MGEM), a double-edge attention module (DEAM), and a channel attention module. The attention mechanism includes local attention and global attention.

[0070] Point cloud classification is challenging because a finite number of points are discretely distributed in an infinite 3D space, and each point is useful for representing the entire point cloud set. Previous research has mainly focused on extracting local features and obtaining global information using symmetric functions, while neglecting the spatial relationships between all points. Attention mechanisms, widely used in vision tasks, can effectively address this problem. Compared to traditional convolution, attention mechanisms allow the model to focus on important regions over a larger scope, enabling the network to better focus on the parts of the input data that are more important for prediction.

[0071] Therefore, this embodiment incorporates an attention mechanism into MGDNet, the multi-geometric dual-edge attention network, enabling the network to focus on learning the global network context and obtain more important point cloud features. Simultaneously, it further enhances the network architecture, ensuring that the network always utilizes the most important point cloud features to perform the current task. This embodiment applies local attention La (La) to the DEAM (dual-edge attention module) and global attention Ga (Ga) to the global feature allocation of the MGDNet backbone.

[0072] Figure 2 This is a schematic diagram of the structure of the multi-geometric dual-edge attention network in an embodiment of the present invention.

[0073] like Figure 2 As shown, the multi-geometric dual-edge attention network in this embodiment utilizes a multi-geometric feature enhancement module ( Figure 2 The MGEM (Medium-Ray Memory Emulation) module obtains sufficiently rich point cloud information as input, providing more geometric cues to enrich CNN-based feature learning. This is then achieved through repeatedly stacked double-edge attention blocks (...). Figure 2 DEAM (Dedicated Application Model) effectively learns local features based on a local attention mechanism. Then, the channel attention module uses global attention to learn global features. Finally, the global and local features are combined, and the representational power of the features is further enhanced through max pooling and average pooling.

[0074] Step S2: Input the unordered point set of the original point cloud into the multi-geometric dual-edge attention network, and obtain the point cloud features of the original point cloud in different directions through the multi-geometric enhancement module, thereby generating multi-dimensional low-level geometric vectors.

[0075] In existing 3D point cloud classification methods based on direct point clouds, most only use the inherent features of the original point cloud as input, such as the 3D coordinates of the point cloud. However, the input information is too simple, resulting in the incomplete extraction of key features. To address this issue, this embodiment proposes a Multi Geometry Enhancement Module (MGEM) that not only considers Euclidean space features but also introduces multiple geometric features. It acquires the original features of the point cloud from different directions, allowing the network to benefit from the geometric relationships in the low-level space when learning features in the subsequent high-level space, thus obtaining sufficiently rich input features.

[0076] The multi-geometry enhancement module in this embodiment operates through a collaborative mechanism between geometric features in Euclidean space and eigenvalue space, aiming to provide rich geometric relationship information in the underlying space for subsequent implicit geometric feature learning. A given point cloud X is represented as a set of three-dimensional points {P}. i|i=1,2,3,....,n}, typically, each point P in the point cloud i It is represented by three-dimensional coordinates. That is, each point P i From its coordinates P i =(x i ,y i ,z i The coordinates can be provided, or may include RGB information, intensity values, and other representations. In conventional deep learning, discrete point clouds only provide three-dimensional coordinates. Based on the motivation of enriching the input features of point clouds, this embodiment recreates other geometric features of the three-dimensional point cloud based on multiple geometry enhancement modules, providing more low-level geometric cues for subsequent high-level feature learning.

[0077] Figure 3 This is a schematic diagram illustrating the visualization effect of multiple geometry enhancement modules enhancing geometric features in an embodiment of the present invention.

[0078] like Figure 3 As shown, to expand the input of the original point cloud information, the multi-geometry enhancement module divides the 3D point cloud space into Euclidean space (dark circles in the figure) and eigenvalue space (light circles in the figure). Given an anchor point (the dark point within the dark circle in the figure), the neighbors of the anchor point in Euclidean space are the areas within the dark circles in the figure, and the neighbors in the eigenvalue space are the areas within the light circles. Normally, convolution operations on 3D points in Euclidean space only apply to points in local regions, as shown by the dark circles. In this embodiment, a triangular contour is constructed for the given anchor point to provide richer information on the geometric shape of the triangle, such as edges (light vectors within the dark circles) and normals (dark vectors within the dark circles). In the eigenvalue space, the anchor point is associated with points having similar local shapes. For example, the points in the light circle region and the anchor point both belong to the wing of an aircraft and have the same geometric features, even if these points are far apart in Euclidean space.

[0079] Traditional point cloud classification methods typically use Euclidean distance in Euclidean space as a metric and employ the K-Nearest Neighbor (KNN) search algorithm to obtain the value of any point P in the point cloud X. i Find the k nearest neighbors of a given point P. i The K-Nearest Neighbors are represented as {P i1 ,P i2 ,....,P ik}, where P ij (1≤j≤k) represents P in Euclidean space. iA point in the k-nearest neighbors. Triangular faces are widely used in mesh or volume data because they can represent the contours of 3D objects and obtain richer geometric features. Based on this idea, this embodiment creates different triangular faces for any point in a 3D point cloud. Using the contour features of these 3D objects, the low-level geometric representation of the 3D point cloud can be expanded, and the generated features can provide richer geometric cues for subsequent high-order feature extraction.

[0080] The specific process of step S2 is as follows:

[0081] Step S2-1: Based on the 3D coordinates of any point in the Euclidean space of the point cloud X, use the K-Nearest Neighbor search algorithm to search for any point P. i The two nearest neighbor points P i1 and P i2 , respectively represented as

[0082] Step S2-2, based on two neighboring points P i1 and P i2 With point P i Form a triangular face, thus obtaining the outline edge of the triangle: E1 = P i1 -P i E2 = P i2 -P i Point P i One of the important geometric features is the normal vector: n = E1 × E2, and the distance between two distinct points in Euclidean space is calculated using the L2 norm: l Eu =|E i |=|P ij -P i |, among which

[0083] Through the above process, this embodiment, based on only three-dimensional coordinates in Euclidean space, utilizes triangular faces to extend low-level geometric features that estimate the edges of geometric shapes and normal vectors: the triangle contour edges E1 and E2, the normal vector n, and the distance l between the two points. Eu Compared to other geometric methods, the method in this embodiment captures richer input information from the original point cloud while reducing the computational process.

[0084] When using feature maps to calculate the structure tensor to measure the local geometric properties of input points, since the structure tensor is invariant to rotation and translation transformations at each point, this embodiment, after introducing geometric information such as Euclidean space edges and normal vectors, simultaneously introduces feature space to further enhance geometric features, thus forming a multi-geometric enhancement module. This multi-geometric enhancement module uses the structure tensor in feature space to measure the local geometric properties of a given point, thereby capturing points that are geographically distant in Euclidean space but possess similar local structural features (such as...). Figure 3 (The light-colored circle in the middle).

[0085] Step S2-3, let M = (E1, E2, ..., E i E k )=(P i1 -P i ,P i1 -P i ,...,P ii -P i ,P ik -P i The three-dimensional structure tensor in the feature space is defined as: T = M × M T , where T is a symmetric positive definite matrix, and the superscript T represents the transpose operation.

[0086] Step S2-4: Decompose the three-dimensional structure tensor T into T = RΛR T , where R is a rotation matrix and Λ is a diagonal and positive definite matrix, called the eigenvector and eigenvalue matrix, respectively.

[0087] Steps S2-5, for each point P i Using three-dimensional structure tensors Point P i Eigenvalues ​​(1≤i≤N, λ) i ∈R 3 ), where the positive eigenvalue λ i ∈R 3 It is ordered, and λ 1 ≥λ 2 ≥λ 3 ≥0.

[0088] Step S2-6: Calculate the distance between different points in the feature space using the L2 norm.

[0089]

[0090] Finally, the multi-geometry enhancement module expands the 3D coordinates of each point into a 14-dimensional low-level geometric vector:

[0091]

[0092] The above process can be represented as: Corresponding to:

[0093]

[0094] Step S3: The dual-edge attention module takes low-level geometric vectors of multiple dimensions as input, extracts global point features and local features of the original point cloud based on local attention, and connects these extracted features in a skip connection manner to obtain the 3D point cloud feature vector.

[0095] Figure 4 This is a schematic diagram of the dual-edge attention module in an embodiment of the present invention.

[0096] After acquiring rich geometric information at the lower level, the multi-geometric dual-edge attention network still needs to learn geometric features in the higher-level space. To address this, this embodiment proposes a dual-edge attention feature module that incorporates a local attention mechanism, such as... Figure 4 As shown, this module has an edge feature branch and a density edge attention branch, the main purpose of which is to acquire and enhance the required local features in parallel through the two branches.

[0097] Local attention, a type of spatial domain attention, uses self-attention (SA) to capture internal edge information. Self-attention, also known as internal attention, is a mechanism for calculating the semantic relevance between different items in a data sequence. Using self-attention in a dual-edge attention module can transform the spatial domain information of a point cloud from the original input to another space, thereby preserving key information.

[0098] Figure 5 This is a schematic diagram of the local attention mechanism in an embodiment of the present invention.

[0099] like Figure 5 As shown, the process of local attention for learning local features is as follows:

[0100] First, given the input feature map Feeding these into convolutional layers respectively yields the Q, K, and V matrices generated by the linear transformation of the input features:

[0101] Q = AF(BN(conv) 1×1 (F)))

[0102] K = AF(BN(conv) 1×1 (F)))

[0103] V = AF(BN(conv) 1×1 (F)))

[0104] In the formula, AF is the activation function, here ReLU is chosen, BN is batch normalization, and conv 1×1It is a 1×1 convolution.

[0105] Then, the product of the query matrix and the key matrix is ​​used to compute long-term dependencies in the point cloud spatial data, and a softmax layer is used to estimate the corresponding attention weight matrix. This process can be expressed as the formula:

[0106] d c×c =softmax(QK) T )

[0107] In the formula, d c×c d represents the attention weight matrix, where the indices indicate the matrix size, and T denotes the transpose operation. c×c The correlation between the positions within the Q matrix and those within the K matrix is ​​measured; higher weights indicate greater similarity, which helps the network better understand the "location" of important information. This is achieved by comparing the V matrix and the attention similarity matrix d. c×c Multiplication, i.e., F′=d c×c ×V can easily achieve this refinement process.

[0108] The Q, K, and V matrices are determined by shared corresponding linear transformation matrices and input features F, all of which are order-independent. Furthermore, softmax and weighted summation are permutation-independent operators. Therefore, self-attention is permutation-invariant, making it well-suited for the disordered and irregular regions of point clouds. Moreover, local attention uses the difference between the original features F and the self-attention output F′ as a corresponding error signal to enhance self-attention, thereby achieving better network performance. Thus, the process of local attention can be represented as:

[0109] La=α·AF(BN(conv 1×1 (F′-F)))+F

[0110] Here, α is the scale parameter. The error signal is important for both the attention output and the original features. The scale parameter α is initialized to 0 and has learnable weights to simplify training.

[0111] In the dual-edge attention module of this embodiment, local attention uses the Edgeconv network to aggregate local features to capture local point cloud information, and the KNN algorithm is used to obtain the center point P. i The neighborhood graph is then used to share parameters through MLP mapping in the feature space, and finally a symmetric function is applied to encode the corresponding region information into the feature of each point.

[0112] Graph Neural Networks (GNNs) are a neural network architecture defined on a graph. They propagate node representations across a graph structure until a fixed point is reached. The resulting node representations are then used as features in classification and regression problems. To apply a GNN to a point cloud, it must first be converted into a directed graph. For a point cloud dataset with N points, assuming its dimension is F, it can be defined as... When color and intensity information are not included, F = 3. Similar to Edgeconv, edge features are considered as implicit geometric features in the embedding space because they are constructed according to the geometric relationships between points, based on constraints of high-dimensional Euclidean distance.

[0113] The process of obtaining this edge feature is as follows:

[0114] First, the center point P is obtained using the KNN search algorithm. i local neighborhood P j (1≤j≤k) and form a neighborhood graph. For a given directed graph G representing the local graph structure of a point cloud. i =(V i E i ) uses points in a subset as vertices V i Its vertex is:

[0115] V i ={P1,P2,...,P j ,...,P k ,P i},

[0116] To avoid the computational complexity that consumes a lot of energy, this embodiment does not construct fully connected edges for the entire point cloud, but instead uses neighboring points and P as the basis for connection. i The connections between them form different directed edges E i :

[0117] E i ={e1,e2,...,e j}={(P1,P i ),(P2,P i ),...,(P j ,P i ),...,(P k ,P i )}

[0118] Thus, a radial local graph was obtained using the KNN algorithm. This graph consists of edges pointing from neighboring points to the centroid. Then, parameters are shared through MLP mapping in the feature space, and finally, a symmetric function is applied to encode the corresponding region information into the features of each point. Specifically:

[0119] Point feature P iWith edge vector P j The feature differences between them are connected to form neighborhood information, and then the neighborhood information P is... j -P i and P i Features are aggregated as P i With P j Edge features between e ij :

[0120] e ij =h(P i ,P j -P i )

[0121] P i Edge features G i The calculation formula is:

[0122]

[0123] In the formula, h() represents the aggregation operation, and the concat function is selected in this embodiment for the experiment.

[0124] Secondly, the MLP is used as a fully connected layer to encode the corresponding region information in each point feature. In practice, this operation can be understood as performing a 1×1 convolution operation on the point cloud feature map to encode the local context given by the edge features, followed by batch normalization layers and activation functions. The processing can be expressed as the formula:

[0125] M(G i ) = AF(BN(conv 1×1 (G i )))

[0126] In the formula, M represents a multilayer perceptron (MLP), AF is the activation function, BN is batch normalization, conv is convolution, and its subscript indicates the filter size.

[0127] Finally, based on the aforementioned local attention, the encoded edge features G are... i Perform feature extraction:

[0128] f e =La(Max(M(G) i )))

[0129] In the formula, La represents local attention, and Max() represents the max pooling operation.

[0130] Since symmetric functions are invariant to permutations of the input point cloud model, and max-pooling is a symmetric function, max-pooling is employed in the edge feature branches to obtain local features as compensation for point features, ensuring that the multi-geometric dual-edge attention network remains invariant to the input permutation. The max-pooling operator obtains the maximum value on each feature channel, and its combination with local attention La makes it easier to capture the most representative features.

[0131] The final output of the edge feature branch in this embodiment is:

[0132] f E =concat(f e1 ,f e2 ,f e3 ,f e4 )

[0133] In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

[0134] For large point clouds, symmetric functions like Maxpooling can efficiently and easily extract prominent features to summarize the contour of a local region. However, in some challenging cases, Maxpooling ignores some key features. Especially for complex edge features, each point has k related neighboring points, requiring a more reasonable approach that considers the detailed features of all neighbors. In this case, to overcome the uneven density of key features obtained by symmetric functions in point cloud edge features, local details can be intuitively aggregated by calculating a weighted sum of neighboring features. Specifically:

[0135] In this embodiment, the density edge attention branch uses density convolution to adaptively aggregate local information from all neighbors, thereby obtaining more fine-grained features from the edge convolution.

[0136] Unlike MLPs that primarily use 1×1 convolutions to encode single-point features, density convolutions act as 1×k convolutions over local regions, learning the weights of k neighbors in that region, and ultimately aggregating the weighted sum of the features of the k neighbors as detailed local context.

[0137] D(G i ) = AF(BN(conv 1×k (G i )))

[0138] In the formula, D represents density convolution. Compared with 1×1 convolution, the kernel size of density convolution is 1×k, where k is the size of k in the k-nearest neighbor algorithm.

[0139] The squeeze operation is used to remove dimensions with only 1 channel, and local attention is applied to the encoded edge features G.i Perform local feature extraction:

[0140] f d =La(squeeze(D(G) i )))

[0141] Finally, the output of the density edge attention branch is represented as:

[0142] f D =concat(f d1 ,f d2 ,f d3 ,f d4 )

[0143] In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

[0144] In step S4, the channel attention module takes low-level geometric vectors of multiple dimensions as input, generates a channel attention mask based on global attention, and scales the 3D point cloud feature vectors through the channel attention mask to further enhance the feature representation capability.

[0145] Unlike local attention, global attention is channel attention. In fact, spatial attention computation often consumes a significant amount of time and memory. Global attention is located at the end of the network backbone, effectively and efficiently avoiding channel redundancy by computing long-term dependencies between channels. Global attention primarily focuses on the channel space, using the attention mechanism to scale each embedded element we generate, thereby further enhancing the architecture and ensuring that the network always uses the most important point representations to perform the current task.

[0146] Figure 6 This is a schematic diagram of the global attention mechanism in an embodiment of the present invention.

[0147] like Figure 6 As shown, Global Attention (GA) can simplify the context in each channel by using a shared MLP that operates in the channel direction. The input of this module is the raw point cloud features, which are processed by multiple geometry enhancement modules to obtain rich geometric features. Two 2D convolutional layers with kernel size of 1×1 are used to reduce the dimensionality of the geometric features, and a sigmoid function is used to generate the channel attention mask Ga. The process is as follows:

[0148]

[0149] Ga = Sigmoid(S)

[0150] In the formula, ABC(·)=AF(BN(conv) 1×1(·))), AF uses the LeakyReLU function.

[0151] The reason for adding global attention to the channel attention module is that, compared to acquiring all points or selecting certain points based on other metrics, channel attention can effectively reduce the size while fully preserving the information of each channel. Typical neural network models do not consider the dependencies between the output and each input channel. Global attention aims to allow the network to automatically select channel features containing a large amount of information for enhancement based on the mask obtained from the input, adaptively selecting features at the end of the main path and improving the network's representational power. Clearly, global attention does not have extra parameters to learn; it only needs to learn the parameters in the MLP layer with shared weights. Therefore, it can effectively reduce the model size, while guiding computational resources to use a large amount of useful information to weight the channels and suppress useless information through the learned mask Ga.

[0152] In this embodiment, each embedding, i.e., f, is generated by scaling the mask Ga obtained by adding global attention to the channel attention module. g =concat(f E ,f D Ga further enhances the architecture and improves the network's representational capabilities, enabling the network to always utilize the most important point representations to perform the current task.

[0153] Step S5: To generate object features after channel aggregation, max pooling and average pooling are used to collect representative object features from the scaled features to infer better channel attention. The feature vectors obtained by max pooling and average pooling are then merged element-wise through an MLP layer to obtain the final output value. Based on this value, the category of the original point cloud is determined.

[0154] fo ut =MLP(concat(AMP(f) g ),AAP(f g )))

[0155] In the formula, AMP represents Adaptive Max Pooling, and AAP represents Adaptive Average Pooling.

[0156] In this embodiment, to evaluate the performance of MGDANet (Multi-Geometric Dual-Edge Attention Network), the network was evaluated on both a synthetic dataset and a real dataset. Experimental environment: Deep learning was implemented using PyTorch 1.2 and CUDA 10.0. Training and testing were conducted on a Windows 10 computing node equipped with a 1.70 GHz Intel(R) Xeon(R) Bronze 3104 CPU, 64 GB of RAM, and an NVIDIA GeForce RTX 2080 graphics processing unit (GPU).

[0157] Among them, the synthetic dataset is the ModelNet40 benchmark dataset. This dataset is the most popular and authoritative in point cloud analysis due to its clean shape and good structure. It is widely used in point cloud shape classification and surface normal estimation. However, due to the limited training data for various categories and the imbalanced distribution, point cloud classification on the ModelNet40 dataset is very challenging. ModelNet40 consists of 12,311 CAD-generated mesh models from 40 object categories, of which 9,843 are used for training and the remaining 2,468 are reserved for testing. In addition, in this experiment, the configuration in PointNet[1] is followed. 1,024 and 2,048 points are uniformly sampled from the mesh model and used for the experiment. The points are then moved to the origin and rescaled to form unit spheres. In the experiment, only the three-dimensional coordinates of the input points of each point cloud sample are used as the input of the network model. The average accuracy (AA) in each category and the overall accuracy (OA) of all categories are used as evaluation metrics.

[0158] To further demonstrate the effectiveness and robustness of the classification network of this invention, experiments were also conducted on a real-world dataset, the ScanObjectNN dataset, used for 3D point set classification tasks. ScanObjectNN is a more robust real-world object dataset containing approximately 15,000 objects across 15 categories. Although it has fewer categories than ModelNet40, it is more practically challenging than synthetic datasets due to complex backgrounds, partial missing values, and various real-world deformations. The ScanObjectNN dataset has several variables, each representing a different level of difficulty. The experiments used PBT50RS as the data object, representing the most difficult case of the ScanObjectNN dataset, involving perturbations with a 50% translation. Each perturbation variable consists of five randomly sampled objects from the original ground reality objects, incorporating rotation and scaling. In the experiments, 1024 points were uniformly sampled as the input point cloud, and overall accuracy (OA) and average classification accuracy (AA) were used as evaluation metrics.

[0159] In testing, the Multiple Geometry Augmentation (AMP) module expands the 3D coordinates of each input point into a low-level geometric vector of 14 degrees. The expanded features are then concatenated through four cascaded Dual-edge Attention Modules (DEAMs). Within each DEAM, the KNN algorithm is used to search for local neighbors of each point (k=20 is optimal), extracting high-level geometric features at different scales (64, 64, 128, 256) in the embedding space. To integrate information from different scales, the output feature maps of the DEAMs are concatenated and further integrated into a 1024-dimensional representation using a mask generated by the Global Attention Module (GAM). We then cleverly apply both AMP and AAP simultaneously to collect representative object features to form a global embedding vector. Using this vector, confidence scores for all possible classes can be regressed using three shared MLP layers (outputs of 512, 256, and c) (ModelNet dataset c=40, ScanObjectNN dataset c=15).

[0160] In testing, stochastic gradient descent (SGD) with a momentum of 0.9 was used as the optimizer for training, with a batch size of 32 and 300 training epochs. The initial learning rate was 0.1, and cosine annealing was used to adjust the learning rate each epoch until it was reduced to 0.001. Cosine annealing is a learning rate scheduler that starts with a large learning rate, decreases it relatively quickly to a minimum, and then increases it rapidly again. Furthermore, during training, the training data was augmented using random anisotropic scaling in the range [-0.66, 1.5] and translation in the range [-0.2, 0.2]. No preprocessing data augmentation or voting methods were used during testing. Finally, cross-entropy was used as the loss function between the predicted and true labels.

[0161] The performance evaluation test results are as follows:

[0162] 1) The performance of the multi-geometry dual-edge attention network in this embodiment on the 3D point cloud classification of the ModelNet40 dataset is shown in Table 1 below (numbers are reported as percentages, coords: 3D coordinates; norm: point normal; voting: multiple voting evaluation strategy):

[0163]

[0164]

[0165] Table 1

[0166] As shown in Table 1 above, our MGDANet achieves an overall accuracy of 93.6% and an average accuracy of 91.0% with only 1k point coordinates as input, surpassing all voxel-based methods [10,12,31]. Compared to view-based methods, our method outperforms other view-based methods [9,34,35], only falling short of MHBN. View-based methods benefit from mature 2DCNN frameworks and massive 2D datasets in terms of accuracy, but are easily limited by the number of views and have low generalization ability for some complex scenes. Notably, our method outperforms most state-of-the-art methods with the same input (using only 1k point coordinates) compared to other point-based methods, with an overall accuracy only slightly lower than the PT accuracy by 0.1%, while improving the average accuracy by 0.4%. It even outperforms some methods that use additional input points, such as SO-Net, which uses rich information, 5k points, and normals as input. Similarly, our method is on par with RS-CNN, which uses a post-processing method that evaluates multiple times during testing. When we use 2k points as input, the overall accuracy is further improved, reaching an optimal 93.8%, demonstrating the superiority and effectiveness of our proposed method.

[0167] 2) The performance of the multi-geometry dual-edge attention network in this embodiment on the 3D point cloud classification of the ScanObjectNN dataset is shown in Table 2 below (numbers are reported as percentages, bold indicates best performance):

[0168]

[0169]

[0170] Table 2

[0171] For real-world point cloud classification, tests were conducted using the same network architecture, training strategy, and 1k 3D coordinates as input. To ensure a fair comparison with state-of-the-art methods, classification experiments were performed using PBT50RS, the most challenging variable in the ScanObjectNN dataset.

[0172] The experimental results are shown in Table 2 above. The overall accuracy of MGDANet in this embodiment is 81.4%, and the average classification accuracy is 78.9%, with both OA and AA reaching the highest levels. We outperform other methods in 7 out of 15 categories. For challenging cases such as bag or box, MGDANet consistently achieves higher accuracy than other methods. Furthermore, compared to other methods which exhibit uneven distribution across categories, our method is very evenly distributed across all categories. Even in categories where accuracy is not optimal, such as table and bed, our accuracy is not significantly lower, indicating that our network achieves a better balance between accuracy and reliability for real-world point cloud classification. The purpose of point cloud analysis is to solve practical problems, and the excellent performance on real-world point clouds is a strong affirmation of our method.

[0173] Although the ScanObjectNN dataset contains hard cases for point cloud classification, MGDANet effectively addresses the main difficulties. For example, residual background points in real-world data can confuse the network because they are irrelevant to shape and structure. However, the global and local attention modules proposed in this invention aim to reweight these points based on their similarity, thereby reducing the importance of background points. Furthermore, to form a global representation of the point cloud, we use the DEAM module to learn sufficient local salient features and fine-grained density features of edge-neighbor points. Aggregating the two branches to form a global representation mitigates the side effects of missing parts in the real point cloud, fully representing the comprehensive local context. The multi-geometry augmentation model, associated with Euclidean space and feature space, enhances the relationships between points, enriching the underlying geometric information and further demonstrating its effectiveness and robustness.

[0174] To verify the functionality and effectiveness of different modules in the proposed network, this embodiment also conducts an ablation study on the modules proposed in this work. Furthermore, we examine the optimal combination of input features, analyze various attention modules, and test the impact of various components on classification performance. Our ablation study on the ModelNet40 benchmark is used for classification tasks to analyze the effectiveness of the proposed method. All experiments in the ablation study were conducted using 1024 point clouds, and K-nearest neighbors were all set to K=20.

[0175] 1) Combinatorial analysis of input features in this embodiment:

[0176] 2) Input features directly affect the representation of local geometry and the relationships between points; therefore, how to define input features is a question worth exploring. To find the most suitable feature combination, in addition to the multi-geometric features used in this embodiment, this embodiment also investigated more possible feature combinations. We experimented with eight setting methods, and the results are shown in Table 3 below (p:(x, y, z), n:normal, e:edge, l:|edge|):

[0177] Where P i :(x, y, z), n i Let P be the point i Construct the normal vectors of the triangular contour, with E1 and E2 as points P. i Both sides, l Eu1 and l Eu2 P represents the corresponding side length. j -P i Let P be the point i With P j The coordinate difference between them. λ i Let P be the point i eigenvalues, λ j -λ i Let P be the point i With P j The characteristic difference between them.

[0178]

[0179] Table 3. Ablation study of geometric features in different dimensions of multi-geometric enhancement modules.

[0180] As can be seen from setting method 1, using only coordinates and a feature dimension of 3, our method can achieve an accuracy of 93.2%. Although the idea of ​​constructing explicit triangular face features is simple and intuitive, the accuracy of setting methods 2, 3, and 4 in Table 4.4 all shows an increase, indicating that the method of constructing triangular contours is feasible. When fully representing the low-level geometric details of the estimated triangular face, such as vertices, face normals, and edges as input features for points, setting method 4 can achieve a result of 93.5%. Although the model using only the feature values ​​of points is not ideal, as shown in setting method 5, by utilizing their differences and coordinates and coordinate differences, the result can be improved to 93.4%, as shown in setting method 6. Based on this, by adding the face normals and the Euclidean distance of their edges, the feature dimension reaches 14, and the accuracy can reach the best of 93.6%, as shown in setting method 7. This proves that the cooperation between Euclidean space features and feature space can complement the optimal representation of point descriptors. However, when too many features are added, such as in setting method 8, its performance is not as good as other settings.

[0181] 3) Analysis of the attention module in this embodiment:

[0182] Table 4 below shows the effects of using different attention mechanisms in the network: EFAB: local attention in the edge feature attention branch; DEAB: local attention in the density edge attention branch; GA: global attention in the main path.

[0183]

[0184] Table 4 shows the ablation study of various attention modules in the classification network on ModelNet40.

[0185] As shown in Model 2, using local attention only in the DEAB branch achieves an accuracy of 93.4%. Using local attention in both branches results in a performance decrease, possibly because local attention is based on calculating long-term dependencies in the spatial data using inner product calculations. Too many similar features can cause interference, and redundant features may lead to overfitting. Applying global attention to the main channel improves the result and further enhances the network to achieve optimal performance. This is because global attention is an inter-channel correlation attention for high-level point cloud feature maps, effectively avoiding redundancy in channels within the abstract embedding space and fully preserving the information of each point. It can be observed that the attention mechanism works well in our network. Various attention mechanisms can improve the performance of the network model without various attention mechanisms (i.e., the baseline model in Table 4) by 0.3-0.5%. This improvement is particularly significant in point-based 3D point cloud classification methods.

[0186] 4) Analysis of the network frame design in this embodiment:

[0187] The effectiveness of the proposed method components was analyzed on the ModelNet40 classification benchmark, and the results are shown in Table 5. In Model 1, no Multiple Geometry Augmentation Module (MGEM) was used, and only 3D coordinates were used as input. As can be seen from Models 2, 3, and 4, the geometric features learned by the MGEM are beneficial for 3D point cloud classification.

[0188]

[0189]

[0190] Table 5 analyzes the effectiveness of the method components of this invention on the ModelNet40 classification task benchmark. All experiments used k=20 nearest neighbors.

[0191] Comparing the two branches of the Dual Edge Attention Module (DEAM), the local salient features extracted by the Edge Feature Attention Branch (EFAB) contribute more to the network's high-level geometric features. It's worth noting that Model 3, using only the Density Edge Attention Branch (DEAB), has the lowest accuracy. This is because in 3D point cloud classification tasks, global information is more important for classifying the overall shape than local information, which is why pioneering methods like PointNet prioritize global information.

[0192] Recent direct point cloud-based methods have rarely achieved accuracy improvements exceeding 0.5%, primarily due to insufficient utilization of point cloud neighborhood information in the high-level space. In contrast, performance significantly improves once we enrich features at both global and local levels. As shown in Model 4, the overall accuracy reaches a peak of 93.6% when both branches are used. Generally, local features are an important complement to global features; the dual-edge attention module tests global and local information as a whole, considering the inherent connections between global and local features. The experimental results demonstrate that our proposed method is a promising and effective classification approach.

[0193] 5) Model complexity analysis in this embodiment:

[0194] The number of model parameters is used to measure model space complexity, and the network forward pass time is used as a rough measure of model time complexity. Runtime was recorded on a single GTX 1080 Ti GPU with a batch size of 32. This GPU and the comparison models were implemented in the same hardware environment, all using PyTorch. The experimental results are shown in Table 6 below:

[0195]

[0196] Table 6 shows the complexity performance of classification networks on ModelNet40.

[0197] As shown in Table 6, the proposed MgdeNet network model is only 34M in size and runs in less time than PointNet. By maintaining a reasonable speed and model size, this method achieves competitive performance in terms of accuracy, with an overall accuracy of 93.6%, achieving a high balance between performance and complexity.

[0198] Functions and effects of the embodiments

[0199] According to the 3D point cloud classification method based on a multi-geometric dual-edge attention network provided in this embodiment, a multi-geometric dual-edge attention network is constructed using an attention mechanism. This network has a multi-geometric enhancement module, a dual-edge attention module, and a channel attention module. First, the multi-geometric enhancement module considers not only Euclidean space features but also geometric features. It acquires the original features of the point cloud from different directions to obtain sufficiently rich information to generate more geometric cues. Second, this invention proposes a dual-edge attention module to extract more meaningful information from low-level geometric input and appropriately weight it to aggregate point features. This dual-edge attention module effectively learns local and global features using two different branches, and finally connects the two branches to aggregate local and global contextual elements. In addition, the designed repeatedly stacked dual-edge attention module can be repeated multiple times, and the extracted features are connected in a skip connection manner, thereby enhancing the network's capabilities and improving the accuracy of point cloud classification. At the same time, a channel attention module is used to further enhance the architecture to extract meaningful information and appropriately weight and aggregate point features in the final output for the 3D point cloud classification task. Furthermore, because the multi-geometric dual-edge attention network employs an attention mechanism, it allows the network to focus on important regions over a larger area, enabling the network to better focus on the more important parts of the input data for prediction, thereby further improving the accuracy of 3D point cloud classification.

[0200] Meanwhile, experimental results show that on the ModelNet dataset and the real point cloud dataset ScanObjectNN, the algorithm proposed in this embodiment outperforms existing advanced point cloud-based direct processing methods in terms of accuracy, and has the characteristics of high efficiency and robustness.

[0201] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments.

Claims

1. A 3D point cloud classification method based on a multi-geometric dual-edge attention network, characterized in that, Includes the following steps: Step S1: Construct a multi-geometric dual-edge attention network based on the attention mechanism. This network has a multi-geometric enhancement module, repeatedly stacked dual-edge attention modules, and a channel attention module. The attention mechanism includes local attention and global attention. Step S2: Input the unordered point set of the original point cloud into the multi-geometric dual-edge attention network, and obtain the point cloud features of the original point cloud in different directions through the multi-geometric enhancement module, thereby generating multi-dimensional low-level geometric vectors. Step S3: The dual edge attention module has an edge feature branch and a density edge attention branch. These two branches take the multiple-dimensional low-level geometric vectors as input, extract the global point features and local features of the original point cloud based on the local attention, and connect the extracted features in a skip connection manner to finally obtain the 3D point cloud feature vector. Step S4: The channel attention module takes the multiple low-level geometric vectors of the multiple dimensions as input, generates a channel attention mask based on the global attention, and scales the 3D point cloud feature vector through the channel attention mask, thereby further enhancing the feature representation capability. Step S5: The multi-geometric dual-edge attention network uses max pooling and average pooling to collect representative feature vectors of the scaled features, and merges the collected feature vectors element by element through the MLP layer to obtain the output value, which is used to determine the category of the original point cloud.

2. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 1, characterized in that: in, The multi-geometry enhancement module divides the 3D point cloud space into Euclidean space and eigenvalue space. Based on the local structural features in the Euclidean space, it obtains similar local structural features in the eigenvalue space, thereby extracting the multi-geometry features of the original point cloud.

3. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 2, Its features are: Step S2 includes the following sub-steps: Step S2-1: Based on the 3D coordinates of any point in the Euclidean space of the point cloud X, use the K-Nearest Neighbor search algorithm to search for any point P in the point cloud X. i The two nearest neighbor points P i1 and P i2 , respectively represented as Step S2-2, based on two neighboring points P i1 and P i2 With point P i Form a triangular face, thus obtaining the outline edge of the triangle: E1 = P i1 -P i E2 = P i2 -P i Point P i The normal vector: n = E1 × E2, and the distance between these two points: l Eu =|E i |=|P ij -P i |; Step S2-3, let M = (E1, E2, ..., E i E k )=(P i1 -P i ,P i1 -P i ,...,P ii -P i ,P ik -P i The three-dimensional structure tensor in the feature space is defined as: T = M × M T ; Step S2-4: Decompose the three-dimensional structure tensor T into T = RΛR T , where R is a rotation matrix, and Λ is a diagonal and positive definite matrix, called the eigenvector and eigenvalue matrix, respectively; Steps S2-5, for each point P i Using three-dimensional structure tensors Point P i Eigenvalues ​​(1≤i≤N, λ) i ∈R 3 ), where the positive eigenvalue λ i ∈R 3 It is ordered, and λ 1 ≥λ 2 ≥λ 3 ≥0; Step S2-6: Calculate the distance between different points in the feature space using the L2 norm. Finally, the multi-geometry enhancement module expands the 3D coordinates of each point into a multi-dimensional low-level geometric vector: In the formula, 4. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 3, characterized in that: in, The dual-edge attention module uses the Edgeconv network to aggregate local features to capture local point cloud information, and employs the KNN algorithm to obtain the center point P. i The neighborhood graph is then used to share parameters through MLP mapping in the feature space, and finally a symmetric function is applied to encode the corresponding region information into the feature of each point.

5. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 4, characterized in that: in, The process of extracting local features from the edge feature branches is as follows: Point feature P i and edge vector P j The feature differences between them are concatenated to form neighborhood information P. j -P i Then, the neighborhood information P j -P i Features P i Aggregation is performed as P i With P j Edge features between e ij : e ij =h(P i ,P j -P i ) In the formula, h() represents the aggregation operation. P i Edge features G i for: Edge features G are analyzed based on a multilayer perceptron (MLP), batch normalization layer, and activation function. i Encode: M(G i )=AF(BN(conv 1×1 (G i ))) In the formula, M represents a multilayer perceptron (MLP), AF is the activation function, BN is batch normalization, conv is convolution, and its subscript indicates the filter size. Based on local attention, the encoded edge features G i Perform global point feature extraction: f e =La(Max(M(G i ))) In the formula, La represents local attention, and Max() represents the max pooling operation. This max pooling operation is a symmetric function that obtains the maximum value on each feature channel through the max-pooling operator. Combining it with local attention La makes it easier to capture the most representative features. The final output of the edge feature branch is: f E =concat(f e1 ,f e2 ,f e3 ,f e4 ) In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

6. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 5, characterized in that: in, The density edge attention branch uses density convolution to adaptively aggregate local information from all neighbors, thereby obtaining more fine-grained features from the edge convolution. The feature extraction process is as follows: Density convolution is used as a 1×k convolution on a local region to learn the weights of k neighbors in that region. Finally, the weighted sum of the features of the k neighbors is aggregated as a detailed local context, as shown below: D(G i )=AF(BN(conv 1×k (G i ))) In the formula, D represents density convolution. Compared to 1×1 convolution, the kernel size of density convolution is 1×k, where k is the size of k in the k-nearest neighbor algorithm. The dimension with 1 channel is removed using the squeeze operation, and the local attention is applied. The output feature is represented as follows: f d =La(squeeze(D(G i ))) The final output of the density edge attention branch is: f D =concat(f d1 ,f d2 ,f d3 ,f d4 ) In the formula, the subscript i∈(1,4) represents the output of the i-th dual edge attention module branch.

7. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 6, characterized in that: in, In step S4, the process of obtaining the channel attention mask is as follows: The channel attention module will process the multi-geometric features, i.e., multi-dimensional low-level geometric vectors, output by the multi-geometric enhancement module. As input, two 2D convolutional layers with a kernel size of 1×1 are used to reduce the dimensionality of the geometric feature, and a sigmoid function is used to generate the channel attention mask Ga, as follows: Ga = Sigmoid(S) In the formula, ABC(·)=AF(BN(conv) 1×1 (·))), AF uses the LeakyReLU function. Scaling based on the channel attention mask: f g =concat(f E ,f D )·Ga。 8. The 3D point cloud classification method based on a multi-geometric dual-edge attention network according to claim 7, characterized in that: in, The process of step S5 is as follows: f out =MLP(concat(AMP(f g ),AAP(f g ))) In the formula, AMP represents Adaptive Max Pooling, and AAP represents Adaptive Average Pooling.

Images