Point cloud classification method based on multi-scale and hybrid attention

Abstract

The present invention belongs to the technical field of laser radar point cloud classification and discloses a point cloud classification method based on multi-scale and mixed attention, which is improved on the basis of the PointNet network. An FE (Feature Extensions) module is added to the PointNet network to complete the expansion of the input feature dimension. Then, convolution is performed using two convolution kernels of different sizes to realize multi-scale feature extraction to obtain richer features. Then, the extracted local features are spliced ​​to enrich the global features finally extracted. Then, an MA (Mix Attention) module is introduced to optimize the extracted global features including the local features. Finally, a soft pooling operation is introduced to retain the features of the extracted point cloud to the greatest extent. These measures all improve the accuracy of the classification model.

Description

Technical Field

[0001] The present invention belongs to the technical field of lidar point cloud classification, and in particular relates to a point cloud classification method based on multi-scale and hybrid attention. Background Art

[0002] In recent years, with the increasing application of LiDAR, the processing of 3D point cloud data has also attracted considerable attention. 3D point cloud data is widely used in fields such as autonomous driving and 3D scene reconstruction. With the rise of deep learning, a growing number of researchers are applying deep learning methods to point cloud data processing. Point cloud data processing includes point cloud preprocessing, classification, and segmentation. Due to the inherent disorder, sparsity, and unstructured nature of point cloud data, point cloud classification is particularly challenging.

[0003] Deep learning-based point cloud processing methods include projection-based, voxel-based, and point-based methods. The projection-based method projects the point cloud in multiple directions to produce multiple two-dimensional images, then uses two-dimensional convolution to extract image features. This projection-based processing method results in some information loss. The voxel-based method divides the point cloud into small three-dimensional grids and uses three-dimensional convolution to extract voxel features. This voxel-based processing method directly uses three-dimensional convolution to extract features, which consumes a large amount of computer resources and time. The point-based method operates directly on the raw data points. This method does not require additional processing of the point cloud data and maximizes the preservation of point cloud information. Point-based methods can be further divided into point-based MLP, convolution-based, graph-based, and hierarchical-based methods. Among these methods, point-based MLP is the most commonly used. Typical point-based MLP networks include PointNet. However, existing PointNet networks suffer from an inability to extract local features and low classification accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a point cloud classification method based on multi-scale and hybrid attention to solve the above technical problems.

[0005] To solve the above technical problems, the specific technical solutions of the point cloud classification method based on multi-scale and hybrid attention of the present invention are as follows:

[0006] A point cloud classification method based on multi-scale and hybrid attention includes the following steps:

[0007] Step 1: Input the point cloud data into the pre-designed point cloud classification network;

[0008] Step 2: Expand the features of the input point cloud through the FE module to obtain feature 1;

[0009] Step 3: Input feature 1 obtained in step 2 into the MP1_1 and MLP3_1 modules for feature extraction, and concatenate the outputs of the MP1_1 and MLP3_1 modules to obtain feature 2.

[0010] Step 4: Input feature 2 obtained in step 3 into the MP1_2 and MLP3_2 modules for feature extraction, and concatenate the outputs of the MP1_2 and MLP3_2 modules to obtain feature 3.

[0011] Step 5, concatenate feature 2 and feature 3 to obtain feature 4;

[0012] Step 6: Input feature 4 obtained in step 5 into the MLP1_3 module to obtain feature 5;

[0013] Step 7: Input feature 5 obtained in step 6 into the MA module to obtain feature 6;

[0014] Step 8: Input feature 6 obtained in step 7 into the soft pooling module to obtain high-dimensional features;

[0015] Step 9: Input the high-dimensional features into the fully connected network to obtain the class score of each class, thus completing the classification operation.

[0016] Furthermore, the point cloud classification network described in step 1 includes an FE module, three MLP1 modules, two MLP3 modules, an MA module and a soft pooling module.

[0017] Furthermore, the step 2 includes the following specific steps:

[0018] Step 2.1, find the K nearest neighbors for each input point cloud;

[0019] Perform a K-nearest neighbor query on each input point cloud to obtain the coordinates of the K nearest neighbor points of each point;

[0020] Step 2.2, for each point's K nearest neighbors obtained in step 2.1, use the PCA algorithm to find the normal vector (nx, ny, nz) of each point;

[0021] Treat the original point cloud as a matrix and perform various mathematical operations on the matrix to obtain the normal vector of each point;

[0022] Step 2.2.1: Compose a point and its K nearest neighbors into a matrix X with K+1 rows and 3 columns.

[0023] Step 2.2.2, perform zero mean on each row of matrix X, and the resulting matrix is ​​denoted as X;

[0024] Step 2.2.3, C = 1 / 3 * (X * X^), where X^ is the transposed matrix of X;

[0025] Step 2.2.4, find the eigenvalues ​​and corresponding eigenvectors of C, arrange the eigenvalues ​​corresponding to the eigenvectors into a matrix from top to bottom, and take the first three rows to get the matrix P;

[0026] In step 2.2.5, the three-dimensional data obtained by P*X is the (nx, ny, nz) that needs to be calculated;

[0027] Step 2.3, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the curvature change value q of each point;

[0028] In detail, the original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k;

[0029] Curvature change q=max{(|Pi1-Pi|,|Pi2-Pi|,...,|Pik-Pi|)}, where |.| is the absolute value operation;

[0030] Step 2.4, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the local feature operator (s1, s2, s3) of each point;

[0031] The original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k;

[0032] D=(d(Pi,Pi1)+d(Pi,Pi2)+…+d(Pi,Pik)) / k, where d represents the Euclidean distance between two point clouds;

[0033] Concatenate D and the original point cloud (x, y, z) to obtain a 4-dimensional vector. Use the PCA algorithm to reduce the input (x, y, z, D) to 3 dimensions, obtaining (s1, s2, s3). The PCA dimensionality reduction process is the same as that in steps 2.2.3 to 2.2.5.

[0034] In step 2.5, the point cloud information (x, y, z) of each point is concatenated with the features obtained in steps 2.2 to 2.5 to obtain the expanded feature, namely feature 1. Each point of feature 1 is represented by (x, y, z, nx, ny, nz, q, s1, s2, s3).

[0035] Furthermore, the step 3 includes the following specific steps:

[0036] Step 3.1: Send feature 1 obtained in step 2 to MLP1_1 and MLP3_1 modules for dimensionality increase.

[0037] In step 3.1.1, feature 1 obtained in step 2 is fed into the MLP1_1 module for dimensionality upscaling. In the MLP1_1 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to a 64-dimensional feature.

[0038] In step 3.1.2, feature 1 obtained in step 2 is fed into the MLP3_1 module for dimensionality upscaling. In the MLP3_1 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to obtain a 64-dimensional feature.

[0039] Step 3.2, concatenate the two high-dimensional feature data obtained in step 3.1 to obtain feature 2.

[0040] Furthermore, the step 4 includes the following specific steps:

[0041] Step 4.1: Send feature 2 obtained in step 3 to MLP1_2 and MLP3_2 modules respectively for dimensionality increase.

[0042] In step 4.1.1, feature 2 obtained in step 3 is fed into the MLP1_2 module for dimensionality upscaling. In the MLP1_2 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature.

[0043] In step 4.1.2, feature 2 obtained in step 3 is fed into the MLP3_2 module for dimensionality upscaling. In the MLP3_2 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature.

[0044] Step 4.2, concatenate the two high-dimensional feature data obtained in step 4.1 to obtain feature 3.

[0045] Furthermore, step 5 splices the extracted features 2 and 3 together to solve the problem of lack of local features in the original network, and splices the 128-dimensional features and 512-dimensional features together to obtain 640-dimensional features 4.

[0046] Furthermore, step 6 inputs feature 4 obtained in step 5 into the MLP1_3 module for dimensionality upgrading. In the MLP1_3 module, a 1*1 convolution kernel is used for convolution, and the input 640-dimensional feature is dimensionalized to obtain a 1024-dimensional feature, namely feature 5, where MLP1_3 represents the third-layer MLP operation, and the convolution kernel size of the MLP operation is 1*1.

[0047] Furthermore, the specific processing process of the MA module in step 7 is as follows:

[0048] Step 7.1, divide the input batch of feature vectors into two small batches;

[0049] The dimension of the features input to the module MA is (B, C, H, W), where B is the value of batch_size during training, C is the number of channels of the input point cloud, and H and W are the height and width of the extracted feature map;

[0050] Specifically, the input batch number B is divided into two small batches, B1 and B2, where B1+B2=B;

[0051] In step 7.2, a small batch of feature vectors is sent to the channel attention module to optimize the feature vectors.

[0052] Use the channel attention module to find the channel weights to optimize a small batch of features;

[0053] Input feature dimensions B1*C*H*W, average all data on a single feature map to obtain a feature of size B1*C*1*1, then perform a one-dimensional convolution operation with a convolution kernel of 3 and a stride of 1. The resulting vector is subjected to exponential normalization by the sigmoid function to obtain the channel weight, and then the channel weight is multiplied by the feature of shape B1*C*H*W to obtain the optimized feature;

[0054] In step 7.3, another small batch of feature vectors is sent to the spatial attention module to optimize the feature vectors.

[0055] Use the spatial attention module to find the spatial point weights to optimize a small batch of features;

[0056] Input feature dimensions B2*C*H*W, average all data on different channels at the same position in the feature map to obtain features of size B2*1*H*W, then perform a two-dimensional convolution operation with a convolution kernel of 3 and a stride of 1. The resulting vector is subjected to exponential normalization by the sigmoid function to obtain spatial point weights, and then the spatial point weights are multiplied by the features of shape B2*C*H*W to obtain the optimized features;

[0057] Step 7.4: Combine the feature vectors obtained in steps 7.2 and 7.3 to obtain feature 6.

[0058] The optimized feature with a shape of B1*C*H*W and the feature with a shape of B2*C*H*W are mixed to obtain a feature with a size of B*C*H*W, namely feature 6.

[0059] Furthermore, in step 8, the feature 6 obtained in step 7 is input to the soft pooling module. The soft pooling module performs soft pooling on the input feature of size N*1024 by column to obtain a high-dimensional feature of size 1*1024.

[0060] Perform soft pooling calculation on the feature vector of size N*1. The calculation process is as follows:

[0061] Let the N*1 eigenvector be (x1,x2,...,xN)

[0062] M1=e^x1+e^x2+...+e^xN

[0063] M2=x1*e^x1+x2*e^x2+...+xN*e^xN

[0064] M=M2 / M1

[0065] The M in the formula is the calculation result of soft pooling on the N*1 feature vector. A soft pooling calculation is performed on each column of feature 7 to obtain a high-dimensional feature of size 1*1024.

[0066] Furthermore, in step 9, the high-dimensional features obtained in step 8 are input into a fully connected network to complete dimensionality reduction, so as to obtain a class probability vector, thereby completing the classification work;

[0067] The feature dimensionality reduction is completed through MLP (1024, 512, K), and a class probability vector of size 1*K is obtained. This vector represents the probability that the input point cloud data belongs to K classes. The class label corresponding to the maximum probability is the class label of the input point cloud, where K is the number of classes in the dataset.

[0068] The multi-scale and hybrid attention-based point cloud classification method of the present invention offers the following advantages: It improves upon the PointNet model, enhancing classification accuracy. First, the FE (Feature Extensions) module is introduced at the input of the underlying network structure to expand the features of the input points and enrich the feature information. Second, two convolution kernels of different sizes are used to map features from low to high dimensions, and the features extracted by the two convolutions are concatenated to further enrich the extracted features. The resulting local features are then concatenated to obtain local features of the point cloud, which are then upscaled to obtain global features that incorporate the local features. Finally, the MA hybrid attention module is introduced to optimize the resulting features. Finally, soft pooling replaces the original maximum pooling to maximize the preservation of the extracted high-dimensional feature information. These operations significantly improve point cloud classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is the structure diagram of the point cloud classification network based on multi-scale and hybrid attention of this invention;

[0070] Figure 2 This is the network structure diagram of the MA module (hybrid attention) module. DETAILED DESCRIPTION

[0071] In order to better understand the purpose, structure and function of the present invention, the point cloud classification method based on multi-scale and hybrid attention of the present invention is further described in detail below with reference to the accompanying drawings.

[0072] Figure 1 This is the network structure diagram of the point cloud classification based on multi-scale and hybrid attention proposed in this invention. The FE module is added to the input of the basic network structure. At the same time, convolution is performed with two convolution kernels of different sizes. The features extracted by convolution are spliced. Then, the extracted local features are spliced ​​and dimensionally upgraded to obtain global features including local features. Then, the self-attention module and soft pooling operation are introduced to improve the classification accuracy of the model. The specific operation steps are as follows:

[0073] Step 1: Input the point cloud data into the pre-designed point cloud classification network;

[0074] In detail, the point cloud classification network includes an FE module, three MLP1 modules, two MLP3 modules, an MA module and a soft pooling module;

[0075] Step 2: Expand the features of the input point cloud through the FE module to obtain feature 1;

[0076] Step 2.1, find the K nearest neighbors for each input point cloud;

[0077] Specifically, a K-nearest neighbor query is performed on each input point cloud to obtain the coordinates of the K nearest neighbor points of each point;

[0078] Step 2.2, for each point's K nearest neighbors obtained in step 2.1, use the PCA algorithm to find the normal vector (nx, ny, nz) of each point;

[0079] Specifically, the original point cloud is regarded as a matrix, and various mathematical operations are performed on the matrix to obtain the normal vector of each point.

[0080] Step 2.2.1: Compose a point and its K nearest neighbors into a matrix X with K+1 rows and 3 columns.

[0081] Step 2.2.2, perform zero mean on each row of matrix X, and the resulting matrix is ​​denoted as X;

[0082] Step 2.2.3, C = 1 / 3 * (X * X^), where X^ is the transposed matrix of X;

[0083] Step 2.2.4, find the eigenvalues ​​and corresponding eigenvectors of C, arrange the eigenvalues ​​corresponding to the eigenvectors into a matrix from top to bottom, and take the first three rows to get the matrix P;

[0084] In step 2.2.5, the three-dimensional data obtained by P*X is the (nx, ny, nz) that needs to be calculated.

[0085] Step 2.3, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the curvature change value q of each point;

[0086] In detail, the original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k;

[0087] The curvature change q=max{(|Pi1-Pi|,|Pi2-Pi|,...,|Pik-Pi|)}, where |.| is the absolute value operation.

[0088] Step 2.4, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the local feature operator (s1, s2, s3) of each point;

[0089] In detail, the original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k;

[0090] D=(d(Pi,Pi1)+d(Pi,Pi2)+…+d(Pi,Pik)) / k, where d represents the Euclidean distance between two point clouds;

[0091] Concatenate D and the original point cloud (x, y, z) to obtain a 4-dimensional vector. Use the PCA algorithm to reduce the input (x, y, z, D) to 3 dimensions, obtaining (s1, s2, s3). The PCA dimensionality reduction process is the same as in steps 2.2.3 to 2.2.5.

[0092] In step 2.5, the point cloud information (x, y, z) of each point is concatenated with the features obtained from steps 2.2 to 2.5 to obtain the expanded feature, namely feature 1. Each point of feature 1 can be represented by (x, y, z, nx, ny, nz, q, s1, s2, s3).

[0093] Step 3: Input feature 1 obtained in step 2 into the MP1_1 and MLP3_1 modules for feature extraction, and concatenate the outputs of the MP1_1 and MLP3_1 modules to obtain feature 2.

[0094] Step 3.1: Send feature 1 obtained in step 2 to MLP1_1 and MLP3_1 modules for dimensionality increase.

[0095] In step 3.1.1, feature 1 obtained in step 2 is fed into the MLP1_1 module for dimensionality upscaling. In the MLP1_1 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to a 64-dimensional feature.

[0096] In step 3.1.2, feature 1 obtained in step 2 is fed into the MLP3_1 module for dimensionality upscaling. In the MLP3_1 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to obtain a 64-dimensional feature.

[0097] Step 3.2, concatenate the two high-dimensional feature data obtained in step 3.1 to obtain feature 2.

[0098] Step 4: Input feature 2 obtained in step 3 into the MP1_2 and MLP3_2 modules for feature extraction, and concatenate the outputs of the MP1_2 and MLP3_2 modules to obtain feature 3.

[0099] Step 4.1: Send feature 2 obtained in step 3 to MLP1_2 and MLP3_2 modules respectively for dimensionality increase.

[0100] In step 4.1.1, feature 2 obtained in step 3 is fed into the MLP1_2 module for dimensionality upscaling. In the MLP1_2 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature.

[0101] In step 4.1.2, feature 2 obtained in step 3 is fed into the MLP3_2 module for dimensionality upscaling. In the MLP3_2 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature.

[0102] Step 4.2, concatenate the two high-dimensional feature data obtained in step 4.1 to obtain feature 3.

[0103] Step 5, concatenate feature 2 and feature 3 to obtain feature 4;

[0104] In detail, the extracted local features (feature 2 and feature 3) are spliced ​​together to solve the problem of lack of local features in the original network, and the 128-dimensional feature and the 512-dimensional feature are spliced ​​together to obtain the 640-dimensional feature 4.

[0105] Step 6: Input feature 4 obtained in step 5 into the MLP1_3 module to obtain feature 5;

[0106] Specifically, feature 4 obtained in step 5 is input to the MLP1_3 module for dimensionality upgrading. In the MLP1_3 module, a 1*1 convolution kernel is used for convolution, and the input 640-dimensional feature is upgraded to a 1024-dimensional feature, namely feature 5. MLP1_3 represents the third-layer MLP operation, and the convolution kernel size of the MLP operation is 1*1.

[0107] Step 7: Input feature 5 obtained in step 6 into the MA module to obtain feature 6;

[0108] For details, refer to the processing of MA module Figure 2 , the MA module can optimize the input features. The specific processing process of the MA module is as follows:

[0109] Step 7.1, divide the input batch of feature vectors into two small batches;

[0110] Specifically, the dimension of the features input to the module MA is (B, C, H, W), where B is the value of batch_size during training, C is the number of channels of the input point cloud, and H and W are the height and width of the extracted feature map;

[0111] Specifically, the input batch number B is divided into two small batches, B1 and B2, where B1+B2=B.

[0112] In step 7.2, a small batch of feature vectors is sent to the channel attention module to optimize the feature vectors.

[0113] Specifically, the channel attention module is used to find the channel weights to optimize a small batch of features;

[0114] Specifically, the feature dimension B1*C*H*W is input, and the average of all data on a single feature map is calculated to obtain a feature of size B1*C*1*1. Then, a one-dimensional convolution operation with a convolution kernel of 3 and a stride of 1 is performed. The resulting vector is normalized by the exponential normalization (sigmoid function) to obtain the channel weight. The channel weight is then multiplied by the feature of shape B1*C*H*W to obtain the optimized feature.

[0115] In step 7.3, another small batch of feature vectors is sent to the spatial attention module to optimize the feature vectors.

[0116] Specifically, the spatial attention module is used to find the spatial point weights to optimize a small batch of features;

[0117] Specifically, the feature dimension B2*C*H*W is input, and all data on different channels at the same position in the feature map are averaged to obtain a feature of size B2*1*H*W. Then, a two-dimensional convolution operation with a kernel of 3 and a stride of 1 is performed. The resulting vector is normalized by the exponential normalization (sigmoid function) to obtain the spatial point weight. The spatial point weight is then multiplied by the feature of shape B2*C*H*W to obtain the optimized feature.

[0118] In step 7.4, the feature vectors obtained in steps 7.2 and 7.3 are mixed together to obtain feature 6.

[0119] Specifically, the optimized feature with a shape of B1*C*H*W and the feature with a shape of B2*C*H*W are mixed to obtain a feature with a size of B*C*H*W, namely, feature 6.

[0120] Step 8: Input feature 6 obtained in step 7 into the soft pooling module to obtain high-dimensional features;

[0121] Specifically, feature 6 obtained in step 7 is input into the soft pooling module. The soft pooling module performs soft pooling on the input feature of size N*1024 by column to obtain a high-dimensional feature of size 1*1024.

[0122] In detail, the soft pooling process is explained in detail using a column as an example. The soft pooling calculation is performed on a feature vector of size N*1. The calculation process is as follows:

[0123] Let the N*1 eigenvector be (x1,x2,...,xN)

[0124] M1=e^x1+e^x2+...+e^xN

[0125] M2=x1*e^x1+x2*e^x2+...+xN*e^xN

[0126] M=M2 / M1

[0127] The M in the formula is the result of soft pooling on the N*1 feature vector. Soft pooling is performed on each column of feature 7 to obtain a high-dimensional feature of size 1*1024.

[0128] Step 9: Input the high-dimensional features into the fully connected network to obtain the class score of each class, thus completing the classification operation.

[0129] Specifically, the high-dimensional features obtained in step 8 are input into the fully connected network to complete the dimensionality reduction work, so as to obtain the class probability vector and complete the classification work.

[0130] Specifically, we perform feature dimensionality reduction using an MLP (1024, 512, K) to obtain a class probability vector of size 1*K. This vector represents the probability that the input point cloud data belongs to one of the K classes. The class label corresponding to the maximum probability is the class label of the input point cloud. K is the number of classes in the dataset.

[0131] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A point cloud classification method based on multi-scale and hybrid attention, characterized in that The steps include: Step 1: Input the point cloud data into the pre-designed point cloud classification network; Step 2: Expand the features of the input point cloud through the FE module to obtain feature 1; Step 3: Input feature 1 obtained in step 2 into the MP1_1 and MLP3_1 modules for feature extraction, and concatenate the outputs of the MP1_1 and MLP3_1 modules to obtain feature 2. Step 3.1: Send feature 1 obtained in step 2 to MLP1_1 and MLP3_1 modules for dimensionality increase. In step 3.1.1, feature 1 obtained in step 2 is fed into the MLP1_1 module for dimensionality upscaling. In the MLP1_1 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to a 64-dimensional feature. In step 3.1.2, feature 1 obtained in step 2 is fed into the MLP3_1 module for dimensionality upscaling. In the MLP3_1 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 10-dimensional feature to obtain a 64-dimensional feature. Step 3.2, concatenate the two high-dimensional feature data obtained in step 3.1 to obtain feature 2; Step 4: Input feature 2 obtained in step 3 into the MP1_2 and MLP3_2 modules for feature extraction, and concatenate the outputs of the MP1_2 and MLP3_2 modules to obtain feature 3. Step 4.1: Send feature 2 obtained in step 3 to MLP1_2 and MLP3_2 modules respectively for dimensionality increase. In step 4.1.1, feature 2 obtained in step 3 is fed into the MLP1_2 module for dimensionality upscaling. In the MLP1_2 module, a 1*1 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature. In step 4.1.2, feature 2 obtained in step 3 is fed into the MLP3_2 module for dimensionality upscaling. In the MLP3_2 module, a 3*3 convolution kernel is used to perform convolution, upscaling the input 64-dimensional feature to a 128-dimensional feature. Step 4.2, concatenate the two high-dimensional feature data obtained in step 4.1 to obtain feature 3; Step 5, concatenate feature 2 and feature 3 to obtain feature 4; Step 6: Input feature 4 obtained in step 5 into the MLP1_3 module to obtain feature 5; Step 7: Input feature 5 obtained in step 6 into the MA module to obtain feature 6; Step 8: Input feature 6 obtained in step 7 into the soft pooling module to obtain high-dimensional features; Step 9: Input the high-dimensional features into the fully connected network to obtain the class score of each class, thus completing the classification operation.

2. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that The point cloud classification network described in step 1 includes an FE module, three MLP1 modules, two MLP3 modules, an MA module and a soft pooling module.

3. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that The step 2 includes the following specific steps: Step 2.1, find the K nearest neighbors for each input point cloud; Perform a K-nearest neighbor query on each input point cloud to obtain the coordinates of the K nearest neighbor points of each point; Step 2.2, for each point's K nearest neighbors obtained in step 2.1, use the PCA algorithm to find the normal vector (nx, ny, nz) of each point; Treat the original point cloud as a matrix and perform various mathematical operations on the matrix to obtain the normal vector of each point; Step 2.2.1: Compose a point and its K nearest neighbors into a matrix X with K+1 rows and 3 columns. Step 2.2.2, perform zero mean on each row of matrix X, and the resulting matrix is ​​denoted as X; Step 2.2.3, C = 1 / 3 * (X * X^), where X^ is the transposed matrix of X; Step 2.2.4, find the eigenvalues ​​and corresponding eigenvectors of C, arrange the eigenvalues ​​corresponding to the eigenvectors into a matrix from top to bottom, and take the first three rows to get the matrix P; In step 2.2.5, the three-dimensional data obtained by P*X is the (nx, ny, nz) that needs to be calculated; Step 2.3, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the curvature change value q of each point; In detail, the original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k; Curvature change q=max{(|Pi1-Pi|,|Pi2-Pi|,...,|Pik-Pi|)}, where |.| is the absolute value operation; Step 2.4, calculate each point obtained in step 2.1 and its K nearest neighbor points to obtain the local feature operator (s1, s2, s3) of each point; The original point cloud is recorded as Pi, and the point cloud Pij is the jth neighbor point corresponding to the point cloud Pi, where the value range of i is 1~N, and the value range of j is 1~k; D=(d(Pi,Pi1)+d(Pi,Pi2)+…+d(Pi,Pik)) / k, where d represents the Euclidean distance between two point clouds; Concatenate D and the original point cloud (x, y, z) to obtain a 4-dimensional vector. Use the PCA algorithm to reduce the input (x, y, z, D) to 3 dimensions, obtaining (s1, s2, s3). The PCA dimensionality reduction process is the same as that in steps 2.2.3 to 2.2.

5. In step 2.5, the point cloud information (x, y, z) of each point is concatenated with the features obtained in steps 2.2 to 2.5 to obtain the expanded feature, namely feature 1. Each point of feature 1 is represented by (x, y, z, nx, ny, nz, q, s1, s2, s3).

4. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that The step 5 splices the extracted features 2 and 3 together to solve the problem of lack of local features in the original network, and splices the 128-dimensional features and 512-dimensional features together to obtain the 640-dimensional feature 4.

5. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that In step 6, feature 4 obtained in step 5 is input into the MLP1_3 module for dimensionality upgrading. In the MLP1_3 module, a 1*1 convolution kernel is used for convolution, and the input 640-dimensional feature is dimensionalized to obtain a 1024-dimensional feature, namely feature 5. MLP1_3 represents the third-layer MLP operation, and the convolution kernel size of the MLP operation is 1*1.

6. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that The specific processing process of the MA module in step 7 is as follows: Step 7.1, divide the input batch of feature vectors into two small batches; The dimension of the features input to the module MA is (B, C, H, W), where B is the value of batch_size during training, C is the number of channels of the input point cloud, and H and W are the height and width of the extracted feature map; Specifically, the input batch number B is divided into two small batches, B1 and B2, where B1+B2=B; In step 7.2, a small batch of feature vectors is sent to the channel attention module to optimize the feature vectors. Use the channel attention module to find the channel weights to optimize a small batch of features; Input feature dimensions B1*C*H*W, average all data on a single feature map to obtain a feature of size B1*C*1*1, then perform a one-dimensional convolution operation with a convolution kernel of 3 and a stride of 1. The resulting vector is subjected to exponential normalization by the sigmoid function to obtain the channel weight, and then the channel weight is multiplied by the feature of shape B1*C*H*W to obtain the optimized feature; In step 7.3, another small batch of feature vectors is sent to the spatial attention module to optimize the feature vectors. Use the spatial attention module to find the spatial point weights to optimize a small batch of features; Input feature dimensions B2*C*H*W, average all data on different channels at the same position in the feature map to obtain features of size B2*1*H*W, then perform a two-dimensional convolution operation with a convolution kernel of 3 and a stride of 1. The resulting vector is subjected to exponential normalization by the sigmoid function to obtain spatial point weights, and then the spatial point weights are multiplied by the features of shape B2*C*H*W to obtain the optimized features; Step 7.4: Combine the feature vectors obtained in steps 7.2 and 7.3 to obtain feature 6. The optimized feature with a shape of B1*C*H*W and the feature with a shape of B2*C*H*W are mixed to obtain a feature with a size of B*C*H*W, namely feature 6.

7. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that In step 8, the feature 6 obtained in step 7 is input to the soft pooling module. The soft pooling module performs soft pooling on the input feature of size N*1024 by column to obtain a high-dimensional feature of size 1*1024; Perform soft pooling calculation on the feature vector of size N*1. The calculation process is as follows: Let the N*1 eigenvector be (x1,x2,...,xN) M1=e^x1+e^x2+...+e^xN M2=x1*e^x1+x2*e^x2+...+xN*e^xN M=M2 / M1 The M in the formula is the calculation result of soft pooling on the N*1 feature vector. A soft pooling calculation is performed on each column of feature 7 to obtain a high-dimensional feature of size 1*1024.

8. The point cloud classification method based on multi-scale and hybrid attention according to claim 1, characterized in that Step 9 inputs the high-dimensional features obtained in step 8 into the fully connected network to complete the dimensionality reduction work, so as to obtain the class probability vector, thereby completing the classification work; The feature dimensionality reduction is completed through MLP (1024, 512, K), and a class probability vector of size 1*K is obtained. This vector represents the probability that the input point cloud data belongs to K classes. The class label corresponding to the maximum probability is the class label of the input point cloud, where K is the number of classes in the dataset.

Images