A point cloud classification method, device and equipment based on multi-head self-attention

Abstract

The application relates to a point cloud classification method, device and equipment based on multi-head self-attention, and belongs to the technical field of big data processing. The application obtains sample data sets of original point clouds, extracts features of the sample data sets of the original point clouds, obtains feature vectors of the point clouds, constructs a to-be-trained model based on multi-head self-attention, feeds the feature vectors into the to-be-trained model, performs model training, obtains a trained point cloud classification model based on multi-head self-attention, normalizes point cloud distribution in sample data sets of point clouds, extracts high-dimensional feature vectors, feeds the high-dimensional feature vectors into the trained point cloud classification model based on multi-head self-attention, extracts features of point clouds under different attentions, and finally obtains point cloud classification results through model operation, so as to help solve the problem that the existing point cloud classification technology is used to classify point clouds, and the classification accuracy is low due to feature loss.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of big data processing, and particularly relates to a point cloud classification method, device and equipment based on multi-head self-attention. BACKGROUND

[0002] With the rapid development of China's social economy, various dangerous chemicals (referred to as "dangerous chemicals") are also more and more widely used, but the occurrence of major safety accidents such as Zhangjiakou 11.23 explosion accident and Tianjin Port 8.12 explosion accident shows that there are still serious deficiencies in the safety management system of China's dangerous chemicals storage. The traditional manual patrol and on-duty mode is time-consuming and laborious, and the efficiency is low. The monitoring method using a remote camera improves the efficiency, but is seriously affected by the problems of shielding and illumination, so a more effective dangerous chemicals warehouse supervision method is needed.

[0003] With the development of three-dimensional imaging technology, structured light measurement and laser scanning technologies tend to be mature, and the three-dimensional coordinates of the object surface can be accurately and quickly obtained, so as to generate three-dimensional data of the scene, which can better perceive and understand the surrounding environment. Three-dimensional data contains depth information of the scene and can represent the surface shape of the object, and has broad application prospects in the fields of robots, AR / VR, human-computer interaction, remote sensing surveying and mapping and the like. Three-dimensional data has depth information, which can better solve many problems in dangerous chemicals storage, so using three-dimensional data to classify and supervise the goods in the dangerous chemicals warehouse is the mainstream research direction in recent years.

[0004] However, unlike the regular arrangement of pixels in two-dimensional images, point cloud data is unordered, which makes it difficult to directly apply convolution to obtain local correlation information between three-dimensional points. At the same time, due to the collection method, point cloud data is often non-uniformly distributed, and the point cloud density of different local areas is often not equal, which will bring difficulties to the sampling of data points when extracting features. In addition, the deformation of objects in three-dimensional space is more complex than that in two-dimensional images, and in addition to three-dimensional affine transformation, non-rigid deformation also needs to be considered. Therefore, the main problems of point cloud classification are its sparsity and disorder.

[0005] At present, the main methods of point cloud classification task mainly include three kinds of classification methods based on multi-view, methods based on voxel representation and methods based on point representation.

[0006] The multi-view based classification method is to project the three-dimensional point cloud into a two-dimensional plane, and classify the point cloud by processing the projection images at different angles. The key problem is how to fuse the features of multiple views into a distinctive global feature. Abdullah Hamdi uses an end-to-end multi-view transformation network (MVTN) to assign weights to images at different angles to find the best view for a specific task, but it requires a large number of views and has a slow operation speed. Tianyu Huang encodes the depth map at different angles using an encoder, and then uses a contrast learning method to compare the depth features and visual features, but it is greatly affected by the depth domain gap of the image.

[0007] The voxel representation based method is to voxelize the point cloud and then classify it. Maosheng Ye voxelizes at multiple scales, then uses the VoxelConv method to extract features from the voxelized point cloud, and then classifies it, but its classification speed is slow at high resolution and it occupies a large amount of memory. Lifa Zhu uses a local encoder to voxelize the point cloud, then uses pyramid pooling to fuse features at multiple scales, and finally uses an adaptive feature extraction layer to pool the features. The voxel-based method has achieved certain results, but this method loses important detailed features, so it is difficult to further improve.

[0008] Most existing methods are point-based representation methods that directly process the original data, preserving the integrity of the point cloud. Charles R. Qi first proposed a new deep learning model for processing point cloud data and verified that it can be used for various cognitive tasks of point cloud data, such as classification, semantic segmentation, and target recognition. The network uses point data as input, uses feature conversion to extract features for each point independently, then aggregates point features through a max-pooling layer, and finally uses a softmax layer to get the classification result of the point cloud. PointNet simply connects all points together, only considering global features, and losing local information for each point. Xu Ma uses a simple hierarchical network to extract local geometric features, then uses a multi-layer residual network to process local features and classify them, which reduces the parameter amount and improves the operation speed, but loses some local features.

[0009] The main drawback of all these classification techniques is the low classification accuracy due to the loss of features. SUMMARY

[0010] The present application provides a point cloud classification method, device and equipment based on multi-head self-attention, which helps to solve the problem of low classification accuracy caused by the loss of features when using existing point cloud classification techniques to classify point clouds.

[0011] To achieve the above object, the application adopts the following technical solutions:

[0012] According to a first aspect of an embodiment of the application, a point cloud classification method based on multi-head self-attention is provided, comprising:

[0013] Obtaining a sample data set of an original point cloud;

[0014] Performing feature extraction on the sample data set of the original point cloud to obtain a feature vector of the point cloud;

[0015] Constructing a to-be-trained model based on multi-head self-attention;

[0016] Sending the feature vector into the to-be-trained model to perform model training, and obtaining a trained point cloud classification model based on multi-head self-attention, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result.

[0017] Preferably, the feature extraction on the sample data set of the original point cloud to obtain the feature vector of the point cloud comprises:

[0018] After the points in the sample data set of the original point cloud are translated and scaled, a first point cloud sample data set is obtained;

[0019] Performing screening on the first point cloud sample data to delete certain noise points, and obtaining a second point cloud sample data set;

[0020] Using a single-layer fully connected neural network to extract larger-range features of each point in the second point cloud sample data set, and combining the features through maximum pooling to obtain the feature vector of the point cloud, wherein the single-layer fully connected neural network comprises a Liner layer, a BatchNorm layer and a ReLU layer.

[0021] Preferably, the use of a single-layer fully connected neural network to extract larger-range features of each point in the second point cloud sample data set comprises:

[0022] Selecting in-class points corresponding to each point in the second point cloud sample data set from the original point cloud sample data set to form an in-class point set, wherein the in-class points are neighborhood points satisfying certain conditions;

[0023] Connecting each point in the second point cloud sample data set and its corresponding in-class points using a curve;

[0024] Using a state descriptor to describe the current state of the curve;

[0025] According to the current state of the curve, the next advancing direction of the curve is determined, and the next point is selected from the original point cloud sample data set until the in-class point set contains enough points, wherein the in-class point set is a larger range of features corresponding to the points in the second point cloud sample data set.

[0026] Preferably, the to-be-trained model comprises:

[0027] An encoder and a decoder.

[0028] The encoder is configured to extract global feature vectors of the point cloud using a multi-head self-attention module, wherein the multi-head self-attention module is composed of multiple independent self-attention modules.

[0029] The decoder is configured to decode the global features to obtain a classification result of the point cloud.

[0030] Preferably, the decoder comprises:

[0031] A three-layer fully connected neural network, wherein each layer of the fully connected neural network comprises a Liner layer, a BatchNorm layer and a ReLU layer.

[0032] Preferably, the feature vectors comprise first feature vectors corresponding to training set data in the sample data set of the point cloud and second feature vectors corresponding to test set data in the sample data set of the point cloud, and the feature vectors are input into a to-be-trained model for model training to obtain a trained multi-head self-attention-based point cloud classification model, comprising:

[0033] The first feature vectors are input into the optimized to-be-trained model to obtain first classification results corresponding to the training set data.

[0034] The first classification results are compared with real classification results to obtain errors.

[0035] The to-be-trained model is optimized according to the errors.

[0036] The second feature vectors are input into the optimized to-be-trained model to obtain second classification results corresponding to the test set data.

[0037] The second classification results are compared with real classification results to obtain a classification accuracy, and the training is completed when the accuracy reaches a stable value.

[0038] The trained to-be-trained model is saved to obtain the multi-head self-attention-based point cloud classification model.

[0039] Preferably, the first feature vector is input into the optimized to-be-trained model to obtain a first classification result corresponding to the training set data; and the second feature vector is input into the optimized to-be-trained model to obtain a second classification result corresponding to the test set data, comprising:

[0040] The first feature vector is input into a multi-head self-attention module of a to-be-trained model encoder to obtain a first global feature vector;

[0041] The first global feature vector is input into a to-be-trained model decoder for decoding to obtain the first classification result corresponding to the training set data;

[0042] The second feature vector is input into a multi-head self-attention module of a to-be-trained model encoder to obtain a second global feature vector;

[0043] The second global feature vector is input into a to-be-trained model decoder for decoding to obtain the second classification result corresponding to the test set data.

[0044] Preferably, the method comprises:

[0045] Obtaining to-be-classified point cloud data;

[0046] Performing feature extraction on the to-be-classified point cloud data to obtain a feature vector of the to-be-classified point cloud;

[0047] Inputting the feature vector into a multi-head self-attention-based point cloud classification model based on claim 1 to obtain a classification result of the to-be-classified point cloud.

[0048] According to a second aspect of the embodiment of the present application, a multi-head self-attention-based point cloud classification device is provided, comprising a point cloud classification module;

[0049] The point cloud classification module is configured to execute the method described above.

[0050] According to a third aspect of the embodiment of the present application, a multi-head self-attention-based point cloud classification device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the method described above is implemented.

[0051] The above technical solution of the present application has at least the following beneficial effects:

[0052] The application obtains a sample data set of an original point cloud, extracts features of the sample data set of the original point cloud, obtains a feature vector of the point cloud, constructs a to-be-trained model based on multi-head self-attention, inputs the feature vector into the to-be-trained model, performs model training, and obtains a trained point cloud classification model based on multi-head self-attention, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result. The application normalizes the point cloud distribution in the sample data set of the point cloud, extracts a high-dimensional feature vector, inputs the high-dimensional feature vector into the trained point cloud classification model based on multi-head self-attention, extracts the features of the point cloud under different attentions, and finally obtains the point cloud classification result through model operation, which helps to solve the problem that the existing point cloud classification technology is used to classify the point cloud, and the classification accuracy is low due to the loss of features.

[0053] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0055] Figure 1 is a flowchart of a point cloud classification method based on multi-head self-attention according to an exemplary embodiment;

[0056] Figure 2 is an original point cloud distribution diagram according to an exemplary embodiment;

[0057] Figure 3 is a point cloud distribution diagram after downsampling according to an exemplary embodiment;

[0058] Figure 4 is a cyclic diagram that may be generated in the process of curve traveling according to an exemplary embodiment;

[0059] Figure 5 is a multi-head self-attention mechanism schematic block diagram according to an exemplary embodiment;

[0060] Figure 6 is a SA calculation process schematic block diagram in a single dimension according to an exemplary embodiment;

[0061] Figure 7 is a whole architecture diagram of a point cloud classification method based on multi-head self-attention according to another exemplary embodiment;

[0062] Figure 8 is a trend chart of the number of attention heads and the final classification accuracy and different head distances according to an example embodiment;

[0063] Figure 9 is an 8-head attention distribution weight grayscale chart according to an example embodiment;

[0064] Figure 10 is a 12-head attention distribution weight grayscale chart according to an example embodiment. DETAILED DESCRIPTION

[0065] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0066] Please refer to Figure 1 , Figure 1 is a flow chart of a point cloud classification method based on multi-head self-attention according to an example embodiment, as Figure 1 shown, the method comprises the following steps:

[0067] Step S11, obtaining a sample data set of an original point cloud;

[0068] Step S12, performing feature extraction on the sample data set of the original point cloud to obtain a feature vector of the point cloud;

[0069] Step S13, constructing a to-be-trained model based on multi-head self-attention;

[0070] Step S14, sending the feature vector into the to-be-trained model to perform model training, and obtaining a trained point cloud classification model based on multi-head self-attention, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result.

[0071] It should be noted that in specific practice, the point cloud classification method based on multi-head self-attention provided by the present embodiment needs to be loaded in the controller of an electronic device for running.

[0072] The electronic device includes but is not limited to desktop computers, notebook computers, etc.

[0073] It can be understood that the technical scheme provided by the embodiment obtains a sample data set of an original point cloud, performs feature extraction on the sample data set of the original point cloud to obtain a feature vector of the point cloud, constructs a to-be-trained model based on multi-head self-attention, inputs the feature vector into the to-be-trained model, performs model training, and obtains a trained point cloud classification model based on multi-head self-attention, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result. The application normalizes the point cloud distribution in the sample data set of the point cloud, extracts a high-dimensional feature vector, inputs the high-dimensional feature vector into the trained point cloud classification model based on multi-head self-attention, extracts the features of the point cloud under different attentions, and finally obtains the point cloud classification result through model operation, which helps to solve the problem that the existing point cloud classification technology is used to classify the point cloud, and the classification accuracy is low due to the loss of features.

[0074] In specific practice, the "sample data set of the original point cloud" mentioned in step S11 contains a plurality of point cloud data, and the point cloud data is used as training sample data for model training in the embodiment.

[0075] It should be noted that since the distribution of point cloud data is a sparse matrix and has very discrete feature points in space, simply using MLP to process the data will reduce the accuracy of the model, because the dispersion degree and geometric structure of different point clouds in space are different. Therefore, the application uses an affine transformation method to make the points in space have a relatively consistent distribution as a whole after translation and scaling according to the characteristics of the data set.

[0076] The attention mechanism of the existing transformer model can well aggregate global information, but it is lacking in local information extraction. NLP can achieve forced attention within a local range by using a non-global head, but point clouds have no order, and the neighborhood in space may not be adjacent in the data set, so the non-global head is not suitable for point clouds.

[0077] Therefore, the application designs a point embedding method, which combines local features through downsampling and feature aggregation. The downsampling process can filter the point cloud, remove certain noise points, and keep the data set consistent before inputting the model, thereby improving the classification performance of the model. The feature aggregation process classifies points with similar distances in space into a class, then combines them, and extracts features in a higher dimension, which increases the feature richness while reducing the number of useless features.

[0078] Since the point cloud distribution of hazardous chemical warehouse goods has similarities, such as steel bottles and glass bottles, it is difficult to classify in low dimensions, and it needs to be mapped to higher dimensions. In PointNet, MLP is used to map features to high dimensions, but local information is lost in the mapping process. Therefore, the application uses a feature aggregation method to map it to a high-dimensional space while preserving the original local features.

[0079] In specific practice, step S12 "feature extraction is performed on the sample data set of the point cloud to obtain the feature vector of the point cloud", specifically:

[0080] 1. The points in the sample data set of the point cloud are translated and scaled to obtain a first point cloud sample data set.

[0081] It should be noted that the application uses an affine transformation method to translate and scale the points in space based on the characteristics of the data set itself, so that they have a relatively consistent distribution as a whole.

[0082] In specific practice, the affine transformation method is specifically: assuming {f i} i＝1,2,…,N is a piece of point cloud with N points, where each point is represented as a three-dimensional vector. The coordinates of each point after affine transformation can be calculated using formula 1.

[0083]

[0084] where is the centroid of the sample as a whole,

[0085] is the standard deviation of the data set, d is the sample dimension, in the point cloud, d is 3, and α, β are two trainable parameters used to control the scale of scaling and translation in the affine process. ε is a control parameter, which is set to 1e-5 in this algorithm to ensure that the calculation process does not have a denominator of 0. In addition, the vector multiplication uses the Hadamard product method.

[0086] Since σ is a parameter related to the distribution of the point cloud, this method can adjust different distributed data sets to a unified form, and the transformed data set conforms to the normal distribution in form, while preserving the original geometric characteristics. A new point cloud containing N points obtained after affine transformation is the first point cloud sample data set.

[0087] 2. Filter the first point cloud sample data to remove certain noise points to obtain a second point cloud sample data set.

[0088] It should be noted that the application uses a downsampling process to filter the point cloud, delete some noise points, and keep the data set consistent before inputting the model, thereby improving the classification performance of the model.

[0089] In specific practice, the point cloud downsampling method in the application adopts the farthest point sampling (FPS) method.

[0090] The process of downsampling N points to c points using FPS is as follows:

[0091] ① Randomly select a point from the point cloud as the initial point of the sampled point set.

[0092] ② At this time, the selected points are called set A, and the points not yet selected are called set B. For the points p b in B, record the minimum distance from the j points in set A, denoted as as shown in formula 2.

[0093]

[0094] ③ Use an array D of length N to store the distances from the points in B to A,

[0095] ④ Select the point with the maximum distance in N and put it from B into A.

[0096] ⑤ Update set A and set B, and then update the value in N according to formula 3.

[0097]

[0098] ⑥ Repeat steps ④ and ⑤ until set A contains c points.

[0099] After many experiments, the application finally obtains c = 128. At this time, the point cloud retains the overall characteristics after downsampling and reduces the number of point clouds. The point cloud distribution before and after sampling is shown in Figure 2 and Figure 3 , wherein Figure 2 is a diagram showing the distribution of the original point cloud according to an example embodiment, Figure 3 is a diagram showing the distribution of the point cloud after downsampling according to an example embodiment. The new point cloud containing 128 points obtained after downsampling of the point cloud is the second point cloud sample data set.

[0100] 3. Use a single-layer fully connected neural network to extract larger range features of each point in the second point cloud sample data set, and combine the features through maximum pooling to obtain a feature vector of the point cloud, wherein the single-layer fully connected neural network includes a Liner layer, a BatchNorm layer and a ReLU layer.

[0101] It should be noted that the application uses a feature aggregation method to map the original local features to a high-dimensional space while retaining the original local features.

[0102] In specific practice, the feature aggregation process includes clustering and merging, wherein the clustering is performed using a curve clustering method, then a single-layer fully connected neural network is used to extract larger range features of each point, and finally a maximum pooling method is used to merge multiple features in each class into one feature.

[0103] In specific practice, assuming that a second point cloud sample dataset A containing 128 points is obtained after down-sampling the point cloud, the feature aggregation process is to find m neighborhood points of any point p in A in the original point cloud sample dataset B according to certain conditions, and then merge these points into a feature, and the overall calculation process is shown in formula 4. A,i

[0104] f i =MP(LBR{p A,i ,p B,j |j=1,2,……,m}) (4)

[0105] Wherein, p A,i represents the i-th point in the second point cloud sample dataset A after down-sampling, p B,j represents the j-th neighborhood point extracted from the original point cloud sample dataset B, m represents the number of neighborhood points, LBR is a module containing three network layers of Linear, BatchNorm and ReLU, the input dimension is 3 and the output dimension is 64, MP is maximum pooling, and f i represents the feature of p A,i and its neighborhood points after feature aggregation.

[0106] In specific practice, the curve clustering method specifically includes the following steps:

[0107] Selecting class-in points corresponding to each point in the second point cloud sample dataset from the original point cloud sample dataset to form a class-in point set, wherein the class-in point is a neighborhood point satisfying certain conditions;

[0108] Connecting each point in the second point cloud sample dataset with its corresponding class-in point using a curve;

[0109] Using a state descriptor to describe the current state of the curve;

[0110] ​According to the current state of the curve, the next point is selected from the original point cloud sample data set until the in-class point set contains enough points, wherein the in-class point set is a larger range of features corresponding to the points in the second point cloud sample data set.

[0111] In specific practice, the existing feature aggregation method generally uses k-means as the condition for selecting neighborhood points, but k-means only uses distance as the clustering condition, while the data of the hazardous chemical point cloud is relatively precise and has the characteristics of similar local features, so using only distance information when clustering will produce errors.

[0112] To solve the above problems, a curve clustering method is proposed to obtain a larger range of features. For the point p A,i in A, a series of related points are selected in B, called the in-class points of p A,i . In the selection process, the curve is used to connect p A,i and its in-class points, and then the state descriptor is used to describe the current state of the curve, and the next point is selected according to the current state of the curve until the in-class point set contains enough points. A,

[0113] First, an initial point p is selected.The process of calculating the initial feature descriptor is shown in formula 5.

[0114] s0=BR(p A, )(5)

[0115] where the input dimension of LBR is 3 and the output dimension is 16, and s0 is the initial feature descriptor. Then, m nearest neighbors are found in B centered on p A,i , and the score after selecting each point is calculated, as shown in formula 6.

[0116] α j =BR2(MP(s0,LBR1(p B,1,j )),j=1,2,……,m(6)

[0117] where the input dimension of LBR1 is 3 and the output dimension is 16; the input dimension of LBR2 is 16 and the output dimension is 1; p B,1,j represents the jth point in the neighborhood when the first point is selected, and α j is the score of p B,1,j . The point with the highest score is selected as the first selected point, i.e. p B,1 = B,1,(()) At this time, the curve contains two points, so the state descriptor of the curve also needs to be updated, and the updating process is shown in formula 7.

[0118] s1= P(s0, LBR(p B,1 ))(7)

[0119] where the input dimension of LBR is 3 and the output dimension is 16, and s1is the updated feature descriptor. Assuming that k points have been selected, when the k+1thpoint is selected, m neighbor points are selected in B centered at the kthpoint, and the score and the updated state descriptor of each neighbor point are calculated according to formulas 8-10.

[0120] α j = LBR2(MP(s k , LBR1(p B,k+1,j )), j = 1, 2, …, m (8)

[0121] p B,k+1 = p B,k+1,argmax(softmax(α)) (9)

[0122] s k+1 = MP(s k , LBR1(p B,k+1 )) (10)

[0123] The process of selecting a point on a curve is essentially to score all possible next points according to the current state of the curve, and then select the point with the highest score as the in-class point. This simple scoring method may cause a loop because for a fixed curve, formula 8 will always have the same output, so the curve should be avoided from selecting a repeated point during its travel. The loop that the curve can produce is shown in Figure 4 , Figure 4 is a schematic diagram of a loop that can be produced during the travel of a curve according to an exemplary embodiment.

[0124] For a single-point loop, this loop can be avoided by excluding the self-point. For other possible loops, this article adjusts the direction of the curve travel to avoid the occurrence of the loop. Assuming that the currently selected point is p B,k , and the current travel direction of the curve is The direction vector of p B,k to each candidate point in its neighborhood is calculated. Then the direction weight of each neighborhood point is calculated using formula 11.

[0125]

[0126] In formula 10, when the angle between the travel direction of the curve and the travel direction of the candidate point is larger, the corresponding direction weight is smaller, and when the two vectors are completely opposite, d j= 0. Using the direction weight can change the score of each candidate point, thereby avoiding the generation of a loop, at which time formula 8 becomes formula 12.

[0127] a j = d j · LBR2(MP(s k , LBR1(p B,k+1,j )), j = 1, 2, …, m (12)

[0128] In specific practice, the number of points contained in each point in the second point cloud sample data set after downsampling should be determined according to the number of point clouds in the original point cloud sample data set. Through experiments, the curve clustering method of the present application selects m = 32 intra-class points for each point in A.

[0129] Therefore, each point in A and its intra-class points are combined into a feature vector using formula 12, and the specific process is shown in formula 13.

[0130] f i = MP(LBR(p A,i , p B,k | k = 1, 2, …, 32)) (13)

[0131] Where p A,i represents the i-th point in set A, p B,k represents the k-th intra-class point of p A,i , the input dimension of LBR is 3, the output dimension is 64, MP is the maximum pooling, f i represents the i-th merged feature vector of the cluster, and f i is the feature vector of the point cloud.

[0132] In specific practice, the “to-be-trained model” mentioned in step S13 includes: an encoder and a decoder. The encoder is used to extract a global feature vector of the point cloud using a multi-head self-attention module, wherein the multi-head self-attention module is composed of multiple independent self-attention modules; and the decoder is used to decode the global feature to obtain a classification result of the point cloud.

[0133] In specific practice, preferably, the decoder is composed of a three-layer fully connected neural network, wherein each layer of the fully connected neural network includes: a Liner layer, a BatchNorm layer and a ReLU layer.

[0134] It should be noted that the general point cloud classification method currently used is PointNet, which achieves a high accuracy, but it only extracts features from individual points, losing the joint features of each point and its surrounding points.

[0135] Inspired by Vaswani A, the application designs a method of using a multi-head self-attention module to extract global features of a point cloud, and then classifying. The self-attention (SA) module has the same dimension of input and output. SA aggregates the joint features of different position points in space according to the attention degree between different features. However, in the point cloud, different categories of points may have different weight matrices, and a single SA module shares weights when processing, so the application uses a multi-head self-attention (MHSA) module, that is, uses multiple independent SA modules to extract the features of the point cloud under different attention, and these SA modules are independent of each other, and each feature only allocates attention within the corresponding SA.

[0136] In specific practice, the "feature vector" mentioned in step S14 includes: a first feature vector corresponding to the training set data in the sample data set of the point cloud and a second feature vector corresponding to the test set data in the sample data set of the point cloud.

[0137] It should be noted that the first feature vector and the second feature vector are obtained by respectively extracting features from the training set data and the test set data, and the training set data and the test set data are both from the sample data set of the point cloud.

[0138] In specific practice, step S14 "sends the feature vector into the to-be-trained model, performs model training, and obtains a trained multi-head self-attention-based point cloud classification model" specifically includes:

[0139] 1. Send the first feature vector into the optimized to-be-trained model to obtain a first classification result corresponding to the training set data.

[0140] In specific practice, sending the first feature vector into the optimized to-be-trained model to obtain a first classification result corresponding to the training set data specifically includes:

[0141] sending the first feature vector into the multi-head self-attention module of the to-be-trained model encoder to obtain a first global feature vector;

[0142] sending the first global feature vector into the to-be-trained model decoder for decoding to obtain the first classification result corresponding to the training set data.

[0143] 2. Compare the first classification result with the real classification result to obtain an error.

[0144] 3. Optimize the to-be-trained model according to the error.

[0145] In specific practice, the weight matrix parameters of each network layer in the to-be-trained model are optimized according to the error using a stochastic gradient descent method.

[0146] 4. The second feature vector is input into the optimized to-be-trained model to obtain a second classification result corresponding to the test set data.

[0147] In specific practice, the second feature vector is input into the optimized to-be-trained model to obtain a second classification result corresponding to the test set data, specifically:

[0148] The second feature vector is input into the multi-head self-attention module of the to-be-trained model encoder to obtain a second global feature vector.

[0149] The second global feature vector is input into the to-be-trained model decoder for decoding to obtain the second classification result corresponding to the test set data.

[0150] 5. The second classification result is compared with the true classification result to obtain a classification accuracy, and the training is completed when the accuracy reaches a stable value.

[0151] 6. The to-be-trained model after training is saved to obtain the point cloud classification model based on the multi-head self-attention.

[0152] In specific practice, the first feature vector is input into the multi-head self-attention module of the to-be-trained model encoder to obtain a first global feature vector, and the second feature vector is input into the multi-head self-attention module of the to-be-trained model encoder to obtain a second global feature vector, and the specific process is as follows:

[0153] Please refer to Figure 5 , Figure 5 is a schematic block diagram of a multi-head self-attention mechanism according to an exemplary embodiment, as shown in Figure 5 The first feature vector or the second feature vector is input as an input feature into the multi-head self-attention module to extract point cloud features under different attentions. The input feature is first extracted by each SA module to obtain a corresponding output feature, and then each output feature is concatenated in row to obtain a final feature. Since the final feature generated by the multi-head self-attention module is a feature of multiple dimensions, in order to facilitate subsequent processing, the final feature needs to be reduced in dimension. A three-layer fully connected neural network is used in the present application to reduce the dimension of the final feature to obtain a final global feature, i.e., a first global feature vector or a second global feature vector.

[0154] It should be noted that the number of attention heads is assumed to be N h , and the feature dimension of each SA output is d eThe dimension of the final feature is N after concatenating the features output by each SA in a row h d e , where d e is the feature dimension. By adjusting the value of the number of attention heads N h , the input and output are no longer the same. Since the multi-head self-attention module generates features of multiple dimensions, in order to facilitate subsequent processing, it needs to be reduced in dimension. The present application uses a three-layer fully connected neural network to reduce the dimension, and the number of neurons is 4 e , 2 e and d e , respectively. At this time, the output of the MHSA module is F outm , which is the final global feature. Through experiments, the number of attention heads (the number of SA modules) N h = 8 is finally taken, and the dimension of the final feature vector is 8 e , where d e is the feature dimension, and the output and the input are no longer the same. The three-layer fully connected neural network consists of: a Liner layer, a BatchNorm layer and a ReLU layer.

[0155] The first global feature vector is sent to the decoder of the model to be trained for decoding to obtain the first classification result corresponding to the training set data, and the second global feature vector is sent to the decoder of the model to be trained for decoding to obtain the second classification result corresponding to the test set data, specifically: F outm is the final global feature, that is, the first global feature vector or the second global feature vector.

[0156] Since F outm is a highly aggregated feature, it needs to be decoded using a decoder to obtain the classification result of the point cloud. The decoder can be generally summarized by formula 14.

[0157] c P = softmax((LBR(LBR(MP(F outm )))))(14)

[0158] Where MP represents maximum pooling, and c P is the classification result of the point cloud.The probability distribution of the object classification category is obtained. First, the output of the encoder is maximum-pooled to retain the most significant part of the feature, and then a three-layer fully connected neural network is used for decoding, with neuron numbers of 256, 256 and 40 respectively, and the result is normalized using the softmax normalization exponential function, and the probability distribution of the object category is output, and the class with the highest probability is taken as the classification result of the object. The three-layer fully connected neural network is composed of: Liner layer, BatchNorm layer and ReLU layer.

[0159] C P The class with the highest probability is taken as the classification result of the point cloud, i.e. the first classification result or the second classification result.

[0160] In specific practice, MHSA only uses multiple SAs, and these SAs are independent of each other, and each feature only allocates attention within the corresponding SA, so the calculation process of each SA is the same.

[0161] Please refer to Figure 6 , Figure 6 The SA calculation process in a single dimension is shown in FIG. 1 according to an exemplary embodiment, as shown in FIG. 2, which shows the SA calculation process of two features in one dimension as follows: Figure 6 Figure 6 In the figure, f1 and f2 are feature vectors after point embedding, which are multiplied by three weight matrices respectively to obtain the query vector q, the key vector k and the value vector v of each feature. For feature f1, the attention weight w1 is obtained by multiplying the query vector q1 of itself with the transpose T of the key vector k1 of itself, and the weighted feature f1' is obtained by multiplying the attention weight w1 with the value vector v1 of itself. The attention weight w2 is obtained by multiplying the query vector q1 of f1 with the transpose T of the key vector k2 of f2, and the weighted feature f2' is obtained by multiplying the attention weight w2 with the value vector v2 of f2. The output fout1 of the SA module after the summation of all weighted features (i.e. the summation of the weighted feature f1' and the weighted feature f2') is obtained.

[0162] When using the SA module on point cloud data, the entire point cloud can be regarded as a sentence, and each embedded feature is regarded as a word. Assuming that in the previous section, the output of point embedding is a feature map with N d e dimensional vectors For F e , the correlation between each feature and the similarity between different features is calculated. SA first extracts the query matrix Q, the key matrix K and the value matrix V of the point cloud, and the specific calculation process is shown in formula 15.

[0163] Q,K,V,=F e ·(W​q ,W k ,W v ) (15)

[0164]

[0165] where W q ,W k ,W v is a shared weight and learnable linear transformation, d a is the dimension of the query matrix and the key matrix, d a and d e are not required to be equal. Then the attention degree between different feature vectors is calculated using Q and K to obtain an attention matrix, as shown in equation 16.

[0166]

[0167] where Φ' is the attention matrix. For the elements in Φ', do softmax by row to obtain the final weight matrix Φ, as shown in equation 17.

[0168]

[0169] Finally, multiply the weight matrix with the value matrix to obtain the final output of the SA module, as shown in equation 18.

[0170] F out = Φ·V (18)

[0171] The processing process of a single SA in MHSA is the same as the above process, except that multiple SAs are used, and these SAs are independent of each other, and each feature only allocates attention within the corresponding SA, so it can be processed in parallel.

[0172] In specific practice, preferably, the method comprises:

[0173] obtaining point cloud data to be classified;

[0174] performing feature extraction on the point cloud data to be classified to obtain a feature vector of the point cloud to be classified;

[0175] sending the feature vector into a point cloud classification model based on multi-head self-attention according to claim 1 to obtain a classification result of the point cloud to be classified.

[0176] Referring to Figure 7 , Figure 7 is a whole architecture diagram of a point cloud classification method based on multi-head self-attention according to another exemplary embodiment, as shown in Figure 7As shown, given a piece of three-dimensional point cloud data with a point number of N, the dimension is N*3. First, the data is subjected to affine transformation, which does not change the dimension of the data; then the feature is aggregated to obtain a high-dimensional feature map, at this time the point number is reduced to 128, but the dimension of the feature on each point is increased to 64 dimensions. Then the feature map is input into an encoder composed of multiple self-attention modules, the output of each self-attention module is concatenated in row, the final feature is obtained, then the final feature is passed through a three-layer fully connected neural network, the number of neurons is 256, 256, 40 respectively, and the result is normalized by using softmax, the probability distribution of the object is output, and the class with the maximum probability is taken as the classification result of the object.

[0177] It can be understood that the application provides a point cloud recognition model based on a multi-head self-attention mechanism, which classifies point clouds by extracting the relationship between points. First, the point cloud sample is subjected to affine transformation, so that it has a uniform distribution in space, then the local adjacent points are aggregated into a feature vector, and then the feature is encoded to obtain a high-dimensional feature. Finally, the high-dimensional feature is decoded by using a decoder to obtain the final classification result.

[0178] According to a second aspect of the embodiment of the application, a point cloud classification device based on multi-head self-attention is provided, comprising: a point cloud classification module;

[0179] The point cloud classification module is used to execute the method described above.

[0180] It can be understood that the point cloud classification module in the point cloud classification device based on multi-head self-attention is used to execute the method described above, by obtaining a sample data set of an original point cloud, extracting features of the sample data set of the original point cloud to obtain a feature vector of the point cloud, constructing a to-be-trained model based on multi-head self-attention, inputting the feature vector into the to-be-trained model, performing model training, and obtaining a trained point cloud classification model based on multi-head self-attention, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result. The application normalizes the point cloud distribution in the sample data set of the point cloud, extracts a high-dimensional feature vector, inputs the high-dimensional feature vector into the trained point cloud classification model based on multi-head self-attention, extracts the features of the point cloud under different attentions, and finally obtains the point cloud classification result by model operation, which helps to solve the problem that the existing point cloud classification technology classifies point clouds, resulting in low classification accuracy due to feature loss.

[0181] The application provides a point cloud classification device based on multi-head self-attention, comprising: a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor executes the computer program to implement the method described above.

[0182] As to the point cloud classification device based on multi-head self-attention in the above embodiment, the memory and the processor specifically execute the method process, which has been described in detail in the embodiment related to the method, and will not be described in detail here.

[0183] It can be understood that when the processor in the point cloud classification device based on multi-head self-attention executes the computer program, the above method is realized, the sample data set of the original point cloud is obtained, feature extraction is performed on the sample data set of the original point cloud, the feature vector of the point cloud is obtained, the to-be-trained model based on multi-head self-attention is constructed, the feature vector is input into the to-be-trained model, model training is performed, and the trained point cloud classification model based on multi-head self-attention is obtained, wherein the point cloud classification model based on multi-head self-attention is used to obtain a point cloud classification result. By normalizing the point cloud distribution in the sample data set of the point cloud, extracting a high-dimensional feature vector, inputting the high-dimensional feature vector into the trained multi-head self-attention point cloud classification model, extracting the features of the point cloud under different attentions, and performing model operation, the point cloud classification result is finally obtained, which helps to solve the problem that the existing point cloud classification technology is used to classify the point cloud, and the classification accuracy is low due to the loss of features.

[0184] In the safety state monitoring of the deep camera point cloud three-dimensional reconstruction of the dangerous chemical warehouse, there are problems such as a spacious space, sparse distribution of cargo image point cloud, and similar classification under low dimension. Based on the above problems, a point cloud recognition method based on a multi-head self-attention mechanism is proposed. The algorithm first normalizes the data set distribution through an affine transformation algorithm, solves the problem of sparse distribution, then fuses data downsampling and aggregation feature algorithm to obtain a high-dimensional warehouse cargo feature map, solves the problem of approximate distribution under low dimension, and then uses a multi-head self-attention encoder to encode the feature map to obtain features under different heads, and combines them into a feature map. Finally, a multi-layer fully connected neural network is used as a decoder to decode the feature map into the final object classification. Comparative experiments are performed on the modelnet40 data set and the self-built data set of dangerous chemical cargo, and the results show that compared with other different classification algorithms, the accuracy of the point cloud classification of the dangerous chemical warehouse cargo image is improved from 0.5% to 7.8%.

[0185] The hardware environment of the present application is Intel Core i9-10900k, Geforce RTX3090 graphics card, 32GB running memory, the debugging environment is Windows 10, python 3.8.3, pytorch 1.10.1, and the cuda version is 11.3.

[0186] The data set used in the comparative experiment in the present application is ModelNet40 data set and self-built data set of dangerous goods warehouse goods. ModelNet40 has a total of 12311 man-made object gridded CAD models, each object has a corresponding category, and there are 40 categories in total. 9843 models in the data set are used for training, and 2486 models are used for testing. In the self-built data set of dangerous goods warehouse goods, there are 8 categories, including 5 dangerous goods, namely oil drum, carton, glass bottle, porcelain bottle and steel bottle, and three kinds of non-dangerous goods sundries, namely table, chair and person.

[0187] In the experimental parameter setting, the optimization method is random gradient descent, the training round is 250 times, the batch size is 32, the learning rate is exponentially decreased from 0.1 to 0.0001, BN normalization and ReLU activation function are used for each network layer, and a dropout layer with a parameter of 0.3 is added to the network layer in the decoder.

[0188] Affine transformation reduces the error caused by the difference in point cloud distribution and geometric structure by normalizing the distribution of the object. In the ModelNet40 data set, the present application trains the model using or not using affine transformation for some existing point cloud classification methods and the present application method, and compares them. The classification accuracy results are shown in Table 1. The experiment shows that affine transformation can improve the classification accuracy to a certain extent.

[0189]

[0190]

[0191] Table 1 Classification accuracy of model using affine transformation

[0192] In order to explore the mechanism of attention head, the present application also studies the case of different number of heads. Sangeetha K believes that the purpose of using multi-head self-attention in attention mechanism is to increase the distribution of different attention and improve the classification accuracy. Therefore, the attention positions of different heads should be different. Attention head is essentially a vector, so the vector distance can be used to measure the difference in attention degree between different heads. If the distance between two heads is small, it means that the two heads pay attention to similar positions, and vice versa, it means that the two heads pay attention to different positions of the features. Figure 8 The average distance between different heads and the change trend of the final classification accuracy with the increase of the number of heads are shown in Table 2.

[0193] From Figure 8As can be seen from Table 1, the accuracy of classification is improved with the increase of the number of heads, and when the number reaches 8, the accuracy reaches 93.7%. When the number of heads continues to increase, the distance between heads decreases, at this time it is indicated that the attention positions of different heads are overlapped, but the accuracy is no longer improved, and since the amount of parameters is increased, the classification speed is decreased, therefore the number of attention heads used in the present application is 8.

[0194] In order to specifically explore the difference in attention weight distribution when the number of heads is different, the present application uses the way of gray scale map to show the weight allocated by each head under each feature. Figure 9 and Figure 10 and

[0195] In which each column represents the weight of the feature of a certain position in different heads, and the weight size is represented by gray scale, the lighter the color, the greater the weight. Comparing Figure 9 and Figure 10 As can be seen from Table 1, the accuracy of classification is improved with the increase of the number of heads, and when the number reaches 8, the accuracy reaches 93.7%. When the number of heads continues to increase, the distance between heads decreases, at this time it is indicated that the attention positions of different heads are overlapped, but the accuracy is no longer improved, and since the amount of parameters is increased, the classification speed is decreased, therefore the number of attention heads used in the present application is 8.

[0196] In the ModelNet40 dataset, there are 40 classes of point clouds. The algorithm of the present application is compared with other classification algorithms, and the evaluation indexes mainly include overall class accuracy (Overall Accuracy, OA) and mean class accuracy (meanAccuracy, mAcc). The results show that the algorithm of the present application has the optimal performance in OA and mAcc. The specific results are shown in Table 2.

[0197]

[0198]

[0199] Table 2 Classification accuracy on ModelNet40 dataset

[0200] As can be seen from Table 2, compared with the voxelized classification method VoxNet, the local feature aggregation method used in the application avoids the loss of detailed features caused by voxelization, thus improving the overall classification accuracy by 7.8%; compared with the multi-view classification method MVCNN, the application directly processes the points, avoiding the loss of features in the process of converting three-dimensional point cloud into two-dimensional plane, and improving the overall classification accuracy by 2.6%. Compared with the classic point-based classification algorithms PointNet, PointNet++ and PointCNN, the application considers the correlation features between points, and improves the overall classification accuracy by 1.5% to 4.5%. Compared with the point-based methods such as DGCNN and PCT, the multi-head self-attention used in the application can better extract the differences between different categories, and improves the overall classification accuracy by 0.5% to 0.8%.

[0201] In the self-built dangerous chemical warehouse goods data set, there are point clouds of 8 categories of objects. The application algorithm is compared with other classification algorithms, and the evaluation indexes are also OA and mAcc. The results show that the application algorithm has the optimal performance in OA and mAcc. The specific results are shown in Table 3.

[0202]

[0203] Table 3 Classification accuracy on self-built data set

[0204] As can be seen from Table 3, compared with the classic PointNet, the application algorithm improves the overall accuracy by 2.5%. Compared with some existing point-based algorithms, the application method improves the overall accuracy by 0.1% to 5.5%.

[0205] In addition, the application provides a computer readable storage medium, which stores computer instructions for making a computer execute the steps of any of the above methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0206] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0207] It should be noted that, in the description of the present application, the terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise stated, the meaning of "a plurality of" or "multiple" is at least two.

[0208] It should be understood that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or a middle element can be present at the same time; when an element is referred to as "connected to" another element, it can be directly connected to the other element or a middle element can be present at the same time, in addition, "connected" used herein can include wireless connection; the phrase "and / or" used includes any unit and all combinations of the associated listed items.

[0209] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing one or more modules, segments, or portions of code that include executable instructions for performing specific logical functions or steps in the process, and the preferred embodiments of the present application include additional implementations in which the functions are performed in different orders, in substantially simultaneous fashion, or in reverse order, depending on the functionality involved, as will be understood by those skilled in the art.

[0210] It should be understood that parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0211] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by program instructions to relevant hardware, and the program can be stored in a computer readable storage medium, which includes one or a combination of steps of the method embodiments when executed.

[0212] In addition, each of the function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware, or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0213] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0214] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0215] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A point cloud classification method based on multi-head self-attention, characterized in that, include: Obtain a sample dataset of the original point cloud, where the original point cloud is an image point cloud; Feature extraction is performed on the sample dataset of the original point cloud to obtain the feature vector of the point cloud; Construct a trainable model based on multi-head self-attention; The feature vector is fed into the model to be trained to train the model and obtain a trained point cloud classification model based on multi-head self-attention. The point cloud classification model based on multi-head self-attention is used to obtain the point cloud classification result. Feature extraction is performed on the sample dataset of the original point cloud to obtain the feature vector of the point cloud, including: The first point cloud sample dataset is obtained by translating and scaling the central points of the original point cloud sample dataset. The first point cloud sample data is filtered, and some noise is removed to obtain the second point cloud sample dataset; A single-layer fully connected neural network is used to extract features over a larger range for each point in the second point cloud sample dataset, and the features are merged by max pooling to obtain the feature vector of the point cloud. The single-layer fully connected neural network includes: a liner layer, a batchNorm layer, and a ReLU layer. A single-layer fully connected neural network is used to extract a wider range of features for each point in the second point cloud sample dataset, including: Select intra-class points from the original point cloud sample dataset that correspond to each point in the second point cloud sample dataset to form an intra-class point set, wherein the intra-class points are neighborhood points that satisfy certain conditions; Connect each point in the second point cloud sample dataset with its corresponding in-class point using a curve; Use a state descriptor to describe the current state of the curve; The direction of the curve's next movement is determined based on the current state of the curve, and the next point is selected from the original point cloud sample dataset until the intra-class point set contains enough points, wherein the intra-class point set is the larger range of features corresponding to the points in the second point cloud sample dataset.

2. The method according to claim 1, characterized in that, The model to be trained includes: Encoder and decoder; The encoder is used to extract global feature vectors from the point cloud using a multi-head self-attention module, wherein the multi-head self-attention module is composed of multiple independent self-attention modules; The decoder is used to decode the global features to obtain the classification result of the point cloud.

3. The method according to claim 2, characterized in that, The decoder includes: A three-layer fully connected neural network, wherein each layer of the fully connected neural network includes: a Liner layer, a BatchNorm layer, and a ReLU layer.

4. The method according to claim 3, characterized in that, The feature vector includes: a first feature vector corresponding to the training set data in the sample dataset of the point cloud and a second feature vector corresponding to the test set data in the sample dataset of the point cloud; and, the feature vector is fed into the model to be trained for model training to obtain a trained point cloud classification model based on multi-head self-attention, including: The first feature vector is fed into the optimized model to be trained to obtain the first classification result corresponding to the training set data; The first classification result is compared with the true classification result to obtain the error; The training model is optimized based on the error. The second feature vector is fed into the optimized model to be trained to obtain the second classification result corresponding to the test set data; The second classification result is compared with the true classification result to obtain the classification accuracy. Training is completed when the accuracy reaches a stable value. Save the trained model to be trained to obtain the point cloud classification model based on multi-head self-attention.

5. The method according to claim 4, characterized in that, The first feature vector is fed into the optimized model to be trained to obtain the first classification result corresponding to the training set data; And, the second feature vector is fed into the optimized model to be trained to obtain the second classification result corresponding to the test set data, including: The first feature vector is fed into the multi-head self-attention module of the encoder of the model to be trained to obtain the first global feature vector. The first global feature vector is fed into the decoder of the model to be trained for decoding to obtain the first classification result corresponding to the training set data; Furthermore, the second feature vector is fed into the multi-head self-attention module of the encoder of the model to be trained to obtain the second global feature vector; The second global feature vector is fed into the decoder of the model to be trained for decoding to obtain the second classification result corresponding to the test set data.

6. A point cloud classification device based on multi-head self-attention, characterized in that, Includes a point cloud classification module; The point cloud classification module is used to perform the method described in any one of claims 1-5.

7. A point cloud classification device based on multi-head self-attention, characterized in that, include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1-5.

Images