Lightweight multi-stage point cloud classification method based on fusion Transform network

Abstract

The invention discloses a lightweight multi-stage point cloud classification method based on a fusion Transform network. The lightweight multi-stage point cloud classification method comprises the steps of original point embedding module construction, local feature extraction, global feature aggregation and multi-stage learning. The network is composed of an efficient feature aggregation branch and a Transform branch, the efficient feature aggregation branch is provided with a learnable position code, and position information which is often neglected by most point cloud classification networks can be fully captured. Sampling points with similar characteristics are clustered by the Transform branch, so that the balance between capturing long-range dependence and calculation complexity is realized. Meanwhile, an extrusion-excitation module is introduced into original point embedding, so that representation of point cloud features is enhanced, and subsequent point cloud learning performance is improved. And a four-stage learning strategy is adopted, so that the understanding capability of the network on the point cloud is effectively enhanced. A large number of experiments prove that excellent classification performance can be achieved while the number of parameters is kept low.

Description

Technical Field

[0001] The present invention belongs to the field of laser radar point cloud data processing, and in particular relates to a lightweight multi-stage point cloud classification method based on a fused Transformer network. Background Art

[0002] Point cloud classification is an important task in point cloud data processing. It has shown great potential in a variety of applications such as urban information modeling, autonomous driving, robotics, and 3D mapping. Unlike 2D images, 3D point clouds are characterized by disorder, irregularity, and sparsity, which poses challenges to the direct application of deep learning-based methods. To address these challenges, many point cloud classification architectures have been proposed in recent years and have achieved remarkable results in point cloud classification tasks. However, most methods unilaterally focus on local features or global features without integrating the two features. In contrast, Transformer, as a new point cloud local and global feature extraction architecture, has achieved excellent results in point cloud classification tasks. Various Point Cloud Transformer methods have been continuously introduced, achieving state-of-the-art results in point cloud classification. However, existing Transformer networks have challenges in terms of structural complexity and the number of parameters, and the exploration of point cloud position encoding is not deep enough. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a lightweight multi-stage point cloud classification method based on a fused Transformer network. The lightweight multi-stage point cloud Transformer classification network (Lightweight Multi-Stage Point Cloud Classification Network Integratedwith Transformer, LMPT for short) of the present invention cleverly integrates efficient feature aggregation branches with Transformer branches, and can effectively extract and integrate local and global information. At the same time, a learnable position encoding is introduced to effectively integrate point cloud geometric information that is often overlooked by many point cloud Transformer architectures. In addition, the SE module contained in the original point embedding significantly enhances the feature extraction capability of the model. The network architecture of the present invention adopts a four-stage learning strategy to improve classification accuracy while ensuring a low number of parameters.

[0004] Technical solution: A lightweight multi-stage point cloud classification method based on a fused Transformer network of the present invention takes the original Lidar point cloud data as input, executes the following steps S1 to S4, obtains a high-dimensional representation of point features through an original point embedding module, then sequentially performs local feature extraction and global feature aggregation to obtain the residual results of each stage, and finally multi-stages learns each residual result to obtain the final classification result output, the steps are as follows:

[0005] Step S1: construct an original point embedding module to project the original point cloud to a higher dimension. The original point embedding module includes a linear layer and a squeeze-excitation SE module Squeeze-Excitation;

[0006] Step S2: local feature extraction, the embedded high-dimensional point cloud features are respectively extracted from the local information by using the farthest point sampling FPS and k nearest neighbor algorithm kNN and sorting and kNN algorithm through two branches, and the number of point clouds is downsampled;

[0007] Step S3: Global feature aggregation: After the local features extracted by the two branches are integrated accordingly, they are respectively passed to the learnable position encoding layer and the self-attention layer to achieve global feature aggregation. Then, the results of the two branches are integrated to obtain the residual results of each stage.

[0008] Step S4: Multi-stage learning, adopting a four-stage learning strategy, taking the residual result of each stage as the input of the next stage, the downsampling rate of each stage is 1 / 2, and the results after the four-stage learning are passed into the classifier to obtain the final classification result.

[0009] Furthermore, the specific steps of step S1 are as follows:

[0010] Step S1.1: Project the point cloud information into a higher dimensional channel. In order to minimize the amount of unnecessary parameters, a one-dimensional convolution operation is used, as shown below:

[0011]

[0012] In the formula, x n is the input channel, h k is the convolution kernel size, y n For the output channel, represents the sum of the convolution kernel indices from k = 0 to k = K-1;

[0013] Step S1.2: After the one-dimensional convolution, the SE module is used to solve the problem of low discrimination between channels. The importance weight of each channel is learned to enhance the network's utilization of different channel information. First, the spatial information of each channel is compressed into a single value through compression operation.

[0014]

[0015] In the formula, H and W are the height and width of the feature map, z is the output of the result, and x ij is the specific value in the feature map; then, a two-layer fully connected neural network is used to recalibrate the importance of each channel, as shown below:

[0016] s=σ(W2δ(W1z)) (3)

[0017] Where W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU nonlinear layer, and σ represents the sigmoid activation function. Finally, in order to obtain the output feature map The calibration vector s is multiplied element-wise with the original input feature map X on a channel-by-channel basis as follows:

[0018]

[0019] Here, × represents element-by-element multiplication.

[0020] Furthermore, the specific steps of step S2 are as follows:

[0021] Step S2.1: Local feature extraction in the efficient feature aggregation branch. After the initial point embedding layer, the FPS method is used to select a subset of local center points to obtain the downsampling process of N point clouds, as shown in equation (5), and finally N / 2 point clouds are obtained;

[0022]

[0023] In the formula, F i and P i Respectively represent the features and coordinates of the i-th original point, F c and P c They represent the features and coordinates of the cth center point in the subset respectively; then, the kNN algorithm is used to group the points into k neighboring spaces to form k local 3D regions for local feature extraction, as shown below:

[0024]

[0025] Where N k Represents the kth neighbor space. The coordinates of the N original points and the local center point subset output by FPS are used as the input of kNN; through these two steps, local features and coordinates are obtained;

[0026] Step S2.2: Local feature extraction in the Transformer branch. Compared with the previous part, the distance between two point clouds is directly calculated to extract the location information of the point clouds in more detail.

[0027] P n =||xyz i -xyz j || 2 (7)

[0028] Where P n is the distance between point i and point j, xyz i and xyz j Represent the coordinates of the i-th and j-th points respectively; then, similar to step S2.1, the calculated point cloud distance is used as the input of the subsequent kNN analysis; first, the point cloud distances are sorted and the k nearest neighbors are selected, and then the coordinates of the corresponding point cloud are extracted according to the distance index, as shown below:

[0029]

[0030] knn_xyz k =xyz j |j∈knn_idx k (9)

[0031] In the formula, argsort k Indicates that all Sort the distances of the points and select the indexes of the first k points as knn_idx k ; Then, use knn_idx k As an index to find the corresponding point coordinates, so as to obtain knn_xyz k .

[0032] Furthermore, the specific steps of step S3 are as follows:

[0033] Step S3.1: Global feature aggregation in the efficient feature aggregation branch. The function of this step is to aggregate the local features extracted in step S2.1 and obtain the global information of the point cloud. First, the neighbor points and the center point from the FPS and kNN operations are connected to prevent the loss of global information, as follows:

[0034] F kc =Concat(f k ,F k ) (10)

[0035] In the formula, F kc It effectively aggregates global and local information and serves as the input of the LPE layer;

[0036] Subsequently, learnable parameters and kNN point coordinates are introduced into the trigonometric position encoding and weighted. Due to the characteristics of the sine function, the trigonometric function can not only describe the precise position in the embedding space, but also implicitly capture the relative position details between two points in the three-dimensional environment. Therefore, starting from a parameter-free trigonometric position encoding, the learning position encoding design is carried out as follows:

[0037]

[0038]

[0039] In the formula, pos represents the coordinate information of one of the x, y or z axes of the point, which is embedded into the feature dimension of c / 3, i represents the dimension of trigonometric function calculation, α and β control the amplitude and wavelength respectively, and then it is spliced ​​along the last dimension, as shown below:

[0040] PE=Concat(PE (pos,2i) ,PE (pos,2i+1) ) (13)

[0041] In order to further improve the learning effect of point cloud position encoding, learnable parameters are introduced to gradually enhance its ability to learn position information as the network deeply learns, as shown below:

[0042] LPE=PE×LP (14)

[0043] F w =(LPE+p i )×LPE (15)

[0044] Where LPE is the position encoding of the learnable parameters, LP represents the learnable parameters, and the parameters of these position encodings are initialized using a random function. As the network depth increases, these parameters will be updated by the optimizer. × represents the product of the elements. Equation (14) describes the specific calculation process of LPE. The LPE result obtained from equation (14) is used as the input of equation (15); p i represents the coordinates of the i-th k-nearest neighbor, F w is the output of the learnable parameter position encoding. Finally, the pooling operation further aggregates F w features to obtain the final output F of the efficient feature extraction module at each stage x , as shown below:

[0045] F x =Max{F w}+Ave{F w} (16)

[0046] Where Max{·} and Ave{·} represent maximum pooling and average pooling, respectively. A combination of maximum pooling and average pooling is used to achieve more accurate point cloud learning.

[0047] Step S3.2: Global feature aggregation in the Transformer branch, which mainly consists of two parts: the position encoding layer and the self-attention layer, to efficiently fuse the position information and global feature information in the Transformer branch;

[0048] Step S3.3: Integration of the learning results of the two branches. Adding the results learned by the two branches can maximize the performance of the model at each stage, as shown in the following formula:

[0049] res=F y +F x (17)

[0050] In the formula, res represents the output of the residual result at each stage, F x is the final output of the efficient feature aggregation branch, and F y It is the final output of the Transformer branch.

[0051] Furthermore, the specific steps of step S3.2 are as follows:

[0052] Step S3.2.1: Position encoding layer. Since 3D point clouds have coordinate position information, a multilayer perceptron consisting of two linear layers and one ReLU nonlinear layer is used to learn the relative coordinates of the point cloud and then use it as the position encoding of the Transformer branch. The position encoding strategy used here is different from that of the efficient feature aggregation branch. By integrating their respective advantages, more comprehensive point cloud position information can be learned. The details are as follows:

[0053] PE=W2δ(W1(pos i -pos j )) (18)

[0054] In the formula, pos i and pos j Represent the three-dimensional position coordinates of the i-th and j-th points respectively, W1 and W2 represent the weight matrices of the two linear layers respectively, δ represents the ReLU nonlinear layer, and PE is the calculated position encoding result;

[0055] Step S3.2.2: Self-attention layer, using vector self-attention to ensure the integrity of the learned point cloud information, as follows:

[0056]

[0057] In the formula, Q, K and V represent the query vector, key vector and value vector respectively, d represents the dimension of the input vector. In order to enhance the global capability of the Transformer branch, a residual connection is made between the output of the self-attention layer and the original point cloud features, as shown in Equation (20):

[0058] F y =Wf y +F (20)

[0059] In the formula, f y represents the output of the self-attention layer, W represents the weight matrix, F represents the original feature, and F y is the final output of the Transformer branch. Through this step, the Transformer branch can aggregate global features more effectively.

[0060] The present invention also discloses a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0061] The present invention also discloses a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the present invention are implemented.

[0062] The present invention also discloses a computer program product, comprising a computer program / instruction, which implements the steps of the method of the present invention when executed by a processor.

[0063] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0064] 1. The original point embedding module of the present invention effectively captures the characteristics of the original points, further enhances the stability and performance of the model, and facilitates subsequent network processing.

[0065] 2. The efficient feature extraction branch with learnable position encoding in the present invention enhances the effect of global feature learning and ensures the learning ability of the model under low parameter conditions.

[0066] 3. The present invention uses the Transformer branch to better capture global information and long-range dependencies, and further avoids the loss of point cloud information.

[0067] 4. The present invention adopts a four-stage learning strategy to make the learning of point cloud features more comprehensive, thereby enhancing the performance of the model and the accuracy of the classification results.

[0068] 5. The model of the present invention achieves an excellent balance between lightweight design and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a schematic diagram of a lightweight multi-stage point cloud classification network integrated with Transformer according to the present invention;

[0070] Figure 2 It is a detailed schematic diagram of each stage in a lightweight multi-stage point cloud classification network fused with Transformer according to the present invention;

[0071] Figure 3 is a detailed schematic diagram of an origin point embedding module according to the present invention;

[0072] Figure 4 is a detailed schematic diagram of learnable position encoding in the efficient feature aggregation branch according to the present invention;

[0073] Figure 5 is a detailed schematic diagram of the Transformer according to the present invention;

[0074] Figure 6 It is a visualization diagram of a data set retrieval example provided by an example of the present invention. DETAILED DESCRIPTION

[0075] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0076] The embodiment of the present invention provides a lightweight multi-stage point cloud classification network integrated with Transformer.

[0077] Combination Figure 1 and Figure 2 As shown, the original points are first projected to a high dimension suitable for point cloud learning through the original point embedding module; then local feature extraction and original feature aggregation are performed respectively through the efficient feature aggregation branch and the Transformer branch, thereby ensuring that the effect of global feature learning is enhanced while maintaining low parameters and complexity of the model; finally, the residual results learned by the two branches are added and aggregated, and input into the final classifier to obtain the final result output.

[0078] This embodiment uses two Lidar point cloud datasets, ModelNet40 and ScanObjectNN, to verify the model. The ModelNet dataset can be obtained through the link on the Princeton University website (http: / / modelnet.cs.princeton.edu) to evaluate the effectiveness of the network in 3D classification tasks. The ModelNet40 dataset contains 12,311 CAD models divided into 40 object categories. Specifically, the training set contains 9,843 models and the test set contains 2,468 models. ScanObjectNN is a real 3D point cloud classification dataset. Due to the characteristics of cluttered background, partial missing and deformation, the dataset is extremely challenging, and there are a large number of outliers in the point cloud. The dataset consists of occluded objects extracted from real indoor scans, including 15,000 real objects from 15 categories, totaling 2,902 3D objects. We train and test the classification task on the most challenging variant of the dataset (PB T50 RS). The specific implementation of this embodiment adopts the same data preparation strategy as PointNet, including data sampling and data enhancement strategies.

[0079] The lightweight multi-stage point cloud classification network of the present invention fused with Transformer has the following specific steps:

[0080] Step S1: Original point embedding module, a new original point embedding method is proposed, which consists of a linear layer and a SE module, which can project the original point cloud to a higher dimension to facilitate the full learning of subsequent point cloud features.

[0081] The specific steps of step S1 are as follows:

[0082] Step S1.1: Project the point cloud information into a higher dimensional channel. In order to minimize the amount of unnecessary parameters, a one-dimensional convolution operation is used as follows:

[0083]

[0084] In the formula, x n is the input channel, h k is the convolution kernel size, y n For the output channel, Represents the sum of convolution kernel indices from k=0 to k=K-1.

[0085] Step S1.2: After the one-dimensional convolution, the distinction between channels is low. To solve this problem, the SE module is used to enhance the network's utilization of different channel information by learning the importance weight of each channel. First, through the compression operation (global average pooling), as shown in Equation 2, the spatial information of each channel is compressed into a single value.

[0086]

[0087] In the formula, H and W are the height and width of the feature map, z is the output of the result, and x ij is the specific value in the feature map. Then, a two-layer fully connected neural network is used to recalibrate the importance of each channel, as shown below:

[0088] s=σ(W2δ(W1z)) (3)

[0089] Where W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU nonlinear layer, and σ represents the sigmoid activation function. Finally, in order to obtain the output feature map The calibration vector s is multiplied element-wise with the original input feature map X on a channel-by-channel basis as follows:

[0090]

[0091] Here, × represents element-by-element multiplication.

[0092] Step S2: Local feature extraction: The embedded high-dimensional point cloud features are respectively extracted from the local information using the FPS and kNN algorithms and the sorting and kNN algorithms through two branches, and the number of point clouds is downsampled.

[0093] The specific steps of step S2 are as follows:

[0094] Step S2.1: Local feature extraction in the efficient feature aggregation branch. After the initial point embedding layer, the FPS method is used to select a subset of local center points to obtain the downsampling process of N point clouds, as shown in Equation 5, and finally N / 2 point clouds are obtained.

[0095]

[0096] In the formula, F i and P i Respectively represent the features and coordinates of the i-th original point, F c and P c They represent the features and coordinates of the cth center point in the subset respectively. Subsequently, the kNN algorithm is used to group the points into k neighboring spaces to form k local 3D regions for local feature extraction, as shown below:

[0097]

[0098] Where N krepresents the kth neighbor space. The coordinates of the N original points and the local center point subset output by FPS are used as the input of kNN. Through these two steps, local features and coordinates are obtained.

[0099] Step S2.2: Local Feature Extraction in Transformer Branch,Compared with the previous part, we directly calculate the distance between two point clouds,in order to extract the location information of point clouds in more,details.

[0100] P n =||xyz i -xyz j || 2 (7)

[0101] Where P n is the distance between point i and point j, xyz i and xyz j Represent the coordinates of the i-th and j-th points respectively. Subsequently, similar to step S2.1, the calculated point cloud distance is used as the input for the subsequent kNN analysis. First, the point cloud distances are sorted and the k nearest neighbors are selected, and then the coordinates of the corresponding point cloud are extracted according to the distance index, as shown below:

[0102]

[0103] knn_xyz k =xyz j |j∈knn_idx k (9)

[0104] In the formula, arg s ort k Indicates that all Sort the distances of the points and select the indexes of the first k points as knn_idx k Then, use knn_idx k As an index to find the corresponding point coordinates, so as to obtain knn_xyz k .

[0105] Step S3: Global feature aggregation. After the local features extracted by the two branches are integrated accordingly, they are passed to the learnable position encoding layer and the self-attention layer respectively to achieve global feature aggregation. Then the results of the two branches are integrated to obtain the residual results of each stage.

[0106] The specific steps of step S3 are as follows:

[0107] Step S3.1: Global feature aggregation in the efficient feature aggregation branch. The function of this step is to aggregate the local features extracted in step S2.1 and obtain the global information of the point cloud. First, the neighbor points and the center point from the FPS and kNN operations are connected to prevent the loss of global information, as follows:

[0108] F kc =Concat(f k ,F k ) (10)

[0109] In the formula, F kc The global and local information are effectively aggregated and used as the input of the LPE layer.

[0110] Subsequently, learnable parameters and kNN point coordinates are introduced into the trigonometric position encoding and weighted. Due to the characteristics of the sine function, the trigonometric function can not only describe the precise position in the embedding space, but also implicitly capture the relative position details between two points in the three-dimensional environment. Therefore, starting from a parameter-free trigonometric position encoding, the learning position encoding design is carried out as follows:

[0111]

[0112]

[0113] In the formula, pos represents the coordinate information of one of the x, y or z axes of the point, which is embedded in the feature dimension of c / 3. i represents the dimension of trigonometric calculation. α and β control the amplitude and wavelength respectively. These two trigonometric formulas can effectively learn the relative position information of the point cloud. Then, it is spliced ​​along the last dimension, as shown below:

[0114] PE=Concat(PEx pos,2i) ,PE (pos,2i+1) ) ((13)

[0115] In order to further improve the learning effect of point cloud position encoding, learnable parameters are introduced to gradually enhance its ability to learn position information as the network deepens, as shown below:

[0116] LPE=PE×LP (14)

[0117] F w =(LPE+p i )×LPE (15)

[0118] Where LPE is the positional encoding of the learnable parameters. LP represents the learnable parameters, which are initialized using a random function and will be updated by the optimizer as the network depth increases. × represents the product of the elements. Equation 14 illustrates the specific calculation process of LPE, and the LPE result obtained from Equation 14 is used as the input of Equation 15. p i represents the coordinates of the i-th k-nearest neighbor. w is the output of the learnable parameter position encoding. Finally, we further aggregate F through the pooling operation w features to obtain the final output F of the efficient feature aggregation module at each stage x , as shown below:

[0119] F x =Max{F w}+Ave{F w} (16)

[0120] Where Max{·} and Ave{·} represent maximum pooling and average pooling, respectively. A combination of maximum pooling and average pooling is used to achieve more accurate point cloud learning.

[0121] Step S3.2: Global feature aggregation in the Transformer branch, which mainly consists of two parts: the position encoding layer and the self-attention layer, to efficiently fuse the position information and global feature information in the Transformer branch.

[0122] Step S3.2.1: Position encoding layer. Since the 3D point cloud has coordinate position information, a multilayer perceptron consisting of two linear layers and one ReLU nonlinear layer is used to learn the relative coordinates of the point cloud and then use it as the position encoding of the Transformer branch, as shown below:

[0123] PE=W2δ(W1(pos i -pos j )) (17)

[0124] In the formula, pos i and pos j Represent the 3D position coordinates of the i-th and j-th points, W1 and W2 represent the weight matrices of the two linear layers, δ represents the ReLU nonlinear layer, and PE is the calculated position encoding result. The position encoding strategy used here is different from the efficient feature aggregation branch, integrating their respective advantages to learn more comprehensive point cloud position information.

[0125] Step S3.2.2: Self-attention layer, using vector self-attention to ensure the integrity of the learned point cloud information, as follows:

[0126]

[0127] Where Q, K, and V represent the query vector, key vector, and value vector, respectively, and d represents the dimension of the input vector. In order to enhance the global capability of the Transformer branch, a residual connection is made between the output of the self-attention layer and the original point cloud features, as shown in Equation 20:

[0128] F y =Wf y +F (19)

[0129] In the formula, f y represents the output of the self-attention layer, W represents the weight matrix, F represents the original feature, and F y is the final output of the Transformer branch. Through this step, the Transformer branch can aggregate global features more effectively.

[0130] Step S3.3: Integration of the learning results of the two branches. Adding the results learned by the two branches can maximize the performance of the model at each stage, as shown in the following formula:

[0131] res=F y +F x (20)

[0132] In the formula, res represents the output of the residual result at each stage, F x is the final output of the efficient feature aggregation branch, and F y It is the final output of the Transformer branch.

[0133] Step S4: Multi-stage learning, combining Figure 1 As shown in the figure, a four-stage learning strategy is adopted, the residual result of each stage is used as the input of the next stage, the downsampling rate of each stage is 1 / 2, and the results after learning in the four stages are passed into the classifier to obtain the final classification result.

[0134] Table 1: Quantitative comparison of classification performance (mAcc, OA, parameters) on the two datasets, distinguishing between non-Transformer and Transformer-based methods, with the best performance highlighted in bold for each metric.

[0135]

[0136] Transformer-based methods

[0137]

[0138] It can be seen that the invention has only 1.44MB of parameters (half the number of parameters of the PCT network), indicating that the network of the invention has the lowest complexity among all Transformer-based methods, thus achieving the lightest architecture. On the ModelNet40 dataset, although the overall accuracy (OA) of the network of the invention is 93.8%, slightly lower than PointTransformerV2, its average accuracy (mAcc) is 91.9%, the highest among all methods (0.3% higher than PointTransformerV2). It is worth noting that on the ScanObjectNN dataset, the overall accuracy (OA) of the network of the invention is 85.0%, and the average accuracy (mAcc) is 86.4%, which is significantly better than other Transformer-based methods, among which mAcc is 5.2% higher than TPNC and OA is 4.8% higher than ACE-Transformer.

[0139] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A lightweight multi-stage point cloud classification method based on a fused Transformer network, characterized in that: Taking the original Lidar point cloud data as input, execute the following steps S1 to S4, obtain the high-dimensional representation of the point features through the original point embedding module, then perform local feature extraction and global feature aggregation in sequence to obtain the residual results of each stage, and finally learn each residual result in multiple stages to obtain the final classification result output. The steps are as follows: Step S1: construct an original point embedding module to project the original point cloud to a higher dimension. The original point embedding module includes a linear layer and a squeeze-excitation SE module; Step S2: local feature extraction, the embedded high-dimensional point cloud features are respectively extracted from the local information by using the farthest point sampling FPS and k nearest neighbor algorithm kNN and sorting and kNN algorithm through two branches, and the number of point clouds is downsampled; Step S3: Global feature aggregation: After the local features extracted by the two branches are integrated accordingly, they are respectively passed to the learnable position encoding layer and the self-attention layer to achieve global feature aggregation. Then, the results of the two branches are integrated to obtain the residual results of each stage. Step S4: Multi-stage learning, adopting a four-stage learning strategy, taking the residual result of each stage as the input of the next stage, the downsampling rate of each stage is 1 / 2, and the results after the four-stage learning are passed into the classifier to obtain the final classification result.

2. According to claim 1, a lightweight multi-stage point cloud classification method based on a fused Transformer network is characterized in that: The specific steps of step S1 are as follows: Step S1.1: Project the point cloud information into a higher dimensional channel to minimize the amount of unnecessary parameters, as shown below: In the formula, x n is the input channel, h k is the convolution kernel size, y n For the output channel, represents the sum of the convolution kernel indices from k = 0 to k = K-1; Step S1.2: After the one-dimensional convolution, the SE module is used to solve the problem of low discrimination between channels. The importance weight of each channel is learned to enhance the network's utilization of different channel information. First, the spatial information of each channel is compressed into a single value through compression operation. In the formula, H and W are the height and width of the feature map, z is the output of the result, and x ij is the specific value in the feature map; then, a two-layer fully connected neural network is used to recalibrate the importance of each channel, as shown below: s=σ(W2δ(W1z)) (3) Where W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU nonlinear layer, and σ represents the sigmoid activation function. Finally, in order to obtain the output feature map The calibration vector s is multiplied element-wise with the original input feature map X on a channel-by-channel basis as follows: Here, × represents element-by-element multiplication.

3. According to claim 1, a lightweight multi-stage point cloud classification method based on a fused Transformer network is characterized in that: The specific steps of step S2 are as follows: Step S2.1: Local feature extraction in the efficient feature aggregation branch. After the initial point embedding layer, the FPS method is used to select a subset of local center points to obtain the downsampling process of N point clouds, as shown in equation (5), and finally N / 2 point clouds are obtained; In the formula, F i and P i Respectively represent the features and coordinates of the i-th original point, F c and P c They represent the features and coordinates of the cth center point in the subset respectively; then, the kNN algorithm is used to group the points into k neighboring spaces to form k local 3D regions for local feature extraction, as shown below: Where N k represents the kth neighbor space; the coordinates of the N original points and the local center point subset output by FPS are used as the input of kNN; through these two steps, local features and coordinates are obtained; Step S2.2: Local feature extraction in the Transformer branch. Compared with the previous part, the distance between two point clouds is directly calculated to extract the location information of the point clouds in more detail. P n =||xyz i -xyz j || 2 (7) Where P n is the distance between point i and point j, xyz i and xyz j Represent the coordinates of the i-th and j-th points respectively; then, similar to step S2.1, the calculated point cloud distance is used as the input of the subsequent kNN analysis; first, the point cloud distances are sorted and the k nearest neighbors are selected, and then the coordinates of the corresponding point cloud are extracted according to the distance index, as shown below: knn_xyz k =xyz j |j∈knn_idx k (9) In the formula, argsort k Indicates that all Sort the distances of the points and select the indexes of the first k points as knn_idx k ; Then, use knn_idx k As an index to find the corresponding point coordinates, so as to obtain knn_xyz k .

4. According to claim 1, a lightweight multi-stage point cloud classification method based on a fused Transformer network is characterized in that: The specific steps of step S3 are as follows: Step S3.1: Global feature aggregation in the efficient feature aggregation branch. The function of this step is to aggregate the local features extracted in step S2.1 and obtain the global information of the point cloud. First, the neighbor points and the center point from the FPS and kNN operations are connected to prevent the loss of global information, as follows: F kc =Concat(f k ,F k ) (10) In the formula, F kc It effectively aggregates global and local information and serves as the input of the LPE layer; Subsequently, learnable parameters and kNN point coordinates are introduced into the trigonometric position encoding and weighted, starting from a parameter-free trigonometric position encoding, and the learning position encoding design is performed as follows: In the formula, pos represents the coordinate information of one of the x, y or z axes of the point, which is embedded into the feature dimension of c / 3, i represents the dimension of trigonometric function calculation, α and β control the amplitude and wavelength respectively, and then it is spliced ​​along the last dimension, as shown below: PE=Concat(PE (pos，2i) ,ON (pos，2i+1) ) (13) The introduction of learnable parameters improves the learning effect of point cloud position encoding and gradually enhances its ability to learn position information as the network deepens, as shown below: LPE=PE×LP (14) F w =(LPE+p i )×LPE (15) Where LPE is the position encoding of the learnable parameters, LP represents the learnable parameters, and the parameters of these position encodings are initialized using a random function. As the network depth increases, these parameters will be updated by the optimizer. × represents the product of the elements. Equation (14) describes the specific calculation process of LPE. The LPE result obtained from equation (14) is used as the input of equation (15); p i represents the coordinates of the i-th k-nearest neighbor, F w is the output of the learnable parameter position encoding. Finally, the pooling operation further aggregates F w features to obtain the final output F of the efficient feature extraction module at each stage x , as shown below: F x =Max{F w }+Ave{F w } (16) In the formula, Max{·} and Ave{·} represent maximum pooling and average pooling respectively. The combination of maximum pooling and average pooling is used to achieve more accurate point cloud learning; Step S3.2: Global feature aggregation in the Transformer branch, which mainly consists of two parts: the position encoding layer and the self-attention layer, to efficiently fuse the position information and global feature information in the Transformer branch; Step S3.3: Integration of the learning results of the two branches. Adding the results learned by the two branches can maximize the performance of the model at each stage, as shown in the following formula: res=F y +F x (17) In the formula, res represents the output of the residual result at each stage, F x is the final output of the efficient feature aggregation branch, and F y It is the final output of the Transformer branch.

5. According to claim 4, a lightweight multi-stage point cloud classification method based on a fused Transformer network is characterized in that: The specific steps of step S3.2 are as follows: Step S3.2.1: Position encoding layer. Since the 3D point cloud has coordinate position information, a multilayer perceptron consisting of two linear layers and one ReLU nonlinear layer is used to learn the relative coordinates of the point cloud and then use it as the position encoding of the Transformer branch, as shown below: PE=W2δ(W1(pos i Possibly j )) (18) In the formula, pos i and pos j Represent the three-dimensional position coordinates of the i-th and j-th points respectively, W1 and W2 represent the weight matrices of the two linear layers respectively, δ represents the ReLU nonlinear layer, and PE is the calculated position encoding result; Step S3.2.2: Self-attention layer, using vector self-attention to ensure the integrity of the learned point cloud information, as follows: In the formula, Q, K and V represent the query vector, key vector and value vector respectively, d represents the dimension of the input vector. In order to enhance the global capability of the Transformer branch, a residual connection is made between the output of the self-attention layer and the original point cloud features, as shown in Equation (20): F y =Wf y +F (20) In the formula, f y represents the output of the self-attention layer, W represents the weight matrix, F represents the original feature, and F y It is the final output of the Transformer branch.

6. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

Images