Point cloud classification method based on state space model

Abstract

The invention discloses a point cloud classification method based on a state space model, relates to the field of deep learning and three-dimensional point cloud target classification, and solves the problem of insufficient extraction of local geometric features and global features of an existing point cloud deep learning model. According to the method, point cloud targets are classified through geometric feature coupling and a cross path feature coordination enhancer. Local feature discrimination capability is improved by utilizing local geometric pooling with a geometric feature coupling mechanism through coordinated propagation and dynamic aggregation of geometric information between a local center point and a neighborhood of the local center point; according to the method, a designed collaborative feature intensifier is used for adopting double-path hybrid interaction, local mutation and sparse key signals are effectively processed, the limitation of traditional SSM global modeling is broken through, and high-precision classification of point cloud targets is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and three-dimensional point cloud target classification, and in particular to a point cloud classification method based on a state space model. Background Art

[0002] Point cloud data overcomes the inherent limitation of depth information loss in traditional two-dimensional imaging by directly capturing the three-dimensional geometric properties of objects, providing indispensable spatial cognition capabilities for intelligent upgrades in various industries. However, existing methods still have significant shortcomings in achieving efficient and accurate classification. The core challenge facing current deep learning models is not just the disordered arrangement characteristics of point cloud data. First, although the voxel-based processing method realizes a regularized data structure, it inevitably causes the loss of geometric details in the process of discretization in three-dimensional space; second, the mainstream point cloud network has limited ability to extract local geometric features, and it is difficult to effectively capture key discriminant features such as changes in the curvature of the point cloud surface. These key defects jointly restrict the large-scale application of point cloud classification technology in actual industrial scenarios, and there is an urgent need to propose solutions with stronger feature learning capabilities and higher computational efficiency. Summary of the Invention

[0003] In order to solve the problems of geometric detail loss in the existing technology of three-dimensional point cloud data processing, and the limited ability of the mainstream point cloud network to extract local geometric features, making it difficult to effectively capture the local and global features of the point cloud, the present invention provides a point cloud classification method based on a state space model.

[0004] A point cloud classification method based on a state space model is implemented by the following steps:

[0005] Step 1: Generate point cloud blocks from the acquired point cloud data, and generate an embedding sequence through the point cloud block embedding layer of the state space model;

[0006] Step 2: Add position code and classification feature sequence to the embedded sequence described in step 1 and input it into the encoder of the state space model; perform feature encoding through the LGP module and CoFE-SiSSM module in the encoder;

[0007] Step 3: Classify through the classifier to obtain the category output after point cloud classification.

[0008] Beneficial effects of the present invention:

[0009] 1. The point cloud classification method described in the present invention adopts a local geometry pooling (LGP) module, which couples the center point features with the neighboring point spatial geometric information through a lightweight feature enhancement strategy based on geometric priors to supplement the local feature space geometric information.

[0010] 2. The point cloud classification method described in the present invention designs a collaborative enhancer module (CoFE) to enhance the bidirectional SSM (BiSSM), namely the CoFE-SiSSM module; this module has channel and multi-scale learning capabilities, and enhances the context perception ability of SSM by mixing multi-dimensional features.

[0011] 3. The point cloud classification method of the present invention was experimentally verified on the Modelnet40 dataset. The method of the present invention can classify 40 types of targets with high accuracy, reaching 95.99%, which is 6.38% higher than pointnet and 3.09% higher than DGCNN. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is an architecture diagram of a point cloud classification method based on a state space model according to the present invention;

[0013] Figure 2 This is the structural principle diagram of the encoder;

[0014] Figure 3 This is the structural principle diagram of LGP;

[0015] Figure 4 This is the structural principle diagram of CoFE;

[0016] Figure 5 Schematic diagram of the structure of CoFE-BiSSM; (a) is a schematic diagram of the forward and reverse SSM; (b) is a schematic diagram of the CoFE-BiSSM module;

[0017] Figure 6 t-SNE graphs of the method of the present invention on Modelnet40 and ScanobjectNN; (a) is the t-SNE graph of ModelNet40, (b) is the t-SNE graph of OBJ_BG, (c) is the t-SNE graph of OBJ_ONLY, and (d) is the t-SNE graph of PB_T50_RS; Figure 7 These are the confusion matrix diagrams of the classification method of the present invention on Modelnet40 and ScanobjectNN; among them, (a) is the confusion matrix diagram of OBJ_BG, (b) is the confusion matrix diagram of OBJ_ONLY, and (c) is the confusion matrix diagram of PB_T50_RS. DETAILED DESCRIPTION

[0018] Specific implementation method 1. Combination Figures 1 to 4 This embodiment describes a point cloud classification method based on a state space model. Figure 1 The state-space model implementation shown.

[0019] In this embodiment, the state space model includes a point cloud block embedding layer, an encoder and a classifier;

[0020] The encoder includes a CoFE-SiSSM module and an LGP module, and a normalization layer and a residual connection layer are provided between the CoFE-SiSSM module and the LGP module;

[0021] In the overall architecture of the model, the input point cloud data first undergoes Farthest Point Sampling (FPS) and K-Nearest Neighbor (KNN) operations to generate point cloud blocks, and each point cloud block is converted into a vector representation through the point cloud block embedding layer.

[0022] Then, the positional encoding and classification feature sequences are added and input into the encoder; finally, the output features are processed by the classifier to obtain the classification results.

[0023] The point cloud classification method described in this embodiment specifically includes the following steps:

[0024] Step S1: Generate point cloud blocks for point cloud data, and generate embedding sequences through the point cloud block embedding layer;

[0025] Step S11, sampling the farthest point of the input point cloud;

[0026] Randomly select the first point s1∈P in the point cloud P and initialize the selected point set P center ={s n}, n represents the nth sampling point, and the target number of sampling points is G. For each unselected point p j , calculate the minimum distance d to the selected point set j :

[0027]

[0028] Select the point farthest from the current sampling point as the next sampling point:

[0029]

[0030] Update the selected point set:

[0031] P center =P center ∪{P next} (3)

[0032] Repeat the above steps for unselected points until the target number of points is reached, and finally G sampling points are obtained.

[0033] Step S12: Taking the farthest sampling point as the center, search for the K points closest to each sampling point to obtain G local domains;

[0034] For any point p other than the query point q Non-q , calculate the square of its Euclidean distance to q:

[0035] distance=||p Non-q -q|| 2 (4)

[0036] Sort the distance set {distance} in ascending order and take the indexes of the first k points KNN , the corresponding KNN neighbor set is:

[0037]

[0038] Where, Represents the local neighborhood constructed by KNN, which participates in subsequent operations as a local point cloud block.

[0039] Step S13: Sequence-encode the local point cloud blocks obtained in step S12;

[0040] A learnable categorical feature sequence is embedded before G local sequences (i.e., G takes centroid points as input sequence) for global representation aggregation. Each point cloud patch is projected and mapped into a C-dimensional feature vector as its initial point cloud patch embedding. The categorical feature sequence token is stacked vertically with all point cloud patch features to form a sequence matrix Z of shape (G+1)×C.

[0041]

[0042] Among them, x cls is a classification feature sequence, Represents G point cloud block features.

[0043] Superimposed position coding preserves the geometric structure information of the original point cloud through point cloud block position coding, providing an initial representation for subsequent layered coding:

[0044]

[0045] Where: Z 0 is the initial input layer feature; are G point cloud patch features processed by the lightweight PointNet variant; Indicates the position code corresponding to the point cloud block feature.

[0046] Step S2: Send the embedded sequence to the encoder and perform feature encoding through the LGP module and CoFE-SiSSM module. Figure 2 As shown, the encoder stacks T layers. For the i-th layer encoder, the input sequence Z i-1Normalization is performed first, and then the LGP module is used to obtain the intermediate output after the residual summation Output the intermediate Then perform normalization again, and finally pass the CoFE-BiSSM module to obtain the final output Z of the current i-th layer encoder i The formula is as follows:

[0047]

[0048] In the formula, Norm represents the normalization operation, P pos Represents positional encoding.

[0049] like Figure 3 As shown, in this embodiment, the processing process of the LGP module is:

[0050] First, normalize the local point cloud features:

[0051]

[0052] Where, is the neighbor point offset feature, ΔF is the relative feature offset, Std(·) represents the standard deviation, and ε is the regularization term to prevent the denominator from being zero.

[0053]

[0054] Where: is the central neighborhood feature, representing local feature information; F C is the center point feature; γ and β are the trainable scale coefficient and shift vector respectively, and ⊙ represents element-by-element multiplication.

[0055] Secondly, normalize the local geometric point cloud blocks:

[0056]

[0057] in, is the normalized coordinate, ΔP is the coordinate of each point converted to the offset relative to the center of mass of the point cloud block, Std(·) represents the standard deviation, and ε is the regularization term to prevent the denominator from being zero.

[0058] Again, construct the Gaussian spatial weights:

[0059]

[0060] Where w is the weight coefficient based on distance decay; W is the expanded weight matrix, representing local geometric information; Broadcast(·) represents the operation of expanding the weight vector to align with the feature dimension.

[0061] Then, the local geometric information is adaptively aggregated with the local feature information:

[0062]

[0063] Where: F weighted is the local feature weight; F agg It is a local feature that integrates local geometric space information; ⊙ represents element-by-element multiplication.

[0064] Finally, the local feature F that integrates the local geometric space information is agg Through a shared perceptron, the output feature F of the LGP module is obtained LGP .

[0065] Step S22: As shown in formula (8), the output feature F LGP The normalized features are encoded through the CoFE-BiSSM module, as shown in Figure 4 As shown in Figure 2, the CoFE module is used to model global features. The specific process is as follows:

[0066] First, set the normalized feature to X and divide the feature X into g groups:

[0067]

[0068] Where Reshape() represents the shape reconstruction operation, B is the batch size, C is the feature dimension, and L is the feature length.

[0069] Secondly, construct the parallel dual-path feature, the formula is:

[0070]

[0071] Where: GN represents group normalization, P avg is the adaptive average pooling. Conv 1x1 and Conv 3x3 They represent one-dimensional convolutions with kernel sizes of 1 and 3, respectively.

[0072] Again, perform cross-channel global feature compression, the formula is:

[0073]

[0074] Where: φ(X1) is the characteristic of the first path; φ(X2) is the characteristic of the second path; F soft It is row-wise Softmax.

[0075] And conduct cross-path bidirectional interaction, the formula is:

[0076]

[0077] Where: X1, X2 represent the first and second paths respectively;

[0078] Then, we proceed to implement the nonlinear scaling of the normalized weights, as follows:

[0079] W CoFE =σ(φ(X1)X2+φ(X2)X1) (18)

[0080] Where W CoFE is the normalized weight, and σ is the Sigmoid function.

[0081] Then combine the weights and adjust the output shape. The formula is:

[0082]

[0083] Where output' represents the output after merging weights, and output represents the output after adjusting the shape of output'

[0084] Finally, if Figure 5 As shown, a two-way SSM model is constructed, and the formula is:

[0085]

[0086] In the formula, Concat represents feature concatenation, MaxPool represents maximum pooling, F' is the feature enhanced by the CoFE module, and Flip C It is a channel flipping operation, FwdSSM represents forward SSM modeling, RevSSM represents reverse SSM modeling, and Linear represents feature mapping.

[0087] Step S3: Finally, the MLP classifier is used to perform classification and obtain the category output.

[0088] Step S31: Map the output channels to 256 through a Linear+BN+ReLu layer, and then pass it through a dropout layer with a value of 0.5;

[0089] Step S32: Pass the output of step S31 through a Linear+BN+ReLu layer to map the output channel to 256, and also pass it through a dropout layer with a value of 0.5;

[0090] Step S33: Use a Linear mapping layer to map the output of step S32 to the number of output categories. For example, if a point cloud object is input, the number of categories to which the object belongs will be output.

[0091] Specific implementation method 2: Figure 5 and Figure 6This embodiment is described as a verification example of a point cloud classification method based on a state space model described in the first embodiment.

[0092] To verify the effectiveness of the point cloud classification method based on the state space model described in the first embodiment, this embodiment is experimentally verified on the ModelNet40, ScanobjectNN and ModelNetFewShot datasets, and ablation experiments are performed on the ModelNet40 and ScanobjectNN datasets. The overall accuracy (OA) is used to evaluate the performance of the classification algorithm.

[0093]

[0094] Experiments on the Modelnet40 dataset and three sub-datasets of ScanobjectNN:

[0095] The experiment used cross entropy as the loss function. NVIDIA Titan RTX graphics cards were used for both training and testing, and mixed precision was disabled. In the state-space model, an encoder layer with T = 12 was set, the feature dimension C was 384, and the LGP center neighborhood expansion parameter was 4. The AdamW optimizer was used, with a cosine scheduler as the decay strategy (CosLR), an initial learning rate of 0.0005, a weight decay of 0.05, 10 warmup epochs, a drop path rate of 0.2, a group size of 32, a num_group of 128, a batch size of 32, and 300 training epochs. Table 1 shows the comparative results of classification accuracy.

[0096] Table 1

[0097]

[0098] As can be seen from Table 1, on the ModelNet40 dataset, the proposed method achieved 94.0%, an improvement of 4.8% and 1.1% over PointNet and DGCNN, respectively, demonstrating its superiority in ideal data environments. In the PB_T50_RS subset of ScanObjectNN (real scans with background interference), the proposed method (91.05%) achieved a significant improvement of 12.95% over DGCNN (78.1%), demonstrating its breakthrough improvement in interference scenarios and effectively solving the problem of feature extraction in noisy and occluded scenes.

[0099] The high-dimensional features are simplified into two-dimensional projection distributions by t-SNE to visualize the clustering effect of the features. The feature clustering results of the method of the present invention are visualized by t-SNE graph, as shown in the figure below. Figure 6 shown.

[0100] from Figure 6 It can be seen that in Figure 6 In (a), the feature distribution is relatively scattered and the boundaries between classes are fuzzy, indicating that the hybrid Emba3D needs to improve its ability to distinguish specific categories. Figure 6 (b) and (d) show a significant clustering trend with little overlap between classes, but (c) achieves excellent class separation. The model maintains robust clustering performance under the interference scenario in (d), confirming its resilience to environmental interference.

[0101] The confusion matrix can be used to determine the misclassification category of a specific category in more detail. The classification confusion matrix result of the method of the present invention is as follows: Figure 7 shown.

[0102] from Figure 7 It can be seen that the comparison Figure 7 In (b) and (a), OBJ_BG with background noise reduces the misclassification rate caused by feature overlap. For example, the accuracy of category 10 and category 12 increases from 86% and 75% to 95% and 88% respectively, and the misclassification rate of category 9 (shelf) and category 10 (table), category 11 (bed) and category 12 (pillow) decreases from 9% and 12% to 0%. Figure 7 In (c), the distribution of misclassifications changes while the overall accuracy remains stable.

[0103] ModelNetFewShot is a small-sample learning version based on the ModelNet40 dataset. The small-sample partitioning uses an N-way K-shot setup, meaning N categories are randomly selected for each task, each category provides K training samples, and the remaining samples are used for testing. The average accuracy and standard deviation of the results are from 10 independent experiments. Table 2 shows the comparative classification accuracy results of the present invention under the small-sample condition.

[0104] Table 2

[0105]

[0106] As can be seen from Table 2, the proposed method significantly outperforms the traditional PointNet and DGCNN methods in the few-sample point cloud classification task, showing strong generalization ability and robustness. Specifically, in the 5-Way classification task, the proposed method achieved an accuracy of 90.5% and 96.0% in the 10-Shot and 20-Shot settings, respectively, which is more than 38% and 58% higher than PointNet (52.0% / 57.8%) and DGCNN (31.6% / 40.8%). In particular, the performance continues to improve when the sample size increases, showing good adaptability to data augmentation. In the more challenging 10-Way task, the proposed method still maintains a high accuracy of 86.3% (10-Shot) and 92.0% (20-Shot), while PointNet and DGCNN show significant performance degradation, especially DGCNN's accuracy drops sharply to 19.9% ​​in 10-Way 10-Shot. It is worth noting that under 10-Way20-Shot, the accuracy of PointNet dropped to 35.2%, while the accuracy of the method of the present invention was still improved to 92%.

[0107] As shown in Table 3, Table 3 shows the ablation experimental results of the LGP and CoFE-BiSSM modules in the method of the present invention.

[0108] Table 3

[0109]

[0110] As can be seen from Table 3, when LGP is activated alone, the accuracy on ModelNet40 reaches 94.00%, demonstrating its powerful geometric feature representation in ideal background-free scenes. In addition, compared with the deactivated baseline, it achieves accuracy gains of +0.97% (OBJ_BG), +1.04% (OBJ_ONLY), and +0.35% (PB_T50_RS), further verifying its ability to resist background and small deformations through local geometric modeling. LGP is a core component that focuses on local feature extraction. This component captures subtle geometric changes and topological structures on the object surface by constructing a nonlinear transformation mechanism within the local neighborhood. CoFE-BiSSM achieves 92.05% accuracy on OBJ_BG, slightly lower than LGP's 92.57%. When LGP and CoFE-BiSSM are used together, a peak of 94.00% (+0.61%) is achieved on ModelNet40, and all subsets of ScanObjectNN comprehensively surpass the single-component configuration, with OBJ-BG improving by 2.03%, OBJ-ONLY improving by 2.41%, and PB_T50_S improving by 0.73%. In summary, LGP and CoFE-BiSSM focus on local geometry and global multi-scale modeling, respectively. While their strengths are scenario-specific, their joint deployment can achieve complementary capabilities and improve generalization performance across different datasets.

[0111] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A point cloud classification method based on a state space model, characterized by: The method is implemented by the following steps: Step 1: Generate point cloud blocks from the acquired point cloud data, and generate an embedding sequence through the point cloud block embedding layer of the state space model; Step 2: Add position code and classification feature sequence to the embedded sequence described in step 1 and input it into the encoder of the state space model; perform feature encoding through the LGP module and CoFE-SiSSM module in the encoder; Step 3: Classify through the classifier to obtain the category output after point cloud classification.

2. The point cloud classification method based on the state space model according to claim 1, characterized in that: The specific process of step one is: Step 1: Sampling the farthest point of the point cloud data to obtain the target number of sampling points, i.e. G sampling points; Step 1 and 2: Use the K nearest neighbor algorithm to search for the K points closest to each sampling point with the farthest sampling point as the center, and obtain G local domains; Step 1-3: Sequence encoding of point cloud blocks in G local domains is performed through a lightweight PointNet network; after adding position encoding and classification feature sequences to the point cloud blocks, they are input into the point cloud block embedding layer of the encoder to generate an embedding sequence.

3. The point cloud classification method based on the state space model according to claim 1, characterized in that: In step 2, the LGP module encodes the input embedded sequence as follows: Step A: normalize the local point cloud features to obtain local feature information; Step B: normalize the local geometric point cloud blocks and construct Gaussian spatial weights to obtain local geometric information; Step C, adaptively aggregating the local feature information described in step A and the local geometric information described in step B to obtain local features fused with local geometric spatial information; Step D: The local features of the fusion of local geometric space information are passed through the shared perceptron to obtain the feature F output by the LGP module. LGP .

4. The point cloud classification method based on the state space model according to claim 3, characterized in that: In step A, the local point cloud features are normalized and expressed as follows: Where, are the center point and the nearest neighbor feature, ΔF is the relative feature offset; Std(·) represents the standard deviation, and ε is the regularization term to prevent the denominator from being zero; For the center point and nearest neighbor features and center point feature F C Perform splicing to obtain the central neighborhood features and use them as local feature information; it can be expressed as follows: Where γ and β are the trainable scale coefficient and shift vector respectively, and ⊙ represents element-wise multiplication.

5. The point cloud classification method based on the state space model according to claim 3, characterized in that: In step B, the local geometric point cloud block is normalized and expressed as follows: Where, To normalize the coordinates, ΔP is the coordinate of each point converted to the offset relative to the center of mass of the point cloud block; Construct Gaussian spatial weights, which can be expressed as follows: W = Broadcast(w) Where w is the weight coefficient based on distance decay; W is the expanded weight matrix as local geometric information; Broadcast(·) represents the operation of expanding the weight vector to align with the feature dimension.

6. The point cloud classification method based on the state space model according to claim 3, characterized in that: In step C, the local geometric information and local feature information are adaptively aggregated, which can be expressed as follows: Where, F weighted is the local feature weight; F agg It is a local feature that integrates local geometric space information.

7. The point cloud classification method based on the state space model according to claim 1, characterized in that: In step 2, the CoFE-BiSSM encoding process is as follows: Step a: Normalize the features output by the LGP module and divide them into g groups to construct parallel dual-path features; Perform cross-channel global feature compression and cross-path bidirectional interactive operations; Step b: performing nonlinear scaling of the normalized weights on the dual paths after the cross-path bidirectional interaction operation in step a, merging the weights, and adjusting the output shape; Step c: construct a bidirectional SSM model for the output shape to obtain global features.

8. The point cloud classification method based on the state space model according to claim 7, characterized in that: In step a, a parallel dual-path feature is constructed, and the formula is: X1=GN(X′⊙σ(Conv 1×1 (P Avg (X′)))) X2=Conv 3×3 (X′) Where GN represents group normalization, P avg For adaptive average pooling, Conv 1x1 and Conv 3x3 They represent one-dimensional convolution with kernel sizes of 1 and 3 respectively, X' is the feature after grouping the normalized feature X, and σ is the Sigmoid function; The formula for cross-channel global feature compression is: φ(X1)=F soft (P avg (X1)) φ(X2)=F soft (P avg (X2)) Where φ(X1) is the characteristic of the first path X1; φ(X2) is the characteristic of the second path X2; F soft Perform Softmax() function operation row by row.

9. The point cloud classification method based on the state space model according to claim 8, characterized in that: In step b, the nonlinear scaling of the normalized weights of the two paths after the cross-path bidirectional interaction operation is performed, and the formula is: W CoFE =σ(φ(X1)X2+φ(X2)X1) Where W CoFE To normalize the weights, merge the weights and adjust the output shape, the formula is: output'=X′⊙W CoFE output=Reshape(output',(B,C,L)) Where output' represents the output feature after merging the weights, output represents the output feature after adjusting the shape of output', B is the batch size, C is the feature dimension, and L is the feature length.

10. The point cloud classification method based on the state space model according to claim 9, characterized in that: In step c, a bidirectional SSM model is constructed, which is expressed as follows: F′=Conv 1×1 (Concat(output,MaxPool(X))) CoFE-BiSSM=Linear(FwdSSM(F′)+RevSSM(Flip C (F′))) In the formula, Concat() represents feature concatenation, MaxPool() represents maximum pooling, F' is the feature enhanced by the CoFE module, and Flip C It is a channel flipping operation, FwdSSM() represents forward SSM modeling, RevSSM() represents reverse SSM modeling, and Linear represents feature mapping.

Images