Sparse point cloud classification method based on mse-mamba

Abstract

The method for classifying sparse point cloud based on MSE-Mamba network relates to the technical field of three-dimensional data processing, and solves the technical problem that the existing sparse point cloud classification method is affected by the sparsity of the point cloud, resulting in insufficient capture of local features, low efficiency of global correlation modeling, and difficulty in coordinating local feature extraction and global modeling, thereby causing the classification accuracy to decrease. The method uses a MSE-Mamba multi-scale local feature coding module to complete the extraction and enhancement of the local geometric features of the sparse point cloud, then captures the long-range semantic correlation between the core points through a Transformer module based on a global attention mechanism, realizes the fusion of the local geometric features and the global semantic information, and finally completes the class probability calculation through a multi-feature aggregation strategy to realize the high-precision and high-efficiency classification of the sparse point cloud. The method realizes the coordinated improvement in classification accuracy, calculation efficiency and anti-sparsity robustness.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and 3D point cloud classification technology, specifically to a sparse point cloud classification method based on MSE-Mamba. Background Technology

[0002] LiDAR, as a core sensor for environmental perception and target detection, achieves precise ranging by emitting high-energy pulsed light waves and calculating the round-trip time difference, making it indispensable in intelligent perception systems. Compared to traditional radar and optical imaging sensors, it boasts superior 3D information acquisition capabilities and accurate and efficient target classification. It can simultaneously collect target distance, angle, and reflection intensity to construct a high-precision 3D point cloud model, and complete target and sub-category classification based on geometric features. It is widely used in key tasks such as precise target detection, multi-target classification and recognition, and real-time target tracking.

[0003] However, the point clouds acquired by detectors are sparse due to various factors in complex environments. This sparsity leads to the loss of key discriminative features such as target geometry and local details, significantly reducing the overall accuracy of point cloud target classification tasks and becoming a core bottleneck for improving target perception and decision-making efficiency in complex environments. Therefore, achieving high-precision point cloud target classification under sparse point cloud conditions has crucial scientific significance and engineering application value for improving target perception and decision-making efficiency in complex environments.

[0004] Currently, research in the field of point cloud classification has made some progress. Researchers have proposed a variety of optimization methods around the core problem of sparse point cloud classification, forming different technical paths. Existing sparse point cloud classification methods are mainly divided into two major technical directions, which rely on efficient sequence modeling and global dependency modeling capabilities for optimization, respectively.

[0005] In terms of efficient sequence modeling, the Mamba model, as an efficient long sequence modeling framework based on the Selective State Space Model (SSM), employs a sequence processing mechanism with linear complexity (O(Nd)). Through gated state updates and local convolution enhancement strategies, it efficiently captures long-range dependencies while significantly reducing computational costs. It is highly compatible with the disordered nature and long sequence characteristics of sparse point clouds, making it a hot research direction in sparse point cloud classification in recent years. However, existing Mamba-based point cloud processing methods still have significant bottlenecks: on the one hand, the single one-dimensional sequence modeling paradigm relies on point cloud serialization strategies, making it difficult to fully preserve the local fine geometric structure in three-dimensional space, resulting in the model's inability to effectively mine key discriminative features of the point cloud; on the other hand, when dealing with multi-scale sparse structures, existing methods lack effective cross-scale feature aggregation mechanisms, leading to insufficient consistency and discriminative power in feature representation. Ultimately, the classification accuracy and robustness of the model in sparse point cloud scenarios need further optimization.

[0006] In global dependency modeling, Transformer, with its multi-head self-attention mechanism, breaks through the inherent limitations of traditional local modeling paradigms such as convolution and multilayer perceptron (MLP), effectively capturing global feature associations in point clouds and becoming a core technical support for global feature learning in point clouds. However, the computational complexity of Transformer's self-attention is quadratic with the number of point clouds (O(N²d), where d is the feature dimension). When facing large-scale sparse point clouds, the excessive computational and memory overhead makes it difficult to meet the real-time processing requirements of scenarios such as autonomous driving and real-time 3D reconstruction, severely limiting its engineering application. Although subsequent research has reduced the complexity to sub-quadratic levels through optimization strategies such as sparse attention and hierarchical attention, these methods either sacrifice some global correlations or rely on specific sampling strategies, resulting in limited generalization. They have not fundamentally solved the core contradiction between the integrity of global modeling and computational efficiency, and there is still considerable room for improvement in feature representation capabilities and classification accuracy in sparse point cloud scenarios.

[0007] Therefore, this invention proposes a sparse point cloud classification method based on MSE-Mamba to solve the problems existing in the prior art, overcome the limitations of the prior art, and achieve high-precision and efficient classification of sparse point clouds. Summary of the Invention

[0008] This invention addresses the problems of insufficient local feature capture, low efficiency of global correlation modeling, and difficulty in coordinating local feature extraction and global modeling in existing sparse point cloud classification methods due to the sparsity of point clouds, which leads to a decrease in classification accuracy. It provides a sparse point cloud classification method based on the MSE-Mamba neural network.

[0009] A sparse point cloud classification method based on the MSE-Mamba neural network is implemented by the following steps:

[0010] Step 1: Perform multi-scale adaptive grouping on the input sparse point cloud to obtain a multi-scale local point group set;

[0011] Step 2: Input the obtained multi-scale local point set into the MSE-Mamba multi-scale local feature encoding module to complete the extraction and enhancement of multi-scale local geometric features and obtain local feature representation;

[0012] Step 3: Input the output local feature representation into the Transformer module based on the global attention mechanism to perform global feature encoding, capture the long-range semantic association between core points, and obtain a high-level feature representation that integrates local geometric features and global semantic information.

[0013] Step 4: Input the high-level feature representation obtained in Step 3 into the category output module, calculate the category probability distribution through a multi-feature aggregation strategy, complete the classification processing of sparse point clouds, and obtain the point cloud classification output result.

[0014] The beneficial effects of this invention are:

[0015] 1. The multi-scale point cloud adaptive grouping strategy proposed in this invention achieves uniform spatial coverage of core points through farthest point sampling (FSP), and combines multi-scale K-Nearest Neighbor (KNN) neighborhood construction and coordinate centering processing to effectively capture sparse point cloud structure information at different scales, eliminate global position change interference, and lay a stable and rich foundation for subsequent feature extraction. This overcomes the limitation that single-scale grouping is difficult to adapt to the multi-scale structure of sparse point clouds.

[0016] 2. The MSE-Mamba multi-scale local feature encoding module designed in this invention integrates the advantages of Mamba's linear sequence processing with multi-scale feature extraction and neighborhood adaptive enhancement strategies. Through the synergistic effect of KNorm and KPool, it effectively solves the problem of feature distribution fluctuation caused by uneven neighborhood density in sparse point clouds. Bidirectional state space modeling further enhances the ability to capture local contextual information and significantly enhances the discriminative power and robustness of local geometric features.

[0017] 3. The Transformer module based on a global attention mechanism designed in this invention retains the advantages of global modeling of the Transformer, avoids the loss of feature information through DropPath and residual connections, injects spatial coordinate information into the feature representation through learnable position embedding, and efficiently models long-range semantic associations between core points by combining a multi-head self-attention mechanism, thereby improving the feature discrimination ability of the overall network model.

[0018] 4. This invention organically integrates the efficient sequence processing advantages of MSE-Mamba with the global attention modeling capabilities of Transformer. While achieving high-precision classification of sparse point clouds, it maintains low computational complexity and parameter count. Experimental verification on ModelNet40, ScanObjectNN, and a self-built 3D imaging LiDAR point cloud dataset shows that the proposed method exhibits excellent point cloud classification performance in various scenarios, with overall accuracy improved compared to current mainstream methods. It provides a new paradigm for high-precision classification of sparse point clouds and is of great significance for promoting the practical application of LiDAR technology in complex scenarios. Attached Figure Description

[0019] Figure 1 This is a diagram showing the overall structure of the sparse point cloud classification method based on MSE-Mamba.

[0020] Figure 2 A schematic diagram of the sampling results of the farthest point in a sparse point cloud;

[0021] Figure 3 A schematic diagram of the K-nearest neighbor query results with the core point as the reference point;

[0022] Figure 4 This is a schematic diagram of multi-scale region division;

[0023] Figure 5 Here is a structural diagram of the MSE-Mamba multi-scale local feature encoding module;

[0024] Figure 6 Here is a diagram of the Transformer module structure based on the global attention mechanism;

[0025] Figure 7 The curves show the point cloud classification results of various methods under downsampling conditions. Detailed Implementation

[0026] Specific Implementation Method 1: Combination Figures 1 to 6 This embodiment describes a sparse point cloud classification method based on MSE-Mamba. This method combines the efficient sequence processing advantages of the MSE-Mamba module with the global modeling capabilities of the global attention Transformer module to achieve high-precision classification of sparse point clouds.

[0027] The specific implementation process of the classification method described in this embodiment is as follows:

[0028] Step 1. Perform multi-scale adaptive grouping on the input sparse point cloud to obtain a multi-scale local point group set;

[0029] Step 11. Figure 2 As shown, the input sparse point cloud is set as... , can be represented as Where B is the batch size (the number of samples in one training or inference session). For the first The three-dimensional coordinates of each point are used. G core points that fully cover the point cloud space are sampled using FPS, forming a core point set. .

[0030] Step 12. Figure 3 As shown, with each core point Centered on a point, a local neighborhood is formed by searching for the K nearest points using KNN. and the nearest points of the core point The neighborhood coordinates are centered relative to the core point to eliminate interference from changes in the target's global position, as expressed by the formula:

[0031]

[0032] In the formula, For the core set, Let G be a K-dimensional vector of all ones, ⊗ be the tensor outer product, the subscript 0 represent the initial neighborhood of each core point, G be the number of core points, and N be the total number of points in the input point cloud. Let g be the initial set of neighborhood points of the g-th core point. This is the initial set of neighborhood points after centralization.

[0033] Step 13. Figure 4 As shown, three are generated using KNN. Sub-scale neighborhoods are extracted in parallel and then fused to achieve multi-scale sparse structure capture, as expressed by the formula:

[0034]

[0035] In the formula, These represent the number of nearest neighbors in the three sub-scale neighborhoods, respectively. , , These are the sets of neighborhood points centered at each of the three sub-scales. Indicates the core point Centered on, search Find the nearest neighbor and complete the centralized processing.

[0036] Step 2. Input the multi-scale local point set obtained in Step 1 into the MSE-Mamba multi-scale local feature encoding module to complete the extraction and enhancement of multi-scale local geometric features and obtain the final output local feature representation.

[0037] Step 21. Figure 5 As shown, the dimensions of all points in the neighborhood of each scale are transposed, converting the 3D coordinates into convolutional channel dimensions, which lays the groundwork for convolutional feature extraction. The transposed result is represented as follows:

[0038]

[0039] In the formula, The set of points centered in the i-th scale neighborhood of the g-th core point. This represents the matrix transpose operation, and `permute` represents the dimension rearrangement operation. This is the characteristic matrix after transpose.

[0040] Step 22. Use two layers of one-dimensional convolution and the ReLU activation function to convert the coordinate features. Mapping to high-dimensional features: the first layer maps to 128 dimensions, the second layer maps to 256 dimensions, obtaining global structural features of the domain, achieving high-dimensional transformation of low-dimensional coordinate features, expressed by the formula:

[0041]

[0042]

[0043] In the formula, This represents a one-dimensional convolution operation. This indicates a batch normalization operation. express Activation function The 128-dimensional high-dimensional features are the output of the first convolutional layer. This is the 256-dimensional high-dimensional feature output by the second convolutional layer;

[0044] Step 23. Extract the global structural features of the neighborhood through global max pooling, expand the global structural features to the same dimension as the number of local feature points, and concatenate them with the local features to achieve local-global semantic fusion and obtain the fused features. This can be expressed as a formula:

[0045]

[0046]

[0047] In the formula, For max pooling operation, The feature vector after global max pooling. Indicated in the 4th dimension, This indicates that the global feature will be repeated in this dimension. Second-rate, For globally extended features, This indicates a feature concatenation operation. Features of fusion;

[0048] Step 24. After concatenating the fused features of each neighborhood and performing one-dimensional convolution, perform neighborhood adaptive normalization KNorm and exponential weighted aggregation KPool to obtain the aggregated feature vector; the formula is:

[0049]

[0050]

[0051]

[0052] In the formula, For multi-scale fusion features, , For learnable parameters, These are the features after neighborhood normalization. This is the aggregated feature vector.

[0053] Step 25. Enhance the local context capture capability through bidirectional state-space modeling (bi-SSM), perform layer normalization on the bidirectional fused features and enhance the nonlinear expression through SiLU activation, and output the final local features, as shown in the formula:

[0054]

[0055]

[0056] In the formula, This indicates a forward state-space modeling operation. This indicates the reverse state-space modeling operation. for The flipping feature, This represents the fusion result of forward and backward modeling features with the original features. Representation layer normalization, This represents the local feature representation of the final output.

[0057] Step 3. Represent the local features output in Step 2. The data is fed into a Transformer (GA-Transformer) module based on a global attention mechanism for global feature encoding, capturing long-range semantic relationships between core points to obtain a high-level feature representation that integrates local geometric features and global semantic information. The specific implementation process is as follows:

[0058] Step 31. Figure 6 As shown, local feature representation The DropPath random dropout strategy is introduced to prevent overfitting, and the original features are fully preserved through residual connections, as expressed by the formula:

[0059]

[0060] In the formula, These are the initial global features.

[0061] Step 32. Through multi-head self-attention calculation, Convert to Q, K, V vectors, and simultaneously calculate global associations through dot product attention, introducing a scaling factor. The balance scale, avoiding excessive attention values, can be expressed by the formula:

[0062]

[0063]

[0064] In the formula, W Q W K W V Here is the attention weight matrix, and H is the number of attention heads. The global association features are calculated after multi-head self-attention.

[0065] Step 33. After MLP enhancement, compared with the initial global features Residual join can be expressed by the following formula:

[0066]

[0067] In the formula, This is a global enhancement feature.

[0068] Step 34. Set the core points The 3D coordinates are expanded to the same dimension D as the global features using MLP to obtain the spatial location features of the core points. And inject learnable position embeddings By merging the two, a complete set of location embeddings can be obtained. Finally, this is added directly to the global augmented feature to output an augmented feature that simultaneously contains semantic and spatial location information. This can be expressed as:

[0069]

[0070]

[0071]

[0072] In the formula, Features that integrate spatial location information and global semantic information.

[0073] Step 4. The high-level feature representation obtained in Step 3 is fed into the category output module. The category probability distribution is calculated through a multi-feature aggregation strategy to complete the classification processing of the sparse point cloud and obtain the point cloud classification output result. The specific process is as follows:

[0074] Step 41. Extract the feature vectors corresponding to the category labels. ,Will Feature vectors obtained by max pooling ,Will Feature vectors obtained by average pooling Three parts.

[0075] Step 42. Map to the target class space through a linear layer, and utilize... The function calculates the probability of each category, expressed by the formula:

[0076]

[0077] In the formula, It is a multilayer perceptron used for class prediction, where the final class probability distribution is obtained by calculating the probabilities using Softmax. C represents the number of categories.

[0078] The sparse point cloud classification method based on MSE-Mamba described in this embodiment is implemented through multi-scale adaptive grouping, an MSE-Mamba multi-scale local feature encoding module, a Transformer module based on a global attention mechanism, and a category output module. Based on this method, systematic training and validation were performed on the ModelNet40 dataset, the ScanObjectNN dataset, and a self-built 3D imaging LiDAR point cloud dataset using the PyTorch deep learning framework to comprehensively verify the classification effectiveness, generalization ability, and robust classification ability for sparse point clouds of the proposed method.

[0079] Specific Implementation Method Two: Combination Figure 7 This embodiment describes a verification example of the sparse point cloud classification method based on MSE-Mamb described in Specific Embodiment 1:

[0080] This invention trains the model on an NVIDIA RTX 4070Ti Super graphics card using the PyTorch framework. The model embedding dimension is 384, and the network depth is... 4 (corresponding to) Figure 5 The subscript k of L in the middle represents the number of heads of multi-head attention. It is 6 (corresponding to) Figure 6 The subscript T of L in the dataset (the category output dimension changes depending on the dataset) is used. During training, the AdamW optimizer is used, with an initial learning rate of 5 × 10⁻⁻⁻⁴. 4 The weight decay is 0.05. The learning rate scheduling uses a cosine annealing with warm restarts strategy, with an initial cycle length of 200 rounds, a cycle multiplication factor of 1.8, and a minimum learning rate of 1×10⁻⁻⁶. 6 The number of warm-up rounds is 20. The batch size is 32, and the gradient clipping threshold is 10. The loss function uses cross-entropy loss with 0.2 label smoothing, and the DropPath rate is 0.2 to enhance regularization. The dimension scaling factor of the MLP layer is 4.0, the Dropout rate in the class output module is set to 0.5, and the feature concatenation dimension is 768 (384×2).

[0081] To measure the model's classification performance, this paper uses Overall Accuracy (OA) as the evaluation metric. Overall accuracy is the most commonly used evaluation metric in point cloud classification tasks, defined as the proportion of correctly classified samples out of the total number of samples.

[0082]

[0083] In the formula, TP (True Positive) represents the number of samples that are actually positive but are correctly predicted as positive, TN (True Negative) represents the number of samples that are actually negative but are correctly predicted as negative, FP (False Positive) represents the number of samples that are actually negative but are incorrectly predicted as positive, and FN (False Negative) represents the number of samples that are actually positive but are incorrectly predicted as negative.

[0084] In order to thoroughly analyze the contribution of each component in the proposed method and the rationality of the experimental parameter settings, this implementation method conducted a comprehensive ablation experiment, mainly divided into module ablation experiments and parameter setting ablation experiments, to quantitatively verify and analyze the effectiveness of the components and the rationality of the parameters.

[0085] (1) Parameter settings for ablation experiment

[0086] To verify the scientific validity and rationality of the key parameter settings of the MSE-Mamba network model, this experiment designed multiple ablation experiments for the core parameter scale levels (D, D / 2, D / 4, D / 8) and the number of anchor points (K=8, 16, 32). The effects of different parameter combinations on the model classification performance were systematically analyzed. The experimental results are shown in Table 1, which presents the overall accuracy results of the parameter setting experiment.

[0087] Table 1

[0088]

[0089] Experimental results with varying scale configurations demonstrate that multi-scale feature fusion is a key factor in improving model classification performance. When using only a single scale (Only D), the overall accuracy (OA) is only 92.4%–93.1%, indicating relatively limited performance. As the number of fused scales increases, classification accuracy gradually improves: with two scales (D, D / 2), OA increases to 92.5%–93.5%; with three scales (D, D / 2, D / 4), OA further improves to 92.8%–94.0%; and with four scales (D, D / 2, D / 4, D / 8), the model achieves optimal performance, with OA reaching a maximum of 94.2%. This result fully demonstrates that multi-scale feature aggregation can comprehensively capture fine-grained edge details at small scales and global shape contour information at large scales in point clouds, effectively solving the limitation of single-scale methods in adapting to targets of different sizes and sparse distributions, and achieving richer and more discriminative feature representations.

[0090] Experimental results on the number of anchor points (K) show that K=16 is the optimal choice balancing model performance and computational efficiency. For all scale configurations, the model's operational accuracy (OA) significantly improves as K increases from 8 to 16: taking four-scale fusion as an example, the OA is 93.2% with K=8, increasing to 94.2% with K=16, a 1 percentage point improvement. However, when K is further increased to 32, the performance improvement plateaus, and the OA remains at 94.2%. This phenomenon indicates that when the number of anchor points is insufficient (K=8), the core points do not adequately cover the entire point cloud space, resulting in incomplete extraction of local neighborhood features and an inability to fully capture the key structural information of the point cloud. While an excessive number of anchor points (K=32) can increase spatial coverage, it introduces redundant computation, leading to increased model complexity without providing additional improvement in feature discrimination. Therefore, in the specific experiment, this paper sets K=16, which not only ensures that the core points can fully cover the key areas of the point cloud, providing stable support for the local feature extraction of the MSE-Mamba module and the long-range correlation feature capture of the GA-Transformer module, but also avoids the decrease in computational efficiency caused by excessive anchor points, thus achieving the best balance between performance and efficiency.

[0091] (2) Modular ablation experiment

[0092] Ablation analysis was performed on the MSE-Mamba and GA-Transformer modules in this implementation using a 3D imaging dataset to verify their effectiveness. The performance difference with and without these modules was compared to evaluate their role in improving classification accuracy. Table 2 shows the overall accuracy results for each module's effectiveness verification.

[0093] Table 2

[0094]

[0095] In this embodiment, the results of the effectiveness verification experiments for each module are shown in Table 2. The ablation experiments show that both the proposed MSE-Mamba feature encoding module and GA-Transformer significantly contribute to the network's point cloud classification performance. The complete model (No. 1) achieves a classification accuracy of 96.82% on the 3D imaging dataset. The MSE-Mamba module (comparing No. 1 and No. 3) brings a performance improvement of 0.87%, and the GA-Transformer module (comparing No. 1 and No. 2) contributes a 0.57% accuracy improvement.

[0096] In this embodiment, after completing the ablation experiment, under the experimental conditions of 16 anchor points K and 8 scale D, a classification experiment was conducted on the synthetic dataset ModelNet40 to verify the performance of MSE-Mamba for point cloud classification.

[0097] (1) Classification experiment on ModelNet40 dataset

[0098] The ModelNet40 dataset spans 40 object categories, making it an ideal test dataset for validating the basic performance of point cloud classification algorithms. The visualization is shown in the figure below. In the specific experiments, the number of training samples was 9843 frames, and the number of test samples was 2468 frames. N=1024 points were used as input, and data augmentation was applied with a random scaling and translation of 0.95. Table 3 presents the classification results for the ModelNet40 dataset.

[0099] Table 3

[0100]

[0101] Table 3 shows the superiority of MSE-Mamba over current mainstream point cloud classification methods on the ModelNet40 dataset. In terms of classification accuracy, MSE-Mamba achieves an overall accuracy of 94.2%, demonstrating a significant advantage among existing methods. Compared to traditional methods that rely on local feature aggregation, such as PointNet++ and DGCNN, MSE-Mamba, with its multi-scale local feature extraction and global modeling capabilities, can more effectively capture the long-distance dependencies of point clouds, thus improving overall accuracy.

[0102] Compared to Transformer-based methods (such as PCT), MSE-Mamba not only surpasses them in classification performance but also avoids the secondary computational complexity inherent in self-attention mechanisms. This allows MSE-Mamba to maintain high accuracy while achieving higher computational efficiency, making it suitable for training and testing tasks in large-scale point cloud scenarios.

[0103] Based on the experimental results, in terms of computational complexity and number of parameters, MSE-Mamba improved by 1.17% compared to PCM, while the number of parameters decreased from 34.2M to 16.9M. This significant efficiency improvement is mainly due to the linear computational complexity of Mamba.

[0104] To verify the comprehensive performance superiority of the proposed MSE-Mamba method from multiple dimensions and scenarios, experiments were conducted using three variant subsets of the ScanObjectNN dataset: OBJ_BG, OBJ_ONLY, and PB_T50_RS.

[0105] (2) Classification experiment on ScanObjectNN dataset

[0106] The OBJ_ONLY subset contains pure object point clouds without background, validating MSE-Mamba's feature extraction and representation capabilities for clean point clouds. The OBJ_BG subset includes point cloud data with cluttered backgrounds, testing the method's feature selection and target information mining capabilities under background interference. The PB_T50_RS subset features complex rotation and scaling (random rotation, 75% scaling). These three variant subsets, with their distinct characteristics, progressively validate the robustness of the MSE-Mamba method in feature extraction compared to existing methods, ensuring the comprehensiveness and persuasiveness of the experimental verification. Table 4 presents the classification results for the ScanObjectNN dataset.

[0107] During the experiment, the training sample size for OBJ_BG and OBJ_ONLY was set to 2309 frames, and the test sample size was set to 581 frames; the training sample size for PB_T50_RS was set to 11416 frames, and the test sample size was set to 2882 frames. During training, the only difference was the input point cloud size N=2048, which differed from ModelNet40; all other strategies remained the same. Table 4 shows the classification results for the ScanObjectNN dataset.

[0108] Table 4

[0109]

[0110] As shown in Table 4, the proposed MSE-Mamba network model exhibits superior classification performance and strong competitive advantages, achieving the best performance on all three subsets. This excellent performance is mainly attributed to the following aspects: First, compared with traditional MLP backbone network methods, MSE-Mamba's long sequence modeling capability based on the Mamba architecture enables it to better capture global dependencies in point clouds, especially demonstrating stronger robustness when dealing with complex backgrounds and noise interference. Second, compared with other Transformer-based methods, MSE-Mamba more effectively handles the local geometry of point clouds through a multi-scale point cloud local feature extraction module, which is particularly important when facing irregular sampling and local missing data in real scanned data. Finally, even compared with PCM and Mamba3D, which are also based on the Mamba architecture, MSE-Mamba exhibits superior performance, which fully verifies the effectiveness of the proposed multi-scale local feature extraction and self-attention global feature integration strategy.

[0111] To fully verify the classification performance and robustness of the proposed MSE-Mamba network model in real-world scenarios and under sparse conditions, this experiment conducted system tests based on a self-made 3D imaging LiDAR point cloud dataset (containing four real-world target classes: Bicycle, Car, Person, and Tree), and focused on exploring the impact of different downsampling point numbers (N=256, 64, 32) on the classification results.

[0112] (3) Real-world scenario dataset classification experiment

[0113] The real-world scene dataset contains four categories. These point clouds are real-world object point clouds obtained by scanning in different real-world scenes. Scene settings are detailed below. Figure 4During the experiment, data augmentation was performed on the collected data, including random translation, rotation, and scaling perturbations. The training sample size was 1200 frames, and the test sample size was 320 frames. The input point cloud size was set to N=256 to compare and analyze the actual classification performance of various methods. Tables 5 and 6 present the classification results of the real-world dataset.

[0114] Table 5

[0115]

[0116] The classification results of the real-world dataset are shown in Table 5. MSE-Mamba achieved an overall accuracy of 96.82%, significantly outperforming various mainstream comparison methods, fully demonstrating its strong adaptability to diverse targets in real-world scenarios. Specifically, it improved by 2.0 percentage points compared to the traditional MLP-based backbone network PointNet++ (94.83%), surpassed the Transformer-based PCT (95.97%) by 0.85 percentage points, and also outperformed Mamba3D (96.55%), which belongs to the Mamba family, by 0.27 percentage points. The core reason for this excellent performance lies in MSE-Mamba's multi-scale point cloud adaptive grouping strategy, which accurately captures the local geometric details and overall structural features of sparse point clouds. In addition, the synergistic effect of KNorm neighborhood adaptive normalization and KPool exponential weighted aggregation mechanism, along with the GA-Transformer module's efficient modeling of long-range semantic relationships between core points through learnable position embedding and multi-head self-attention mechanisms, breaks through the limitations of traditional local modeling paradigms.

[0117] In addition, point cloud classification capability tests were set up with 64 and 32 point cloud downsampling points. The experimental results are shown in Table 6, which shows the point cloud classification results under various downsampling conditions.

[0118] Table 6

[0119]

[0120] Under downsampling conditions, MSE-Mamba's performance advantage is even more pronounced, fully demonstrating its strong robustness to sparse point clouds. When the number of input points is reduced to N=64, the model's OA still reaches 92.7%, and even in sparse scenarios with N=32, the model maintains a high accuracy of 92.1%. This result fully verifies the effectiveness of the MSE-Mamba core module, demonstrating that the model can still extract stable and effective discriminative features and complete high-precision point cloud classification tasks even when a large number of point clouds are lost.

[0121] In addition, this invention conducted experiments on the ModelNet40 dataset to test the classification capabilities of various methods under different sparsity levels (point number N ranging from 1024 to 32), aiming to explore in detail the robustness of different methods' classification capabilities as the point cloud quantity gradually decreases. The experimental results are as follows: Figure 7 As shown.

[0122] Based on the ModelNet40 dataset, sparsity verification experiments were conducted by progressively downsampling the number of point clouds from 1024 to 32. The system compared the classification robustness of various mainstream methods under scenarios where the sparsity of point clouds continuously increases. The experimental results clearly show the performance degradation differences of different methods as the number of point clouds decreases, and further validate the significant advantages of the MSE-Mamba model in sparse point cloud classification tasks. The overall classification accuracy of all participating methods decreased to varying degrees as the number of input point clouds decreased. Among them, the PCT method based on the Transformer architecture, although it achieved the capture of global feature associations in the point cloud through a multi-head self-attention mechanism and had superior classification performance under moderate sparsity, saw a rapid decline in the effectiveness of global feature modeling as the number of point clouds continuously decreased and feature information significantly reduced. The degradation trend of classification accuracy gradually accelerated, making it difficult to maintain stable performance in extremely sparse scenarios. PCM and Mamba3D, both belonging to the Mamba architecture, show better stability than the former in sparse point cloud processing, and the accuracy decay is relatively gentle. However, due to the lack of collaborative optimization design of multi-scale feature aggregation and global semantic association, they cannot fully mine the effective discriminative features in the remaining point cloud under the condition of sparse point cloud with the number of point clouds reduced to 32, resulting in a significant decline in classification performance.

[0123] In stark contrast, the MSE-Mamba model consistently maintained the lowest accuracy decay rate among all compared methods throughout the point cloud downsampling test. Even under sparse conditions with only 32 points in the point cloud, it maintained a classification accuracy far exceeding other methods, demonstrating extremely strong robustness in sparse point cloud classification. The core of this lies in MSE-Mamba's fusion of multi-scale local feature extraction and global semantic association modeling. Its multi-scale point cloud adaptive grouping strategy combines KNorm neighborhood adaptive normalization and KPool exponential weighted aggregation mechanisms, enabling it to accurately capture fine-grained edge details and key structural features in the remaining point cloud even when there is significant feature loss and insufficient feature information. This effectively compensates for the feature loss problem in sparse point clouds, preserving core discriminative information for the classification task. Simultaneously, the GA-Transformer global feature encoding module incorporates spatial coordinate information into the feature representation through learnable position embedding, and combines a multi-head self-attention mechanism to efficiently model long-range semantic associations between core points. This breaks through the limitations of a single local modeling paradigm, allowing the model to mine more effective discriminative information through global associations, improving classification stability.

[0124] In summary, the MSE-Mamba network model demonstrates excellent classification stability, generalization ability, and robustness in real-world complex scenarios and under sparse conditions with varying degrees of downsampling, effectively solving the problem of decreased classification accuracy caused by feature loss in sparse point cloud classification.

[0125] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A sparse point cloud classification method based on MSE-Mamba, characterized by: This method is implemented by the following steps: Step 1: Perform multi-scale adaptive grouping on the input sparse point cloud to obtain a multi-scale local point group set; Step 2: Input the obtained multi-scale local point set into the MSE-Mamba multi-scale local feature encoding module to complete the extraction and enhancement of multi-scale local geometric features and obtain local feature representation; Step 3: Input the output local feature representation into the Transformer module based on the global attention mechanism to perform global feature encoding, capture the long-range semantic association between core points, and obtain a high-level feature representation that integrates local geometric features and global semantic information. Step 4: Input the high-level feature representation obtained in Step 3 into the category output module, calculate the category probability distribution through a multi-feature aggregation strategy, complete the classification processing of sparse point clouds, and obtain the point cloud classification output result.

2. The sparse point cloud classification method based on MSE-Mamba according to claim 1, characterized in that: In step one, the specific steps for obtaining the multi-scale local point set are as follows: Step 11: Using FPS to extract data from the original sparse point cloud. Select G core points to form a core point set. ; Steps one and two: Focusing on each core point Centered on the core point, a local neighborhood is formed by searching for the K nearest neighbors using KNN, and the nearest neighbors of the core point are then... The neighborhood coordinates are centered relative to the core point: ； In the formula, For the core set, Let be a K-dimensional vector of all 1s, ⊗ be the tensor outer product, subscript 0 represent the initial neighborhood of each core point, B be the batch size, G be the number of core points, and N be the total number of points in the input point cloud. For the g-th core point The initial set of neighborhood points, This is the initial set of neighborhood points after centralization; Step 13: Generate three sub-scale neighborhoods using KNN, extract features from each scale in parallel, and then fuse them to achieve multi-scale sparse structure capture. The specific formula is as follows: ； In the formula, These represent the number of nearest neighbors in the three sub-scale neighborhoods, respectively. , , These are the sets of neighborhood points centered at each of the three sub-scales. Represents the core point of the g-th point. Centered on, search Find the nearest neighbor and complete the centralized processing.

3. The sparse point cloud classification method based on MSE-Mamba according to claim 1, characterized in that: In step two, the specific process of the MSE-Mamba multi-scale local feature encoding module is as follows: Step 2: Transpose the dimensions of all points in each scale neighborhood. Converting 3D coordinates to convolutional channel dimensions yields the following coordinate features: ； In the formula, The set of points centered in the i-th scale neighborhood of the g-th core point. This represents the matrix transpose operation, and `permute` represents the dimension rearrangement operation. The characteristic matrix after transpose; Step 22: Use two layers of one-dimensional convolution and ReLU activation to extract coordinate features. Mapping to high-dimensional features: the first layer maps to 128 dimensions, and the second layer maps to 256 dimensions, obtaining the global structural features of the neighborhood. The formula is as follows: ； ； In the formula, This represents a one-dimensional convolution operation. This indicates a batch normalization operation. express Activation function The 128-dimensional high-dimensional features are the output of the first convolutional layer. This is the 256-dimensional high-dimensional feature output by the second convolutional layer; Steps 2 and 3: Extract global structural features from the neighborhood using global max pooling, and concatenate them with local features to achieve local-global semantic fusion. The formula is as follows: ； ； in, For max pooling operation, The feature vector after global max pooling. Indicated in the 4th dimension, This indicates that the global feature will be repeated in this dimension. Second-rate, For globally extended features, This indicates a feature concatenation operation. This is a feature of fusion.

4. Step Two, Part Four: After concatenating the fused features of each neighborhood and performing one-dimensional convolution, perform KNorm neighborhood adaptive normalization and KPool exponential weighted aggregation. The formula is as follows: ； ； ； in, For multi-scale fusion features, , For learnable parameters, These are the features after neighborhood normalization. The aggregated feature vector; Step 2.5: Enhance local context capture capability through bidirectional state-space modeling, and perform layer normalization on the bidirectional fused features and SiLU activation to enhance nonlinear expression, outputting the final local feature representation, as shown in the formula: ； ； In the formula, This indicates a forward state-space modeling operation. This indicates the reverse state-space modeling operation. for Flip feature, This represents the fusion result of forward and backward modeling features with the original features. Representation layer normalization, This represents the local feature representation of the final output.

5. The sparse point cloud classification method based on MSE-Mamba according to claim 1, characterized in that: In step three, the specific process of the global attention Transformer module is as follows: Step 3.1: For local features, introduce a DropPath random discarding strategy and retain the original features through residual connections, expressed by the formula: ； In the formula, Initial global features; Step 32: Through multi-head self-attention calculation, Convert to Q, K, V vectors, and simultaneously calculate global associations through dot product attention, introducing a scaling factor. The equilibrium scale; expressed by the formula: ； ； In the formula, W Q W K W V Here is the attention weight matrix, and H is the number of attention heads. The global association features are calculated after multi-head self-attention. Step 33, Enhanced by MLP and compared with the initial global features Residual join can be expressed by the following formula: ； In the formula, For global enhancement features; Steps 3 and 4: Calculate the 3D coordinates of the core point. The spatial location features of the core points are obtained by expanding the dimension to the same dimension D as the global features using MLP. And inject learnable position embeddings By merging the two, a complete set of location embeddings can be obtained. Finally, it is added directly to the global enhanced features to output an enhanced feature that contains both semantic and spatial location information. Expressed as a formula: ； ； In the formula, Features that integrate spatial location information and global semantic information.

6. The sparse point cloud classification method based on MSE-Mamba according to claim 1, characterized in that, In step four, the specific process of the multi-feature aggregation strategy is as follows: Step 41: Extract the feature vectors corresponding to the category labels. ,Will Feature vectors obtained by max pooling ,Will Feature vectors obtained by average pooling ; Step 42: Map to the target class space through a linear layer, and utilize... The function calculates the probability of each category, expressed by the formula: ； In the formula, For a multilayer perceptron used for class prediction, the final class probability distribution is obtained by calculating the probabilities using Softmax. C represents the number of categories.

Images