A GRSNet-based lidar point cloud lightweight classification and segmentation method and system

Abstract

This invention relates to the fields of computer vision and machine learning, specifically to a lightweight classification and segmentation method and system for LiDAR point clouds based on GRSNet. The method includes: acquiring LiDAR point cloud data; generating a representative point set using a sampling method based on the golden ratio to reduce preprocessing complexity; inputting the point set into a GRSNet network for feature extraction, which consists of multiple processing stages. Each stage constructs a local region through a Sample_and_Group module, uses two SA_Ghost Block modules to learn shared weights and extract deep aggregation features, and performs feature aggregation through a Mixed Pooling module with randomly selected pooling methods; finally, classification or segmentation is completed based on the extracted features. This invention, through innovative sampling methods, a lightweight dual-path feature extraction module, and a randomized mixed pooling mechanism, significantly reduces the number of model parameters and computational complexity while maintaining high-precision classification and segmentation performance.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and machine learning technology, and in particular to a lightweight classification and segmentation method and system for lidar point clouds based on GRSNet. Background Technology

[0002] LiDAR point cloud data processing has become an important research area in modern perception. Classification and segmentation are key tasks in autonomous driving, environmental perception, and digital twins.

[0003] Algorithms that extract features directly from raw point cloud data have a simple architecture, but are limited by computational requirements and efficiency. This makes effective deployment on resource-constrained devices challenging. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight classification and segmentation method and system for lidar point clouds based on GRSNet, which reduces the number of network parameters and computational load while ensuring the accuracy and speed of point cloud classification and segmentation, thereby reducing the difficulty of model deployment.

[0005] To achieve the above objectives, this invention provides a lightweight classification and segmentation method for lidar point clouds based on GRSNet, comprising the following steps: Acquire lidar point cloud data; A representative point set is generated from the lidar point cloud data using the golden ratio sampling method; The representative point set is input into the GRSNet network for multi-level feature extraction. Based on the features extracted by the GRSNet network, point cloud classification or point cloud segmentation tasks are performed, and the corresponding results are output.

[0006] The GRSNet network comprises multiple sequentially connected processing stages, each of which includes: A Sample_and_Group module is used to construct local regions; Two SA_Ghost Block modules are used to learn shared weights and extract deep aggregated features from the local regions, respectively; A Mixed Pooling module is used to aggregate the deep aggregated features.

[0007] Specifically, generating a representative point set from the lidar point cloud data using the golden ratio sampling method includes: A predetermined index sequence is generated based on the golden ratio; Based on the index sequence, corresponding points are directly selected from the lidar point cloud data to form the representative point set.

[0008] The feature extraction process performed by the SA_Ghost Block module includes: The input features are processed through a Primary MLP to obtain the first feature; The first feature is input into a self-attention layer for processing to obtain the second feature; Simultaneously, the first feature is processed through a Cheap_MLP to obtain the third feature; The second feature is concatenated with the third feature, and then residually connected with the input feature to obtain the output feature.

[0009] The Cheap_MLP generates features through linear transformation operations. For each feature channel in the first feature, multiple new feature channels are generated using different linear transformations.

[0010] The feature aggregation process of the Mixed Pooling module is as follows: During the model training phase, for each pooling operation, either the max pooling function or the average pooling function is randomly selected to process the input features; The output of the selected pooling function is used as the result of this feature aggregation.

[0011] On the other hand, the present invention also includes a lightweight classification and segmentation system for lidar point clouds based on GRSNet, comprising a data acquisition module, a sampling module, a feature extraction module, and a task execution module, wherein the sampling module is connected to the data acquisition module, the feature extraction module is connected to the sampling module, and the task execution module is connected to the feature extraction module; The data acquisition module is used to acquire lidar point cloud data; The sampling module is used to generate a representative point set from the lidar point cloud data using the golden ratio sampling method; The feature extraction module includes a GRSNet network for multi-level feature extraction of the representative point set. The GRSNet network includes multiple sequentially connected processing stages, each of which includes: a Sample_and_Group module, two SA_Ghost Block modules, and a Mixed Pooling module. The task execution module is used to perform point cloud classification or point cloud segmentation tasks based on the features extracted by the feature extraction module, and output the corresponding results.

[0012] This invention presents a lightweight classification and segmentation method and system for LiDAR point clouds based on GRSNet. By employing a sampling method based on the golden ratio to directly generate sampling indices, it avoids the complex distance calculations inherent in traditional farthest point sampling, significantly reducing the computational complexity of data preprocessing. Secondly, the SA_Ghost Block module constructs a dual-path feature generation mechanism by combining the mainstream paths of Primary_MLP and self-attention layers with the inexpensive linear transformation paths of Primary_MLP and Cheap_MLP. This significantly expands the feature channels with minimal additional parameters, achieving an efficient balance between feature representation capability and model lightweighting. Simultaneously, the Mixed Pooling module randomly selects the pooling method during training, acting as a regularization mechanism and effectively improving the model's generalization ability and robustness. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0014] Figure 1 This is a diagram of the overall framework of the GRSNet network of the present invention.

[0015] Figure 2 This is a schematic diagram of the golden ratio sampling process of the present invention.

[0016] Figure 3 This is a schematic diagram of the Mixed Pooling module of the present invention.

[0017] Figure 4 This describes the classification performance of the GRSNet model of this invention on the ScanObjectNN dataset.

[0018] Figure 5 This describes the segmentation performance of the GRSNet model of this invention on the ShapeNetPart dataset.

[0019] Figure 6 This is a flowchart of the lightweight classification and segmentation method for lidar point clouds based on GRSNet according to the present invention. Detailed Implementation

[0020] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0021] This invention provides a lightweight classification and segmentation method for lidar point clouds based on GRSNet, comprising the following steps: S1: Acquire LiDAR point cloud data.

[0022] In this embodiment, the lidar point cloud data can be obtained in real time by vehicle-mounted, airborne, or fixed lidar sensors, or it can be an existing point cloud dataset read from a storage device or database.

[0023] S2: Generate a representative point set from the lidar point cloud data using the golden ratio sampling method.

[0024] Specifically, S21: Generate a predetermined index sequence based on the golden ratio; S22: Based on the index sequence, directly select the corresponding points from the lidar point cloud data to form the representative point set.

[0025] In this embodiment, point cloud sampling aims to extract a representative set of points from the original point cloud data. This process is effective in simplifying the original data and improving the efficiency and accuracy of subsequent processing. The traditional farthest point sampling (FPS) method has shown effectiveness to some extent, but it faces significant challenges such as high computational complexity and inability to comprehensively characterize point cloud correlations. This application proposes a novel point cloud sampling method called Golden Ratio Sampling (GRS). GRS uses the golden ratio to generate a set of specialized sampling point indices, and then uses these indices to retrieve the corresponding sampling points. Since this method does not require distance calculation, its computational complexity is relatively low.

[0026] In point cloud datasets, point cloud data is typically stored as an array, consisting of N points. Each point can be uniquely identified by an index, with the index value ranging from 0 to N-1, such as... Figure 2 As shown in (a). Unlike the farthest point sampling method, the GRS method does not require calculating the index of the farthest point; instead, it directly generates a set of indices using the golden ratio. Using these indices, we can find the corresponding sampling points, such as... Figure 2 As shown in (b).

[0027] S3: Input the representative point set into the GRSNet network for multi-level feature extraction.

[0028] Specifically, the GRSNet network comprises multiple sequentially connected processing stages, each of which includes: A Sample_and_Group module is used to construct local regions; Two SA_Ghost Block modules are used to learn shared weights and extract deep aggregated features from the local regions, respectively; A Mixed Pooling module is used to aggregate the deep aggregated features.

[0029] In this embodiment, the present invention establishes a lightweight network model called GRSNet. The GRSNet network structure is as follows: Figure 1 As shown, the model consists of four phases, each including Sample_and_Group, SA_GhostBlock, and Mixed Pooling. Each phase can be represented by the following formula: in and Both refer to the SA_Ghost Block module. Used to learn shared weights from local regions. Used to extract deep aggregation features. Aggregation function. This is a mixed pooling operation. Geometric_Affine is a geometric affine transformation.

[0030] Furthermore, the feature extraction process performed by the SA_Ghost Block module includes: The input features are processed through a Primary MLP to obtain the first feature; The first feature is input into a self-attention layer for processing to obtain the second feature; Simultaneously, the first feature is processed through a Cheap_MLP to obtain the third feature; The second feature is concatenated with the third feature, and then residually connected with the input feature to obtain the output feature.

[0031] The Cheap_MLP generates features through linear transformation operations. For each feature channel in the first feature, multiple new feature channels are generated using different linear transformations.

[0032] In this embodiment, the SA_Ghost Block module is a lightweight feature extraction module. Its structure is as follows: Figure 1 As shown, this module mainly consists of two MLP layers (Primary_MLP and Cheap_MLP) and a self-attention mechanism layer (SA). Primary_MLP is the main feature extractor. Cheap_MLP is a cheap linear transformation. Both Primary_MLP and Cheap_MLP consist of a 1D convolutional layer, a batch normalization layer, and an activation function layer.

[0033] After the input data X is processed by the SA_Ghost Block module, a feature vector Y with n channels can be obtained. This process can be divided into the following four steps: (1) After the input data X passes through the Primary_MLP layer, the output feature vector Y1 is obtained. It can be expressed by the following formula: Where X has C channels, For size The convolution kernel, the number of channels of Y1 , 𝑚≤ 𝑛, s is a hyperparameter, and 𝑛 is the number of channels in the feature vector Y finally obtained by the Ghost Block module.

[0034] (2) Pass Y1 through the SA layer to obtain the output feature vector Y2. The formula is as follows: Where Q, K, and V are input matrices, representing the query matrix, key matrix, and value matrix, respectively. It is the vector dimension. The Attention formula works by weighting the similarity between Q and K to obtain a weighted sum of V corresponding to the input.

[0035] (3) Pass Y1 through a Cheap_MLP layer to obtain the output feature vector Y3. Specifically, for each feature channel in the feature vector Y1, a series of inexpensive linear transformations are applied. To generate Each feature channel. The formula is expressed as follows: Where i represents the i-th feature channel in Y1, and j represents the j-th linear operation.

[0036] (4) Concatenate the feature vectors Y2 and Y3, and add them to the input data to obtain the final output feature vector Y. The formula is expressed as follows: .

[0037] Furthermore, the feature aggregation process performed by the Mixed Pooling module is as follows: During the model training phase, for each pooling operation, either the max pooling function or the average pooling function is randomly selected to process the input features; The output of the selected pooling function is used as the result of this feature aggregation.

[0038] In this implementation, max pooling and average pooling are two common pooling methods. Max pooling selects the largest feature value as the output within each pooling window, which helps retain the most salient features. However, it is prone to losing other detailed information. Average pooling, on the other hand, takes the average of all feature values ​​within the pooling window as the output, which helps reduce the dimensionality of the feature map and can smooth the features. However, it can also lead to the loss or blurring of feature details.

[0039] In the GRSNet model, hybrid pooling is used, such as... Figure 3 As shown, hybrid pooling combines average pooling and max pooling, using them simultaneously to extract features by comprehensively considering both the extreme values ​​and average values ​​of local features. It can be expressed by the following formula.

[0040] In the formula, It is a random number of 0 or 1, indicating that max pooling and average pooling are randomly selected for each pooling operation.

[0041] Hybrid pooling replaces conventional deterministic pooling operations with stochastic processes. By randomly applying max pooling and average pooling methods during model training, it helps prevent network overfitting to some extent. Furthermore, the computational cost of this method is negligible.

[0042] S4: Based on the features extracted by the GRSNet network, perform point cloud classification or point cloud segmentation tasks and output the corresponding results.

[0043] In this embodiment, the deep features extracted by the GRSNet network through the aforementioned processing stages can be directly used as input for downstream perception tasks. To verify the effectiveness of this method in practical tasks, two embodiments are described below: point cloud classification and point cloud component segmentation.

[0044] Example 1: Classification tasks In this section, the classification performance of the GRSNet model on the ScanObjectNN dataset is evaluated and analyzed. The initial learning rate is set to 0.003, cosine decay to 0.05, epoch to 300, seed to 7240, and SmoothCrossEntropy and AdamW are used. Data augmentation is performed using PointCloudScaling, PointCloudCenterAndNormalize, and PointCloudRotation. Given the inherent biases of this series of models, each network model was trained and tested four times consecutively, and the optimal value was taken as the final experimental result. The results are as follows... Figure 4As shown; Figure 4 This represents the classification performance of the GRSNet model on the ScanObjectNN dataset. Bold numbers indicate the optimal values.

[0045] As can be seen, GRSNet achieved an overall accuracy (OA) of 87.4% and a mean accuracy (mAcc) of 85.4% on the ScanObjectNN dataset with only 0.7M parameters. This demonstrates high accuracy achieved with a relatively small number of parameters.

[0046] Example 2 Task splitting In this section, we evaluate and analyze the segmentation performance of the GRSNet model on the ShapeNetPart dataset. We set the initial learning rate to 0.003 and the epoch to 350. We used the CosineAnnealingLR scheduler and the AdamW optimizer to train our model. Due to the nature of partial segmentation, no data augmentation was used. The results are as follows... Figure 5 As shown; Figure 5 This represents the segmentation performance of the GRSNet model on the ShapeNetPart dataset. Bold numbers indicate the optimal values.

[0047] As can be seen, GRSNet achieves an accuracy of 85.9% in instance segmentation, comparable to PointMLP but superior to PointNet, PointNet++, and DGCNN. Furthermore, it achieves the highest accuracy in categories such as aero, bag, cap, car, laptop, pistol, and rock.

[0048] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A lightweight classification and segmentation method for lidar point clouds based on GRSNet, characterized in that, Includes the following steps: Acquire lidar point cloud data; A representative point set is generated from the lidar point cloud data using the golden ratio sampling method; The representative point set is input into the GRSNet network for multi-level feature extraction. Based on the features extracted by the GRSNet network, point cloud classification or point cloud segmentation tasks are performed, and the corresponding results are output.

2. The lightweight classification and segmentation method for lidar point clouds based on GRSNet as described in claim 1, characterized in that, The GRSNet network comprises multiple sequentially connected processing stages, each of which includes: A Sample_and_Group module is used to construct local regions; Two SA_Ghost Block modules are used to learn shared weights and extract deep aggregated features from the local regions, respectively. A Mixed Pooling module is used to aggregate the deep aggregated features.

3. The lightweight classification and segmentation method for lidar point clouds based on GRSNet as described in claim 1, characterized in that, A representative point set is generated from the lidar point cloud data using the golden ratio sampling method, specifically including: A predetermined index sequence is generated based on the golden ratio; Based on the index sequence, corresponding points are directly selected from the lidar point cloud data to form the representative point set.

4. The lightweight classification and segmentation method for lidar point clouds based on GRSNet as described in claim 2, characterized in that, The feature extraction process performed by the SA_Ghost Block module includes: The input features are processed through a Primary MLP to obtain the first feature; The first feature is input into a self-attention layer for processing to obtain the second feature; Simultaneously, the first feature is processed through a Cheap_MLP to obtain the third feature; The second feature is concatenated with the third feature, and then residually connected with the input feature to obtain the output feature.

5. The lightweight classification and segmentation method for lidar point clouds based on GRSNet as described in claim 4, characterized in that, The Cheap_MLP generates features through linear transformation operations. For each feature channel in the first feature, multiple new feature channels are generated using different linear transformations.

6. The lightweight classification and segmentation method for lidar point clouds based on GRSNet as described in claim 2, characterized in that, The process of feature aggregation in the Mixed Pooling module is as follows: During the model training phase, for each pooling operation, either the max pooling function or the average pooling function is randomly selected to process the input features; The output of the selected pooling function is used as the result of this feature aggregation.

7. A lightweight classification and segmentation system for lidar point clouds based on GRSNet, used to implement the method as described in any one of claims 1-6, characterized in that, It includes a data acquisition module, a sampling module, a feature extraction module, and a task execution module. The sampling module is connected to the data acquisition module, the feature extraction module is connected to the sampling module, and the task execution module is connected to the feature extraction module. The data acquisition module is used to acquire lidar point cloud data; The sampling module is used to generate a representative point set from the lidar point cloud data using the golden ratio sampling method; The feature extraction module includes a GRSNet network for multi-level feature extraction of the representative point set. The GRSNet network includes multiple sequentially connected processing stages, each of which includes: a Sample_and_Group module, two SA_Ghost Block modules, and a Mixed Pooling module. The task execution module is used to perform point cloud classification or point cloud segmentation tasks based on the features extracted by the feature extraction module, and output the corresponding results.

Images