A point cloud segmentation method based on feature bias value and attention mechanism
The point cloud segmentation method using feature bias values and attention mechanisms solves the problems of insufficient local feature extraction and feature loss in point cloud classification, achieving more efficient point cloud scene segmentation, especially improving accuracy in complex scenes and small-scale objects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING FORESTRY UNIV
- Filing Date
- 2023-04-10
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies suffer from insufficient local feature extraction and feature loss in point cloud classification, especially when dealing with irregular and disordered point cloud data, making it difficult to effectively utilize convolutional neural networks.
A point cloud segmentation method based on feature bias and attention mechanism is adopted. The method uses the farthest sampling method to downsample, the KNN algorithm to extract feature points, calculates the feature bias value of the spherical neighborhood, and uses residual multilayer perceptron and attention pooling to fuse local features to form global features.
It improves the performance of point cloud classification, especially enhancing the model's segmentation ability in complex scenes, improving the average accuracy and segmentation accuracy of small-scale objects, reducing feature loss, and improving the model's scene segmentation ability.
Smart Images
Figure CN116958956B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of 3D point cloud classification technology, specifically relating to a point cloud segmentation method based on feature deviation values and attention mechanisms. Background Technology
[0002] The widespread application of point cloud data in fields such as autonomous driving, robotic mapping, and 3D reconstruction necessitates effective point cloud classification as a prerequisite for its application. To this end, numerous researchers have proposed various models to explore better ways to analyze and identify point cloud data. The emergence of neural networks, especially convolutional neural networks (CNNs), has spurred the development of computer vision. However, because point clouds are irregular and disordered geometric data, CNNs with regular architectures are inefficient in point cloud classification. Therefore, how to leverage the advantages of neural networks in point cloud classification is a current research hotspot.
[0003] Currently, deep learning-based point cloud classification methods can be broadly categorized into four types: voxel-based methods, multi-view projection-based methods, methods that directly use the original point cloud format, and graph-based methods.
[0004] Traditional point cloud classification methods often employ simple distance-based normalization for local feature extraction, which leads to feature loss when dealing with point cloud data of varying geometries and densities. Furthermore, point clouds themselves are disordered and irregular, making them unsuitable for direct processing using convolutional neural networks. This invention optimizes local geometric feature extraction based on group normalization and utilizes a residual multilayer perceptron to aggregate local features into global point cloud features for classification and recognition. Summary of the Invention
[0005] The purpose of this invention is to address the aforementioned shortcomings of existing technologies by proposing a point cloud segmentation method based on feature bias values and an attention mechanism. The proposed model effectively extracts both local and global features of point clouds, and the symmetrical network structure of the residual multilayer perceptron perfectly suits the disordered and irregular characteristics of point clouds, further improving the performance of point cloud classification.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] Design a point cloud segmentation method based on feature bias values and attention mechanism, including the following specific steps:
[0008] Step 1) Downsample the point cloud using the furthest sampling method;
[0009] Step 2) Using the sampling points in Step 1) as the center, set a threshold radius and use the KNN algorithm to extract feature points within the range;
[0010] Step 3): For the feature points extracted in Step 2), calculate the feature deviation value for each spherical neighborhood with Step 1) as the center point to obtain the 64-dimensional local features of the spherical neighborhood.
[0011] First, a spherical neighborhood is constructed with the sampling point as the center, and k nearest neighbors are selected within the spherical neighborhood. Then, the Euclidean distance from the sampling point to each nearest neighbor is calculated.
[0012]
[0013] In the formula: Represents Euclidean distance. Indicates the first One sampling point, express The first in the neighborhood A nearby point.
[0014] Next, calculate the eigenvalue deviation of the neighborhood:
[0015]
[0016] In the formula: Indicates the characteristic deviation value. The number of neighboring points in the neighborhood. Indicates the number of sampling points. This indicates the dimension.
[0017] Finally, the feature values of the region where the sampling point is located are obtained:
[0018]
[0019] In the formula: , These are two self-learning parameters. It is a decimal used for numerical stability.
[0020] Step 4) Utilize the residual multilayer perceptron to share weight parameters and employ attention pooling to fuse local features to obtain 1024-dimensional global features of the point cloud:
[0021] Employing attention mechanism for feature sets To aggregate features, a shared function is designed to learn the attention score for each feature within the feature set.
[0022]
[0023] In the formula: It is a function for calculating feature scores. It is a learnable parameter. It is 3D array, It consists of a multilayer perceptron and a Softmax function.
[0024] The aggregated features are the sum of the products of the features within the feature group and their weights. Compared with maximum value aggregation, feature aggregation based on the attention mechanism can provide more feature information for scene segmentation.
[0025]
[0026] Step 5) Combine the local features obtained in Step 3) with the global features of the point cloud obtained in Step 4) to segment the point cloud.
[0027] This invention proposes a point cloud segmentation method based on feature bias values and an attention mechanism. The advantages are as follows: In the local feature extraction stage of the point cloud, a lightweight feature extractor is designed to extract combined local features of the point cloud, thereby enhancing the local features. Furthermore, an attention mechanism is introduced to aggregate the local features of the point cloud, assigning weights to different features while preserving the point cloud features, thus enhancing the model's scene segmentation capability. Specifically:
[0028] (1) For complex point cloud scenes, the geometric feature extraction module based on feature deviation values can effectively extract deep geometric features of point clouds and fully explore the semantic information of point clouds. In addition, in order to avoid feature loss in scene segmentation tasks, an attention mechanism is introduced when aggregating features. For input features, it can adaptively learn the importance of different features, make full use of the extracted geometric information, reduce the interference of redundant features, avoid the loss of some valuable information, and improve the model's segmentation ability in complex scenes.
[0029] (2) The model proposed in this invention can effectively complete the scene segmentation task of point cloud. Compared with the results of existing technologies, this model has achieved the highest average accuracy OA. At the same time, it has also achieved the highest F1 score for small-scale objects such as cars and fences. This shows that the improved model proposed in this invention can effectively mine deep information and effectively realize the scene segmentation of point cloud. Attached Figure Description
[0030] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0031] Figure 1 The above is a flowchart of the point cloud segmentation method based on feature deviation value and attention mechanism of the present invention.
[0032] Figure 2 This is the architecture of the point cloud segmentation method based on feature deviation value and attention mechanism of the present invention.
[0033] Figure 3 This invention is a feature extraction module based on feature deviation values.
[0034] Figure 4 This is the feature aggregation module based on the attention mechanism of the present invention.
[0035] Figure 5 , is the Vaihingen3D semantic dataset in this invention.
[0036] Figure 6 , are the segmentation results of the Vaihingen3D test set in this invention.
[0037] Figure 7 The model of this invention is a comparison of some erroneous segmentation results in the Vaihingen3D test set with standard samples.
[0038] Figure 8 , are the standard values of the model of this invention compared with some erroneous segmentation results in the Vaihingen3D test set and the standard samples. Detailed Implementation
[0039] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0040] The structural features of the present invention will now be described in detail with reference to the accompanying drawings.
[0041] See Figures 1-4 This paper presents a point cloud segmentation method based on feature deviation and attention mechanisms. For point cloud scene segmentation, given the complex geometric structures contained within point clouds, spherical sampling is used to construct different local neighborhoods to distinguish different set structures and mine deeper semantic features. For each neighborhood, its feature deviation value is calculated to obtain its geometric features. Specifically:
[0042] Step 1: Input the 3D point cloud data to be segmented;
[0043] Step 2: Downsample the point cloud using the farthest point sampling method;
[0044] Step 3: Construct a spherical neighborhood with the sampling point as the center, and select k points in each spherical neighborhood;
[0045] Step 4: For each point cloud neighborhood obtained in Step 3, calculate its feature deviation value, and then obtain its local geometric features based on the feature deviation value:
[0046] Calculate the Euclidean distance from the sampling point to each neighboring point:
[0047]
[0048] In the formula: Represents Euclidean distance. Indicates the first One sampling point, express The first in the neighborhood A nearby point.
[0049] Next, calculate the eigenvalue deviation of the neighborhood:
[0050]
[0051] In the formula: Indicates the characteristic deviation value. The number of neighboring points in the neighborhood. Indicates the number of sampling points. This indicates the dimension.
[0052] Finally, the feature values of the region where the sampling point is located are obtained:
[0053]
[0054] In the formula: , These are two self-learning parameters. It is a decimal used for numerical stability.
[0055] Using the above method, when extracting local features, different feature deviation values will be obtained due to the different geometric structures of local regions. By calculating the feature deviation values, the semantic information of the point cloud can be mined in depth, thereby enhancing the local features of the point cloud.
[0056] Because of the high density of point cloud information, when extracting point cloud features, multilayer perceptrons may encounter an excessive number of parameters as the depth of the multilayer perceptron increases. To solve this problem, a residual multilayer perceptron is introduced, which can extract higher-order features and reduce the number of parameters, thereby optimizing the model.
[0057] Step 5: Extract local geometric features of the point cloud obtained in Step 4 using a residual multilayer perceptron;
[0058] Step 6: For the local geometric features of the point cloud extracted in Step 5, attention pooling is used to aggregate the local geometric features of the point cloud to obtain the global features of the point cloud:
[0059] Employing attention mechanism for feature sets To aggregate features, a shared function is designed to learn the attention score for each feature within the feature set.
[0060]
[0061] In the formula: It is a function for calculating feature scores. It is a learnable parameter. It is 3D array, It consists of a multilayer perceptron and a Softmax function.
[0062] The aggregated features are the sum of the products of the features within the feature group and their weights. Compared with maximum value aggregation, feature aggregation based on the attention mechanism can provide more feature information for scene segmentation.
[0063]
[0064] Step 7: Combine the global and local features of the point cloud, use a multilayer perceptron to segment the scene and output the segmentation results.
[0065] See Figures 5-8 To further illustrate this point, this embodiment also provides experiments and results analysis, as follows:
[0066] The experimental data used was the Vaihingen3D dataset from the International Society for Photogrammetry and Remote Sensing (ISPRS). This dataset provides nine different classes of data, including power lines, low vegetation, ground, vehicles, fences, roofs, facades, shrubs, and trees. The training set contains 753,876 points, and the test set contains 411,722 points. Table 1 shows the number of points for each class in the training and test sets (see Table 1). Figure 5 The test set and training set are shown.
[0067] The evaluation metrics used are overall accuracy (OA), precision, recall, and F1 score. The formula for calculating OA is as follows:
[0068]
[0069] In the formula, TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative, respectively. TP represents the number of positive samples correctly classified by the classifier, TN represents the number of negative samples correctly classified by the classifier, FP represents the number of false positive samples, and FN represents the number of false negative samples.
[0070] The F1 score is calculated based on the precision and recall of each category. When the number of categories is unbalanced, the F1 score may be higher than the OA score.
[0071]
[0072] In the formula: Precision and recall are calculated as follows:
[0073]
[0074]
[0075] Table 1. Number of points in each object category in the training and testing datasets:
[0076]
[0077] Experimental environment and network parameter settings:
[0078] The experimental hardware configuration was as follows: 16GB of RAM, an Intel(R) Xeon(R) CPU @ 2.30GHz processor, and a Tesla P100 graphics card. Network parameters were set as follows: stochastic gradient descent was used to improve training accuracy and efficiency; the initial learning rate was set to 0.1; the minimum learning rate was set to 0.005; cosine annealing was used to dynamically adjust the learning rate; stochastic gradient descent was used as the optimizer; and the Dropout parameter for fully connected layers was set to 0.5.
[0079] Cosine annealing is a technique for dynamically adjusting the learning rate. It starts with a large learning rate, quickly reduces it to near zero, and then increases it again. Since the model's weights are randomized at the beginning of training, using a large learning rate at this stage may cause the model to oscillate. With a smaller learning rate for initial warm-up, the model tends to stabilize. Cosine annealing helps accelerate the model's convergence speed.
[0080]
[0081] In the formula: It represents current learning efficiency. Which cycle is it?
[0082] Analysis of experimental results:
[0083] To verify the model's performance in point cloud segmentation, the model was trained on the training set and its segmentation effect was verified on the test set. The segmentation results were visualized (see [link]). Figure 6 The confusion matrix is shown in Table 2.
[0084] Low vegetation, ground, rooftops, and trees all achieved F1 scores exceeding 80, with ground scoring 92.08 and rooftops scoring 96.67. Good segmentation accuracy was also achieved for small-scale objects, such as cars, with an F1 score of 83.28. (Combined...) Figure 6 As shown in Table 2, classification errors were mainly concentrated in fences and shrubs (see Table 2). Figures 7-8 The results show a comparison between some incorrect segmentation results and standard samples. The reason for the incorrect segmentation is that fences and shrubs are structurally similar and the number of training points for fences is small. The model did not learn the features of fences sufficiently, resulting in most fences being identified as shrubs by the model, while some points with lower elevations were identified as low vegetation.
[0085] Table 2 Confusion matrix of test set classification results:
[0086]
[0087] Ablation experiment:
[0088] To verify the effectiveness of the proposed feature extraction module and attention pooling module in the model, different model architectures were designed and tested through ablation experiments. The experimental results are shown in Table 3. The introduction of the geometry extraction module and attention pooling module improved the average accuracy and average F1 score of the model.
[0089] As shown in Table 3, the improved feature extraction module based on feature deviation and the feature aggregation module based on attention mechanism proposed in this embodiment are replaced by KNN and max pooling, respectively, to verify the effectiveness of the improved modules. Ours-1 shows the segmentation results of KNN replacing the feature extraction module based on feature deviation, which is 0.12% higher than PointNet. Ours-2 shows the segmentation results of max pooling replacing the feature aggregation module based on attention mechanism, which is 0.53% higher than PointNet++. The geometric feature extractor based on feature deviation can effectively obtain deep semantic information of point clouds, enhance the model's ability to recognize different categories of objects, and at the same time, the attention mechanism can obtain more effective features by learning the weight parameters of different features, thus improving the accuracy of the model in small-scale models such as cars. The model based on PointNet++ of this invention can effectively achieve scene segmentation of point clouds, effectively extract deep geometric features of point clouds, and obtain rich semantic information through attention fusion of geometric information and local features. It can achieve good segmentation tasks for both large and small scales in point cloud scenes.
[0090] Table 3 Ablation Experiment Results:
[0091]
[0092] Note: Ours-1 represents ablation from geometric extraction, and Ours-2 represents ablation from pooling modules.
[0093] Comparison with existing methods:
[0094] The segmentation results of this embodiment are compared and analyzed with the experimental results provided by ISPRS and other deep learning segmentation methods. The results are shown in Table 4. In the results provided by ISPRS, the RIT_1 model is a point cloud segmentation model based on the elevation values of ground points, incorporating the spectral information of the point cloud as input. NANJ2 constructs feature maps from multiple layers and extracts deep semantic information of the point cloud through a convolutional network to achieve scene segmentation of the point cloud. The point cloud segmentation method based on feature bias and attention mechanism proposed in this embodiment improves the OA by 1.49 compared to NANJ2. NANJ2 achieved the highest F1 score in shrub classification by introducing features such as RGB, elevation, intensity, and roughness. This embodiment uses the three-dimensional coordinates of the point cloud as input. Comparing various segmentation results, it can be seen that the model of this invention achieves good segmentation results for small-scale models such as fences and cars.
[0095] Meanwhile, this embodiment selects three deep learning methods for directly segmenting point clouds: PointNet++, PointNetSIFT, and PointNetCNN. Due to the introduction of an attention mechanism, this model can adaptively extract better fine-grained local features for point cloud segmentation during feature aggregation. Furthermore, in the geometric feature extraction part, an improved feature bias value is introduced, which can better distinguish the differences between different local neighborhood structures. Compared to PointNetCNN, the accuracy of this embodiment is improved by 3.39. In summary, this model, by improving the geometric extraction method and feature aggregation method, can effectively improve the point cloud segmentation accuracy in complex scenes.
[0096] Table 4 Comparison with existing models:
[0097]
[0098] in conclusion:
[0099] To address the issues of PointNet++'s failure to deeply mine local geometric features of point clouds and feature loss during feature aggregation in point cloud scene segmentation tasks, a point cloud semantic segmentation model based on feature bias values and attention mechanism was designed, and the effectiveness of the model was verified on the Vaihingen dataset provided by ISPRS.
[0100] (1) For complex point cloud scenes, the geometric feature extraction module based on feature deviation values can effectively extract deep geometric features of point clouds and fully explore the semantic information of point clouds. In addition, in order to avoid the problem of feature loss in scene segmentation tasks, an attention mechanism is introduced when aggregating features. For input features, it can adaptively learn the importance of different features, make full use of the extracted geometric information, reduce the interference of redundant features, avoid the loss of some valuable information, and improve the model's segmentation ability in complex scenes.
[0101] (2) The model proposed in this embodiment can effectively complete the scene segmentation task of point cloud. Compared with the experimental results provided by ISPRS and the results of PointNet++, PointNetSIFT and PointNetCNN, this model achieved the highest average accuracy OA. At the same time, it also achieved the highest F1 score for small-scale objects such as cars and fences. This shows that the improved model proposed in this invention can effectively mine deep information and effectively realize the scene segmentation of Lidar point cloud.
[0102] The point cloud segmentation method based on feature bias and attention mechanism of the present invention designs a lightweight feature extractor to extract local combined features of point cloud in the local feature extraction stage to enhance the local features of point cloud, and introduces an attention mechanism to aggregate the local features of point cloud. While preserving the point cloud features, different features are assigned weights to enhance the scene segmentation capability of the model.
[0103] Specifically, in practice, firstly, during the feature extraction stage, a feature deviation-based extraction method is adopted to calculate the feature deviation values of different local neighborhoods, thereby obtaining the geometric features of different neighborhoods; secondly, during the feature aggregation stage, a pooling module based on an attention mechanism is used to aggregate features for scene segmentation, solving the problem of feature loss caused by the max pooling aggregation method, which affects the model's segmentation accuracy; in addition, during the model design process, a residual multilayer perceptron is used instead of a multilayer perceptron. The use of the residual multilayer perceptron avoids parameter redundancy caused by the increase in model depth, further optimizing the model's structure.
[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A point cloud segmentation method based on feature bias values and an attention mechanism, characterized in that, The specific steps include the following: S1. Downsample the point cloud using the furthest sampling method; S2. Using the sampling points in S1 as the center, set a threshold radius and use the KNN algorithm to extract feature points within the range. S3. For the feature points extracted from S2, calculate the feature deviation value for each spherical neighborhood with S1 as the center point to obtain the 64-dimensional local features of the spherical neighborhood. S4. Using residual multilayer perceptrons to share weight parameters and employing attention pooling to fuse local features, we obtain 1024-dimensional global features of the point cloud. S5. Segment the point cloud using the local features of S3 and the global features of S4 after stitching. In S3, the specific steps for obtaining the local features of the spherical neighborhood are as follows: S31. Construct a spherical neighborhood with the sampling point as the center, and filter out samples within the spherical neighborhood. Then, calculate the Euclidean distance from the sampling point to each neighboring point: In the formula: Represents Euclidean distance. Indicates the first One sampling point, express The first in the neighborhood A nearby point; S31. Calculate the feature deviation value of the neighborhood: In the formula: Indicates the characteristic deviation value. The number of neighboring points in the neighborhood. Indicates the number of sampling points. Indicates dimension; S33. Obtain the feature values of the region where the sampling point is located: In the formula: , These are two self-learning parameters. It is a decimal used for numerical stability; In S4, attention pooling is used to aggregate the local features of the point cloud to obtain the global features of the point cloud. The specific steps are as follows: S41. Employ an attention mechanism for the feature set. Aggregation is performed by designing a shared function to learn the attention score for each feature within the feature set; S42. The aggregated features are the sum of the products of the features within the feature group and their weights. Compared with maximum value aggregation, feature aggregation based on the attention mechanism can provide more feature information for scene segmentation.
2. The point cloud segmentation method based on feature deviation value and attention mechanism according to claim 1, characterized in that, In step S41, the shared function learns the attention score for each feature within the feature set: In the formula: It is a function for calculating feature scores. It is a parameter that is learned. It is 3D array, It consists of a multilayer perceptron and a softmax function; In step S42, the global features of the point cloud are: In the formula: It is the global feature of the point cloud obtained through attention pooling.