A road plane intersection point cloud semantic segmentation method
By using an improved RandLA-Net network and a weighted cross-entropy loss function, the problem of low segmentation accuracy of point clouds at road intersections is solved, achieving higher segmentation accuracy and completeness, which is suitable for road infrastructure information management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2024-04-29
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies do not achieve high accuracy in point cloud semantic segmentation in road intersection scenarios. Traditional methods are limited by manually set empirical thresholds, and deep learning methods do not fully represent features.
A lightweight laser scanning system was used to collect point cloud data from multiple locations and perspectives. The data was then segmented using an improved RandLA-Net point cloud semantic segmentation network. The network architecture adopted an encoder-decoder system, combined with a feature aggregation module and nearest neighbor interpolation. A weighted cross-entropy loss function was set to balance the imbalance of sample data.
It improves the segmentation accuracy and completeness of point clouds at road intersections, enhances the ability to extract local and global features, reduces the negative impact of imbalanced sample numbers on model segmentation performance, and achieves higher prediction accuracy and F1 score.
Smart Images
Figure CN118429637B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of point cloud semantic segmentation, specifically a method for point cloud semantic segmentation of road intersections. Background Technology
[0002] With the development of digital road mapping technology, LiDAR-based 3D scanning technology has shown promising prospects in road infrastructure information management. Lightweight LiDAR can quickly acquire point cloud data while ensuring equipment performance and has been flexibly applied to various scenarios such as ordinary highways and road intersections. The ancillary facilities of urban road intersections are diverse, including traffic lights, barriers, drainage, lighting, greening and landscaping facilities, which place higher demands on semantic segmentation performance when processing point cloud information.
[0003] Traditional point cloud semantic segmentation methods mostly employ supervised learning algorithms. The segmentation accuracy of these algorithms is often limited by manually set empirical thresholds, making them difficult to adapt to complex and ever-changing road intersection scenarios. In recent years, some methods have begun to use deep learning for semantic segmentation, such as projection-based semantic segmentation methods and point-based semantic segmentation methods. Although these methods have achieved good results on some public point cloud datasets, their segmentation performance in road intersection scenarios still faces the problem of incomplete feature representation. Summary of the Invention
[0004] Purpose of the invention: This invention addresses the problem of low accuracy in segmenting point clouds of road intersections using existing algorithms by providing a semantic segmentation method for point clouds of road intersections.
[0005] The technical solution adopted in this invention is:
[0006] A semantic segmentation method for point clouds at road intersections includes the following steps:
[0007] Step 1: Use a lightweight laser scanning system to acquire point cloud data from multiple locations and multiple perspectives;
[0008] Step 2: Register and stitch the original point cloud data to stitch the point cloud data in the local coordinate system into complete point cloud data of the road intersection;
[0009] Step 3: Construct an improved RandLA-Net point cloud semantic segmentation network. The overall network architecture adopts an encoder-decoder system: In the spatial information encoding stage, random downsampling (RS) is used layer by layer to save computational resources. The improved feature aggregation module (FA*) is input to gradually increase the feature dimension of each point to enrich the key feature information. In the spatial information decoding stage, to improve the computational efficiency of the model, the point cloud feature set is upsampled (US) through Nearest Interpolation, taking the features of the nearest neighbor points as the features of each point. At the same time, it is concatenated with the feature information in the encoding stage through cross-layer skip connections, and then input into a multilayer perceptron for feature restoration. Finally, the predicted values of the point cloud semantic labels are output through three shared fully connected (FC) layers and a Dropout layer.
[0010] Step 4: Train the improved RandLA-Net point cloud semantic segmentation network, and use the trained improved RandLA-Net point cloud semantic segmentation network for point cloud segmentation at road intersections to predict point cloud semantic labels.
[0011] Furthermore, the lightweight laser scanning system measures using a static setup method, with multiple measurement stations arranged during measurement. Point clouds from different field angles are collected at the same station to ensure the acquisition of complete point cloud data of the road intersection.
[0012] Furthermore, the improved feature aggregation module consists of a spatial information encoding layer, an aggregation pooling layer, and an expanded residual layer.
[0013] Furthermore, the spatial information encoding layer explicitly encodes the coordinate information of the point cloud, enabling the network to learn the local spatial structure of the original point cloud from the relative distances and positional relationships between points. The steps are as follows:
[0014] First, calculate the Euclidean distance from each point in the point cloud to the central sampling point, and establish a neighborhood point set;
[0015] Secondly, based on the feature information of the center point and its neighboring points, a feature similarity matrix is constructed using equation (1) to obtain the K points with the highest similarity to each center point. new A set of similar neighborhood points is formed by 1,000 neighboring points;
[0016]
[0017] In the formula: f i f i k The center point p are respectively iand its neighboring points The corresponding feature vector;
[0018] Then, the relative positions of the point cloud are explicitly encoded based on the set of similar neighborhood points, and the relative position features are obtained through equation (2);
[0019]
[0020] In the formula: s i The coordinates of the center point, The coordinates of the neighboring points are given by the symbol. This indicates concatenating feature matrices of the same dimension.
[0021] Finally, the features corresponding to each neighboring point of each center point are concatenated with their relative position features to obtain the enhanced feature vector.
[0022] Furthermore, the aggregate pooling layer enhances point cloud features based on multi-pooling fusion by combining the characteristics of different pooling operations, while retaining the original network attention mechanism. The steps are as follows:
[0023] First, the maximum pooling function H is used. max Average pooling function H mean Obtain the global features and local features in the neighborhood of each point;
[0024] Then, attention scores for each point are learned through a shared function, and neighborhood features are aggregated based on the attention scores using a weighted summation method to obtain attention pooling features;
[0025] Finally, the results of max pooling and average pooling are concatenated with the high-dimensional features output by attention pooling.
[0026] Furthermore, the loss function of the improved RandLA-Net point cloud semantic segmentation network is set as a weighted cross-entropy loss function, which assigns weight coefficients to each category according to the sample size. Categories with more samples receive smaller weights, while categories with fewer samples receive larger weights, thereby reducing the impact of sample imbalance on the semantic segmentation results.
[0027] The formulas for calculating the weighting coefficients are shown in equations (3) and (4);
[0028]
[0029] w i =exp(-r i (4)
[0030] In the formula, n i Let r be the number of points in the i-th class. i w represents the proportion of the number of point clouds in the i-th category to the total number of point clouds in all categories.i Let n be the weighting coefficient for the i-th class. class The number of categories for which the model is trained;
[0031] Weighted cross-entropy loss function L w The calculation formula is:
[0032]
[0033] Among them, y i For the one-hot vector encoding of class i, This represents the probability of the predicted type given by the model.
[0034] The beneficial effects of this invention are:
[0035] This invention constructs a semantic segmentation framework for road intersection point clouds, consisting of a "lightweight LiDAR data acquisition - point cloud registration and stitching - road semantic segmentation network." This framework mines the features of 3D point cloud data in road intersection scenarios, providing a reference for information extraction and digital asset management of existing road infrastructure. The invention adds an improved feature aggregation module to the original network. This module introduces a feature similarity matrix into its spatial information encoding layer, using features of neighboring points similar to the local features of the central sampling point as neighborhood feature information. The results of max pooling, average pooling, and attention pooling are aggregated in the aggregation pooling layer to extract local neighborhood features and global information from the point cloud. Furthermore, to ensure a relatively balanced contribution of different categories of sample data to the loss function during model training, a weighted cross-entropy loss function is used, reducing the negative impact of uneven point cloud numbers of different categories on the model's segmentation performance. Attached Figure Description
[0036] Figure 1 This is a flowchart of the method of the present invention;
[0037] Figure 2 This is a diagram of the laser scanning system of the present invention;
[0038] Figure 3 This is a diagram of the improved RandLA-Net point cloud semantic segmentation network structure of the present invention;
[0039] Figure 4 This is a structural diagram of the improved feature aggregation module of the present invention. Detailed Implementation
[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. These specific embodiments are only used to further illustrate the present invention and do not limit the scope of protection of the claims of this application.
[0041] like Figure 1As shown, this invention provides a semantic segmentation method for point clouds at road intersections, comprising the following steps:
[0042] Step 1: As Figure 2 As shown, a lightweight laser scanning system was used to collect point cloud data from multiple points and multiple perspectives.
[0043] Due to the limited field of view of lightweight lidar, the location and number of test stations need to be set up according to the actual scene. After a single scan of the lightweight lidar is completed, the lidar is rotated in the horizontal direction to scan the next area, and it is ensured that there is an overlapping area between adjacent point clouds to facilitate subsequent registration and stitching. Repeat the above operation until the collected point cloud data covers the test intersection area.
[0044] Step 2: Register and stitch the original point cloud data to stitch the point cloud data in the local coordinate system into complete point cloud data of the road intersection;
[0045] The method uses a sequential registration and stitching strategy to register and stitch the original point cloud. That is, the local coordinate system where a point cloud is located is used as the reference coordinate system, and then all spatial point clouds are transformed to the reference coordinate system in turn. This method only registers and stitches two point clouds at a time and gradually expands the range of point clouds in the reference coordinate system. Therefore, the amount of data processed at the same time is small, and it has a high computational speed when registering scene-level point clouds.
[0046] Step 3: Construct as follows Figure 3 The improved RandLA-Net point cloud semantic segmentation network is shown. The overall network architecture adopts an encoder-decoder system: In the spatial information encoding stage, random downsampling is performed layer by layer to save computational resources. The improved feature aggregation module is input to increase the feature dimension of each point layer by layer to enrich the key feature information. In the spatial information decoding stage, in order to improve the computational efficiency of the model, the point cloud feature set is upsampled by nearest neighbor interpolation, and the features of the nearest neighbor points are used as the features of each point. At the same time, the feature information in the encoding stage is concatenated through cross-layer skip connections, and then input into a multilayer perceptron for feature restoration. Finally, the predicted values of the point cloud semantic labels are output through three shared fully connected layers and a Dropout layer.
[0047] like Figure 4 As shown, the improved feature aggregation module consists of a spatial information encoding layer, an aggregation pooling layer, and an expanded residual layer.
[0048] The spatial information coding layer explicitly encodes the coordinate information of the point cloud, enabling the network to learn the local spatial structure of the original point cloud from the relative distances and positional relationships between points. The steps are as follows:
[0049] First, the KNN algorithm is used to calculate the Euclidean distance from each point in the point cloud to the center sampling point, where p is the center sampling point of the input point cloud.i Select the K nearest neighboring points, denoted as . Establish a neighborhood point set;
[0050] Secondly, based on the feature information of the center point and its neighboring points, a feature similarity matrix is constructed using equation (1), and the center point p is selected according to the feature similarity matrix. i K with the highest similarity new 100 neighboring points, i.e. Form a set of similar neighborhood points;
[0051]
[0052] In the formula: f i f i k The center point p are respectively i and its neighboring points The corresponding feature vector;
[0053] Then, the relative positions of the point cloud are explicitly encoded based on the set of similar neighborhood points, and the relative position features are obtained through equation (2);
[0054]
[0055] In the formula: s i The coordinates of the center point, The coordinates of the neighboring points are given by the symbol. This indicates concatenating feature matrices of the same dimension.
[0056] Finally, for each neighborhood point of each center point Compare its corresponding features with its relative position features The features are concatenated to obtain the enhanced feature vector. After passing through the MLP multilayer perceptron, the output of the spatial information coding unit is the neighborhood features of each central sampling point.
[0057] The pooling layer enhances point cloud features by combining the characteristics of different pooling operations, while preserving the original network's attention mechanism, based on multi-pooling fusion. The steps are as follows:
[0058] First, the maximum pooling function H is used. max Average pooling function H mean Obtain the global features and local features in the neighborhood of each point;
[0059] Then, based on the neighborhood features output by the spatial information coding unit... Set up a shared function g() to learn individual attention scores for each point. This function consists of a shared multilayer perceptron (Shared MLP) with shared weights W and an activation function Softmax. Based on this, a weighted summation method is used to aggregate neighborhood features;
[0060] Finally, the results of max pooling and average pooling are concatenated with the high-dimensional features output by attention pooling.
[0061] The improved RandLA-Net point cloud semantic segmentation network sets the loss function to the weighted cross-entropy loss function, which assigns weight coefficients to each category according to the sample size. Categories with more samples receive smaller weights, while categories with fewer samples receive larger weights, thereby reducing the impact of sample imbalance on semantic segmentation results.
[0062] The formulas for calculating the weighting coefficients are shown in equations (3) and (4);
[0063]
[0064] w i =exp(-r i (4)
[0065] In the formula, n i Let r be the number of points in the i-th class. i w represents the proportion of the number of point clouds in the i-th category to the total number of point clouds in all categories. i Let n be the weighting coefficient for the i-th class. class The number of categories for which the model is trained;
[0066] Weighted cross-entropy loss function L w The calculation formula is:
[0067]
[0068] Among them, y i For the one-hot vector encoding of class i, This represents the probability of the predicted type given by the model.
[0069] Step 4: Train the improved RandLA-Net point cloud semantic segmentation network, and use the trained improved RandLA-Net point cloud semantic segmentation network for point cloud segmentation at road intersections to predict point cloud semantic labels.
[0070] To verify the effectiveness of the point cloud semantic segmentation method for road intersections proposed in this invention, point cloud data of urban road roundabouts and crossroads were acquired using a laser scanning system. After processing, two datasets were formed: a roundabout dataset (dataset 1) and a crossroads dataset (dataset 2). The datasets contain six categories: road surface, tall vegetation, low vegetation, sidewalk, guardrail, and poles.
[0071] Tables 1 and 2 compare the segmentation results of different networks on datasets 1 and 2, respectively. It can be seen that the method of this invention is significantly better than the other two networks in terms of overall segmentation performance. The overall prediction accuracy of the method of this invention on the two test sets is 87.68% and 89.61%, respectively, which is 2.40% and 2.85% higher than that of the RandLA-Net network; the average F1 scores are 80.61% and 82.76%, respectively, which is 1.67% and 1.52% higher than that of the RandLA-Net network.
[0072] The method of this invention introduces a spatial information encoding module based on feature similarity, which enhances the network's ability to learn neighborhood features of the input point cloud, and improves the accuracy and completeness of segmenting point clouds of most categories. The method uses max pooling and average pooling functions to enhance the salient local features and global features of the input point cloud, respectively, so that the model can extract more spatial structure information. The method sets a weighted cross-entropy loss function to improve the problem of imbalance in the sample size of the input point cloud.
[0073] Table 1. Semantic segmentation results of point clouds at intersections in different network planes (Dataset 1)
[0074]
[0075] Table 2. Semantic segmentation results of point clouds at intersections in different network planes (Dataset 2)
[0076]
Claims
1. A semantic segmentation method for point clouds at road intersections, characterized in that: The method includes the following steps: Step 1: Collect point cloud data from multiple points and multiple perspectives using a lightweight laser scanning system; Step 2: Register and stitch the original point cloud data to stitch the point clouds in the local coordinate system into complete road intersection point cloud data; Step 3: Construct an improved RandLA-Net point cloud semantic segmentation network; An encoder-decoder architecture is adopted: In the spatial information encoding layer, random downsampling is performed layer by layer to save computational resources, and the improved feature aggregation module is input to gradually increase the feature dimension of each point to enrich key feature information; In the spatial information decoding layer, the point cloud feature set is upsampled through nearest neighbor interpolation, and the features of the nearest neighbor points are used as the features of each point. Simultaneously, cross-layer skip connections are used to concatenate the feature information from the encoding stage, and then the data is input into a multilayer perceptron for feature reconstruction; Finally, three shared fully connected layers and a Dropout layer are used... Step 4: Train the improved RandLA-Net point cloud semantic segmentation network and use the trained improved RandLA-Net point cloud semantic segmentation network for point cloud segmentation at road intersections to predict point cloud semantic labels; The improved feature aggregation module in Step 3 consists of a spatial information encoding layer, an aggregation pooling layer, and an expanded residual layer; The spatial information encoding layer explicitly encodes the coordinate information of the point cloud, enabling the network to learn the local spatial structure of the original point cloud from the relative distance and positional relationship between each point. The steps are as follows: First, calculate the Euclidean distance from each point in the point cloud to the center sampling point and establish a neighborhood point set; Second, based on the feature information of the center point and its neighborhood points, construct a feature similarity matrix through Equation (1) to obtain the Knew neighborhood points with the highest similarity to each center point, forming a similar neighborhood point set; In the formula: , The center point and its neighboring points The corresponding feature vectors; then, based on the set of similar neighborhood points, the relative positions of the point cloud are explicitly encoded, and the relative position features are obtained through equation (2); In the formula: The coordinates of the center point, The coordinates of the neighboring points are given by the symbol. This involves concatenating feature matrices of the same dimension; finally, concatenating the features corresponding to each neighboring point of each center point with its relative position features to obtain the enhanced feature vector; the aggregation pooling layer enhances point cloud features based on multi-pooling fusion by combining the characteristics of different pooling operations, while retaining the original network attention mechanism. The steps are as follows: First, the global features and local features in the neighborhood of each point are obtained by using the max pooling function Hmax and the average pooling function Hmean, respectively; then, the attention score of each point is learned through a shared function, and then the neighborhood features are aggregated by weighted summation based on the attention scores to obtain the attention pooled features; finally, the results of max pooling and average pooling are concatenated with the high-dimensional features output by the attention pooling.
2. The semantic segmentation method for point clouds at road intersections according to claim 1, characterized in that: In step 1, the lightweight laser scanning system is used for measurement in a static setup manner. During the measurement, multiple measurement stations are set up, and point clouds with different field of view angles are collected at the same station to ensure that complete point cloud data of the road intersection is obtained.
3. The semantic segmentation method for point clouds at road intersections according to claim 2, characterized in that: In step 3, the loss function of the improved RandLA-Net point cloud semantic segmentation network is set as the weighted cross-entropy loss function. The weight coefficients are assigned to each category according to the sample size. The category with more samples gets a smaller weight, and the category with fewer samples gets a larger weight, thereby reducing the impact of sample imbalance on the semantic segmentation results. The calculation formulas for the weight coefficients are as shown in equations (3) and (4). In the formula, ni is the number of points in class i, ri is the proportion of the number of points in class i to the total number of points in all classes, wi is the weighting coefficient of class i, and nclass is the number of classes trained by the model; the weighted cross-entropy loss function Lw is calculated as follows: Where yi is the one-hot vector code of the i-th class. This represents the probability of the predicted type given by the model.