Point cloud semantic segmentation method for improving RandLA-Net network

Abstract

The invention provides a point cloud semantic segmentation method for improving a RandLA-Net network. The method comprises the following steps: step 1, carrying out data preprocessing grid downsampling; step 2, calculating a point cloud normal vector by a local curved surface fitting method; thirdly, local multi-dimensional feature fusion is carried out, and local geometric information is captured; fourthly, important features are automatically learned and aggregated through a self-attention module and attention pooling; 5, cascading an up-sampling result with the intermediate feature of the coding layer through jump connection; and 6, predicting the semantic tag by using a full connection layer. According to the invention, high-precision and high-efficiency automatic point cloud semantic segmentation in a large-scale outdoor scene is realized.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud semantic segmentation, and specifically is a point cloud semantic segmentation method of an improved RandLA-Net network. Background Art

[0002] With the development of digital surveying and mapping towards intelligent surveying and mapping, intelligent scene interpretation based on three-dimensional laser point clouds has received widespread attention in recent years. Among them, point cloud semantic segmentation classifies each point and labels the semantic information of different areas, which is the basic link of intelligent perception of autonomous driving. Due to factors such as differences in data acquisition equipment and methods, and different levels of scene complexity, point cloud data often have problems such as large local density changes, spatial discreteness and uneven distribution, and large occlusion effects, which will lead to incomplete point cloud data and missing features. Therefore, how to accurately and efficiently segment large-scale complex point cloud scenes is still a challenging task and a current research hotspot.

[0003] At present, traditional point cloud semantic segmentation is mainly based on geometric features such as elevation, normal vector, curvature, and intensity of point clouds for classification. However, these methods need to set more thresholds, and require more prior knowledge for target structures with weak generalization. In addition, the degree of automatic extraction is low, making it difficult to apply to complex and diverse road environments. Compared with traditional methods, deep learning methods have made great progress in image and natural language processing due to their high processing efficiency and ability to recognize complex features. Related research has also gradually expanded to the field of point cloud data processing. RangNet++ converts irregular point cloud data into image information, but some features will inevitably be lost during the projection process, thus affecting the segmentation accuracy. VoxelNet solves the unstructured nature of point clouds by voxelization, but the voxelization process significantly increases the computing time and memory cost. The PointNet algorithm, based on the consideration of permutation invariance, realizes the semantic segmentation task directly based on point clouds, which is of pioneering significance. However, when segmenting large-scale, multi-category road point cloud data, the computational efficiency of the above methods will be greatly reduced as the segmentation accuracy increases. The RandLA-Net algorithm uses the random sampling (RS) method to reduce the amount of data, and proposes a local feature aggregation module (LFA) to compensate for the information loss problem in the random downsampling process, achieving very high efficiency in terms of memory and computing. However, the RS method used above is prone to losing effective feature information while improving efficiency, resulting in sparse point cloud data and difficulty in accurately reflecting the local differences and similarities of point clouds.

[0004] In view of the defects of existing point cloud semantic segmentation methods, such as large point cloud feature loss, insufficient local feature extraction and low segmentation accuracy, this paper proposes a point cloud semantic segmentation method based on an improved RandLA-Net network. By improving the local feature mining capability of the network, the overall segmentation capability of large-scale outdoor scene point clouds is improved, thereby achieving more accurate semantic segmentation. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a method for semantic segmentation of point clouds based on an improved RandLA-Net network. On the basis of the RandLA-Net algorithm, the relative position encoding feature of the normal vector is added, and local multi-dimensional feature fusion is used to expand the network receptive field and improve the segmentation accuracy of the overall network. Taking advantage of the fact that the self-attention mechanism is good at extracting the internal correlation of features, a self-attention module is designed to enhance the performance of local feature extraction of the network. In view of the problem of imbalanced point cloud categories on complex roads, a loss function of fusion weights is established, and different weights are assigned to categories to further improve the segmentation performance of the network.

[0006] A method for improving the point cloud semantic segmentation of the RandLA-Net network includes the following steps:

[0007] Step 1: Input the original point cloud data, and use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data;

[0008] Step 2: Based on the point cloud data obtained in step 1, the laser point cloud normal vector is calculated using the local surface fitting method, the obtained point cloud normal vector is added to the preprocessed point cloud data, saved as a ply format file, and a kdtree file and a projection file are generated;

[0009] Step 3: Based on step 2, the RandLA-Net network encoder structure is optimized, and the normal vector is input into the network together with the point cloud coordinates and their features. The local multi-dimensional feature fusion module can easily embed the coordinates, features and normal vectors of all adjacent points, and then capture the local geometric information through the relative spatial positions of adjacent points.

[0010] Step 4: Based on the local feature information of the center point and each neighborhood point obtained in step 3, a feature aggregation operation with a self-attention mechanism is used, and then attention pooling is used to automatically learn and aggregate important features;

[0011] Step 5: In the decoding stage, the K-nearest neighbor algorithm (KNN) is used to search for the nearest neighbor point, and the point cloud feature set is upsampled (Up Sampling, US) using the nearest neighbor interpolation. The upsampling result is cascaded with the intermediate features generated by the encoding layer through skip connections;

[0012] Step 6: Use a shared multi-layer perceptron (MLP) to restore the number of data points and feature dimensions. Finally, the fully connected layer predicts the semantic label and converts the feature dimension into the number of categories, and outputs the predicted value of the point cloud segmentation result; substitute the predicted value of the point cloud segmentation result into the weighted cross entropy loss function, and perform back propagation derivation on the cross entropy loss function to optimize the neural network parameters.

[0013] In step 2, based on the point cloud data obtained in step 1, a local surface fitting method is used to calculate the laser point cloud normal vector;

[0014] The specific steps are as follows:

[0015] Step 2-1, fit the local optimal space tangent plane in combination with the least squares principle to construct the objective function;

[0016]

[0017] Where n is the normal vector of the target plane p; d is the spatial distance from the origin to the target plane;

[0018] Step 2-2: Construct the covariance matrix based on the results of step 2-1:

[0019]

[0020] in, is the centroid of the neighborhood point set;

[0021] Step 2-3: Perform eigenvalue decomposition on the covariance matrix A as follows:

[0022]

[0023] Among them, λ is the eigenvalue of matrix A, λ 1 ≥λ 2 ≥λ 3 When the eigenvalue is the minimum, the corresponding eigenvector is the normal vector of the target plane p, that is, point p i The normal vector of .

[0024] The optimized RandLA-Net network encoder structure described in step 3 inputs the normal vector into the network together with the point cloud coordinates and their features. The local multi-dimensional feature fusion module can easily embed the coordinates, features and normal vectors of all adjacent points, thereby capturing local geometric information through the relative spatial positions of adjacent points.

[0025] The specific steps are as follows:

[0026] Step 3-1: For a given input point P and corresponding input feature F, the main purpose of feature extraction is to enable each point to obtain relevant features with neighboring points. Use the KNN algorithm to obtain the center point p i , center point feature f i and its normal vector n i The 3D coordinates and normal vectors of k neighboring points are added in the process of encoding the input features, which provides a good initial condition for the network to mine deeper geometric features. Euclidean distance Cascaded together through a shared MLP to calculate the relative position encoding

[0027]

[0028] in, represents feature concatenation, ||·|| represents the calculation of Euclidean distance;

[0029] Step 3-2: Use the same method to correct the normal vector n i and its k adjacent normal vectors Encode to obtain the relative position encoding of the normal vector

[0030]

[0031] In step 4, based on the local feature information of the center point and each neighborhood point obtained in step 3, a feature aggregation operation with a self-attention mechanism is adopted, and then attention pooling is used to automatically learn and aggregate important features;

[0032] The specific steps are as follows:

[0033] Step 4-1: For the input center point p i , obtain k neighboring points through KNN algorithm Then for p i and k neighboring points The feature vectors of are linearly transformed to map the dimensions to high-dimensional space, and then the attention relationship is calculated through the relationship function;

[0034] Step 4-2: Add the obtained attention relationship to the position encoding vector with the normal vector feature added, and then pass it through the softmax normalized exponential function to obtain the attention map, and calculate the position encoding vector with the normal vector added:

[0035]

[0036] in, The center point p iIts k neighboring points The relative position encoding of The center point p i Its k neighboring points The relative position encoding of the normal vector;

[0037] Step 4-3: Multiply the attention map score by the transformed feature matrix of the neighboring points element by element to assign corresponding weights to different neighboring points; finally, aggregate the neighboring point features through the sum operation to obtain the output of the self-attention module:

[0038]

[0039] Among them, f i The center point p i The feature point set, For p i The feature point set of the k neighboring points of a point; η, α, β, γ are all MLP multi-layer perceptron operations; δ is the subtraction of the relational function; ⊙ is the Hadamard product, that is, the element-by-element multiplication of two matrices; θ is the position encoding operation;

[0040] Step 4-4: Using the attention pooling function without additional information can automatically learn and aggregate important local features, thereby reducing the amount of data to improve network processing efficiency. In the attention pooling, for each local enhanced feature Define a ρ(·) function to learn the attention score for each feature:

[0041]

[0042] Where W is the learning weight of MLP. Unimportant features are removed by masking, and local features are weighted and summed to obtain local aggregate features.

[0043] In step 6, the predicted value of the point cloud segmentation result is substituted into the weighted cross entropy loss function, and the cross entropy loss function is back-propagated and derived to optimize the neural network parameters;

[0044] The specific steps are as follows:

[0045] Step 6-1: Use the cross entropy function weighted by the number of point clouds as the loss function of the network model:

[0046]

[0047] Among them, B is the batch size; y is the true value distribution of the point cloud label; is the distribution of point cloud label prediction values; w j is the weight of the point cloud loss value calculation result; L is the number of point clouds of each category in the training set point cloud file.

[0048] Advantages of the present invention:

[0049] 1. The present invention takes the normal vector as an important factor in local feature extraction, adopts the relative position encoding of the normal vector to fuse other local features, and enhances the local feature extraction ability of the network. It is verified by examples that it can effectively improve the semantic segmentation results of the network. The OA and MIoU are improved by 2.77% and 2.88% respectively compared with the traditional model that encodes the center point coordinates, neighborhood point coordinates, relative position and Euclidean distance together.

[0050] 2. Compared with the traditional RandLA-Net network, the introduction of the local multi-dimensional feature fusion module in the present invention improves 3.08% (OA) and 3.07% (MIoU), respectively, and improves the effectiveness and robustness of the network in learning local geometric structures from large-scale complex road scenes; the introduction of the self-attention module can also improve the segmentation accuracy of the traditional network, and its OA and MIoU are increased by 2.11% and 2.15% respectively, which is slightly less than the network that introduces the local multi-dimensional feature fusion module. The network that integrates the local multi-dimensional feature fusion and attention modules proposed in the present invention obtains the best accuracy, and its OA and MIoU reach 97.38% and 78.70% respectively, which fully verifies that different modules have certain contributions to the classification performance, and the fusion of the two modules can further improve the semantic segmentation performance of point cloud.

[0051] 3. Compared with the PointNet++ network, the OA and MIoU values ​​of the present invention are improved by 10.53% and 25.01% respectively. Compared with the KPConv network, the OA and MIoU are improved by 2.62% and 3.46% respectively. In addition, compared with other point cloud semantic segmentation algorithms, the present invention is superior to several other point cloud semantic segmentation methods in most categories, further verifying the effectiveness of the fusion of local multidimensional feature fusion and attention modules. The single-class intersection-over-union ratio of other objects such as roads, vegetation, buildings, wires, and cars is higher than 80%. For the rod-shaped object category, the network proposed in the present invention also achieved higher results. Compared with the RandLA-Net network, the rod-shaped object and tree trunk categories are improved by 7.6% and 6.1% respectively. This shows that the network proposed in the present invention can not only achieve the best overall segmentation performance, but also significantly improve the feature extraction capability of small target categories in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flow chart of a method for point cloud semantic segmentation using an improved RandLA-Net network according to the present invention;

[0053] Figure 2 A specific flow chart of step 2 of an embodiment of the present invention;

[0054] Figure 3A specific flow chart of step 3 of an embodiment of the present invention;

[0055] Figure 4 A specific flow chart of step 4 of an embodiment of the present invention;

[0056] Figure 5 A summary flow chart of one embodiment of the present invention;

[0057] Figure 6 A comparison chart of semantic segmentation results of different algorithms for Toronto-3D dataset using the present invention;

[0058] Figure 7 This is a comparison chart of the semantic segmentation results of different algorithms for the Fuxin campus dataset using the present invention. Specific implementation method:

[0059] An embodiment of the present invention is further described below with reference to the accompanying drawings.

[0060] In the embodiment of the present invention, the point cloud semantic segmentation method of the RandLA-Net network is improved, such as Figure 1 As shown, the following steps are included:

[0061] Step 1: Input the original point cloud data, and use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data;

[0062] Step 2: Based on the point cloud data obtained in step 1, the laser point cloud normal vector is calculated using the local surface fitting method, the obtained point cloud normal vector is added to the preprocessed point cloud data, saved as a ply format file, and a kdtree file and a projection file are generated;

[0063] Step 2-1: Fit the local optimal space tangent plane using the least squares principle to construct the objective function:

[0064]

[0065] Where n is the normal vector of the target plane p; d is the spatial distance from the origin to the target plane;

[0066] Step 2-2: Construct the covariance matrix based on the results of step 2-1:

[0067]

[0068] Among them, p is the centroid of the neighborhood point set;

[0069] Step 2-3: Perform eigenvalue decomposition on the covariance matrix A as follows:

[0070]

[0071] Among them, λ is the eigenvalue of matrix A, λ 1 ≥λ 2 ≥λ 3 When the eigenvalue is the minimum, the corresponding eigenvector is the normal vector of the target plane p, that is, point p i The normal vector of .

[0072] The optimized RandLA-Net network encoder structure described in step 3 inputs the normal vector into the network together with the point cloud coordinates and their features. The local multi-dimensional feature fusion module can easily embed the coordinates, features and normal vectors of all adjacent points, thereby capturing local geometric information through the relative spatial positions of adjacent points.

[0073] Step 3-1: For a given input point P and corresponding input feature F, the main purpose of feature extraction is to enable each point to obtain relevant features with neighboring points. Use the KNN algorithm to obtain the center point p i , center point feature f i and its normal vector n i The 3D coordinates and normal vectors of k neighboring points are added in the process of encoding the input features, which provides a good initial condition for the network to mine deeper geometric features. Euclidean distance Cascaded together through a shared MLP to calculate the relative position encoding

[0074]

[0075] in, represents feature concatenation, ||·|| represents the calculation of Euclidean distance;

[0076] Step 3-2: Use the same method to correct the normal vector n i and its k adjacent normal vectors Encode to obtain the relative position encoding of the normal vector

[0077]

[0078] In step 4, based on the local feature information of the center point and each neighborhood point obtained in step 3, a feature aggregation operation with a self-attention mechanism is adopted, and then attention pooling is used to automatically learn and aggregate important features;

[0079] Step 4-1: For the input center point p i , obtain k neighboring points through KNN algorithm Then for p i and k neighboring points The feature vectors of are linearly transformed to map the dimensions to high-dimensional space, and then the attention relationship is calculated through the relationship function;

[0080] Step 4-2: Add the obtained attention relationship to the position encoding vector with the normal vector feature added, and then pass it through the softmax normalized exponential function to obtain the attention map, and calculate the position encoding vector with the normal vector added:

[0081]

[0082] in, The center point p i Its k neighboring points The relative position encoding of The center point p i Its k neighboring points The relative position encoding of the normal vector;

[0083] Step 4-3: Multiply the attention map score by the transformed feature matrix of the neighboring points element by element to assign corresponding weights to different neighboring points; finally, aggregate the neighboring point features through the sum operation to obtain the output of the self-attention module:

[0084]

[0085] Among them, f i The center point p i The feature point set, For p i The feature point set of the k neighboring points of a point; η, α, β, γ are all MLP multi-layer perceptron operations; δ is the subtraction of the relational function; ⊙ is the Hadamard product, that is, the element-by-element multiplication of two matrices; θ is the position encoding operation;

[0086] Step 4-4: Using the attention pooling function without additional information can automatically learn and aggregate important local features, thereby reducing the amount of data to improve network processing efficiency. In the attention pooling, for each local enhanced feature Define a ρ(·) function to learn the attention score for each feature:

[0087]

[0088] Where W is the learning weight of MLP. Unimportant features are removed by masking, and local features are weighted and summed to obtain local aggregate features.

[0089] Step 5: In the decoding stage, the K nearest neighbor algorithm is used to search for the nearest neighbor point, the point cloud feature set is upsampled using the nearest neighbor interpolation, and the upsampled result is cascaded with the intermediate features generated by the encoding layer through the skip connection;

[0090] Step 6: Use a shared multi-layer perceptron to restore the number of data points and feature dimensions. Finally, use the fully connected layer to predict the semantic label and convert the feature dimension into the number of categories, and output the predicted value of the point cloud segmentation result. Substitute the predicted value of the point cloud segmentation result into the weighted cross entropy loss function, and perform back propagation derivation on the cross entropy loss function to optimize the neural network parameters.

[0091] Step 6-1: Use the cross entropy function weighted by the number of point clouds as the loss function of the network model:

[0092]

[0093] Where B is the batch size; y is the point cloud label true value distribution; y is the point cloud label predicted value distribution; w j is the weight of the point cloud loss value calculation result; L is the number of point clouds of each category in the training set point cloud file.

[0094] In the examples of the present invention, in order to evaluate the effectiveness and accuracy of the proposed network, two different large-scale outdoor point cloud scenes were used for testing. The two data are the Toronto-3D dataset and the point cloud dataset of a university campus in Fuxin City, Liaoning Province. The Toronto-3D dataset was captured by the vehicle-mounted Teledyne Optech Maverick MLS system. The dataset has approximately 78.3 million points in a 1 km road section. The point cloud density is approximately 1000 points / m 2 . The dataset contains 8 categories, which are defined as roads, road markings, natural trees, buildings, utility lines, pole-shaped targets, cars, and fences. The point cloud dataset of a university campus in Fuxin City, Liaoning Province, obtains laser point cloud data through a mobile backpack laser scanning device, and finally obtains an original point cloud of 86.42 million points. The original point cloud is manually annotated by the CloudCompare tool and divided into 10 categories, namely roads, other ground, vegetation, buildings, wires, tree trunks, poles, cars, fences, and unclassified. The experimental dataset covers multiple scenes such as urban areas and campuses. The data is massive, the targets are diverse, and they are mutually occluded. It is representative and challenging.

[0095] The present invention is implemented based on the Tensorflow deep learning framework. The network model training and testing are carried out on a server whose hardware device CPU is Intel Xeon Silver 4210R and GPU is NVIDIA RTX 6000. The network model uses the Adam optimizer to update the model, the initial learning rate is 0.01, the learning rate decreases by 5% after each period, the batch size is 6, the number of training rounds (epoch) is set to 100, the number of closest points K is set to 16, and the sample capacity is 40960.

[0096] like Figure 6As shown, it is the segmentation result of the test data selected from the Toronto-3D point cloud dataset, in which the parts with obvious contrast are marked with black boxes. It can be seen that compared with the RandLA-Net network, the segmentation results achieved by the network proposed in the present invention are more consistent with the ground truth labels and produce fewer misclassifications, especially in the sub-areas marked by the example boxes. The main scenes within the C box are fences and buildings. The RandLA-net network ignores the fences in the space and marks them as buildings. The algorithm proposed in the present invention can clearly distinguish the two categories and separate them perfectly. It can be seen that the overall semantic segmentation effect of the network proposed in the present invention performs well, but there are also certain misidentifications.

[0097] like Figure 7 As shown, in order to verify that the network proposed in the present invention has better application capabilities in complex road scenarios, Figure 7 The prediction results of the Fuxin campus dataset are visualized and analyzed, and compared with the visualization results of the RandLA-Net network. The areas with obvious improvements are marked with black boxes. It can be seen that after incorporating local multidimensional features, the local feature extraction capability is significantly enhanced, the rod segmentation results are more obvious and complete, and the cut road edges are smoother. The network proposed in the present invention performs better than the RandLA-Net network in semantic segmentation effects in most categories, and the misclassification phenomenon is effectively suppressed, which verifies the reliability of the present invention.

[0098] The above is only the most basic specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any replacement that can be understood by a person skilled in the art within the technical scope disclosed by the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A point cloud semantic segmentation method based on an improved RandLA-Net network, characterized in that: The following steps are involved: Step 1: Input the original point cloud data, and use the voxel grid filtering method to downsample the original scene to obtain the preprocessed point cloud data; Step 2: Based on the point cloud data obtained in step 1, the laser point cloud normal vector is calculated using the local surface fitting method, the obtained point cloud normal vector is added to the preprocessed point cloud data, saved as a ply format file, and a kdtree file and a projection file are generated; Step 3: Based on step 2, the RandLA-Net network encoder structure is optimized, and the normal vector is input into the network together with the point cloud coordinates and their features. The local multi-dimensional feature fusion module can easily embed the coordinates, features and normal vectors of all adjacent points, and then capture the local geometric information through the relative spatial positions of adjacent points. Step 4: Based on the local feature information of the center point and each neighborhood point obtained in step 3, a feature aggregation operation with a self-attention mechanism is used, and then attention pooling is used to automatically learn and aggregate important features; Step 5: In the decoding stage, the K-nearest neighbor algorithm (KNN) is used to search for the nearest neighbor point, and the point cloud feature set is upsampled (Up Sampling, US) using the nearest neighbor interpolation. The upsampling result is cascaded with the intermediate features generated by the encoding layer through skip connections; Step 6: Use a shared multi-layer perceptron (MLP) to restore the number of data points and feature dimensions. Finally, the fully connected layer predicts the semantic label and converts the feature dimension into the number of categories, and outputs the predicted value of the point cloud segmentation result; substitute the predicted value of the point cloud segmentation result into the weighted cross entropy loss function, and perform back propagation derivation on the cross entropy loss function to optimize the neural network parameters.

2. According to claim 1, the method for semantic segmentation of point cloud using an improved RandLA-Net network is characterized in that: In step 2, based on the point cloud data obtained in step 1, a local surface fitting method is used to calculate the laser point cloud normal vector; The specific steps are as follows: Step 2-1, fit the local optimal space tangent plane in combination with the least squares principle to construct the objective function; Where n is the normal vector of the target plane p; d is the spatial distance from the origin to the target plane; Step 2-2: Construct the covariance matrix based on the results of step 2-1: in, is the centroid of the neighborhood point set; Step 2-3: Perform eigenvalue decomposition on the covariance matrix A as follows: Among them, λ is the eigenvalue of matrix A. When λ1≥λ2≥λ3 is the minimum eigenvalue, the corresponding eigenvector is the normal vector of the target plane p, that is, point p i The normal vector of .

3. According to claim 1, the method for semantic segmentation of point cloud using an improved RandLA-Net network is characterized in that: The optimized RandLA-Net network encoder structure described in step 3 inputs the normal vector into the network together with the point cloud coordinates and their features. The local multi-dimensional feature fusion module can easily embed the coordinates, features and normal vectors of all adjacent points, thereby capturing local geometric information through the relative spatial positions of adjacent points. The specific steps are as follows: Step 3-1: For a given input point P and corresponding input feature F, the main purpose of feature extraction is to enable each point to obtain relevant features with neighboring points. Use the KNN algorithm to obtain the center point p i , center point feature f i and its normal vector n i The 3D coordinates and normal vectors of k neighboring points are added in the process of encoding the input features, which provides a good initial condition for the network to mine deeper geometric features. Euclidean distance Cascaded together through a shared MLP to calculate the relative position encoding in, represents feature concatenation, ||·|| represents the calculation of Euclidean distance; Step 3-2: Use the same method to correct the normal vector n i and its k adjacent normal vectors Encode to obtain the relative position encoding of the normal vector 4. According to the improved RandLA-Net network point cloud semantic segmentation method of claim 1, it is characterized in that: In step 4, based on the local feature information of the center point and each neighborhood point obtained in step 3, a feature aggregation operation with a self-attention mechanism is adopted, and then attention pooling is used to automatically learn and aggregate important features; The specific steps are as follows: Step 4-1: For the input center point p i , obtain k neighboring points through KNN algorithm Then for p i and k neighboring points The feature vectors of are linearly transformed to map the dimensions to high-dimensional space, and then the attention relationship is calculated through the relationship function; Step 4-2: Add the obtained attention relationship to the position encoding vector with the normal vector feature added, and then pass it through the softmax normalized exponential function to obtain the attention map, and calculate the position encoding vector with the normal vector added: in, The center point p i Its k neighboring points The relative position encoding of The center point p i Its k neighboring points The relative position encoding of the normal vector; Step 4-3: Multiply the attention map score by the transformed feature matrix of the neighboring points element by element to assign corresponding weights to different neighboring points; finally, aggregate the neighboring point features through the sum operation to obtain the output of the self-attention module: Among them, f i The center point p i The feature point set, f i k For p i The feature point set of the k neighboring points of a point; η, α, β, γ are all MLP multi-layer perceptron operations; δ is the subtraction of the relational function; ⊙ is the Hadamard product, that is, the element-by-element multiplication of two matrices; θ is the position encoding operation; Step 4-4: Using the attention pooling function without additional information can automatically learn and aggregate important local features, thereby reducing the amount of data to improve network processing efficiency. In the attention pooling, for each local enhanced feature Define a ρ(·) function to learn the attention score for each feature: Where W is the learning weight of MLP. Unimportant features are removed by masking, and local features are weighted and summed to obtain local aggregate features.

5. According to claim 1, the method for semantic segmentation of point cloud using an improved RandLA-Net network is characterized in that: In step 6, the predicted value of the point cloud segmentation result is substituted into the weighted cross entropy loss function, and the cross entropy loss function is back-propagated and derived to optimize the neural network parameters; The specific steps are as follows: Step 6-1: Use the cross entropy function weighted by the number of point clouds as the loss function of the network model: Among them, B is the batch size; y is the true value distribution of the point cloud label; is the distribution of point cloud label prediction values; w j is the weight of the point cloud loss value calculation result; L is the number of point clouds of each category in the training set point cloud file.

Images