A road marking recognition method based on vehicle-mounted LiDAR point cloud

Abstract

The application relates to a road marking line recognition method based on a vehicle-mounted LiDAR point cloud, and belongs to the technical field of intelligent transportation. The method comprises the following steps: constructing a road marking line point cloud data set; training a road marking line point cloud recognition model; the road marking line point cloud recognition model is constructed based on a RandLA-Net model and is improved on the RandLA-Net model; and outputting a road marking line point cloud recognition result. The application can be directly applied to a large-scale road scene, retains complete spatial geometric information, does not need to depend on an experience threshold, and greatly improves the adaptability and automation level of the model in different road scenes.

Description

Technical Field

[0001] The present invention relates to a road marking recognition method based on vehicle-mounted LiDAR point cloud, belonging to the technical field of intelligent transportation. Background Art

[0002] Road markings provide guidance for pedestrians, drivers, and other traffic participants, and are a core component of the high-precision maps required for autonomous driving. Therefore, research on how to accurately, efficiently, and completely identify road markings from onboard LiDAR point clouds is crucial for building high-precision maps.

[0003] At present, there are two main methods for identifying road marking point clouds: (1) Projecting the vehicle-mounted laser point cloud onto a two-dimensional plane to convert it into a feature image, extracting road marking information using methods such as threshold segmentation, semantic segmentation, and neural networks, then optimizing it through methods such as morphology, and finally using template matching for classification. This type of method is relatively simple and easy to implement, and has a good extraction and classification effect when the road scene is relatively simple. However, the road environment is usually complex and diverse, and it is difficult to identify obscured or worn road marking point clouds. In addition, there are many problems such as multiple and difficult to determine empirical thresholds, and loss of accuracy in projection and back-projection. (2) Directly obtain from road point clouds. ① Extracting road marking point clouds using multi-threshold segmentation. This type of method faces the problem of scale selection when segmenting point cloud data, and when the intensity difference between road markings and road points is small, it is easy to cause missed or incorrect extraction of road markings. ② Extracting road marking point clouds using the scanning line method. This type of method has a high degree of automation, but the extraction accuracy is easily affected by noise. ③ Extracting road marking point clouds using machine learning methods. This type of method requires large-scale point clouds for training and cannot guarantee both segmentation accuracy and extraction efficiency.

[0004] Furthermore, many methods only consider the intensity information of point clouds and lack the effective use of semantic information, which limits the accuracy and robustness of recognition. Therefore, there is an urgent need for a vehicle-mounted LiDAR point cloud road marking recognition method with better recognition performance, higher degree of automation and higher segmentation accuracy. Summary of the Invention

[0005] In order to solve the above problems, the present invention discloses a road marking recognition method based on vehicle-mounted LiDAR point cloud, and its specific technical solution is as follows:

[0006] A road marking recognition method based on vehicle-mounted LiDAR point cloud includes the following steps:

[0007] Step 1: Construct a road marking point cloud dataset;

[0008] Step 2: Train the road marking point cloud recognition model: The road marking point cloud recognition model is built based on the RandLA-Net model, a lightweight point cloud semantic segmentation network with an encoder-decoder structure consisting of four encoding layers and four decoding layers. The corresponding encoding and decoding layers stack features via skip connections. To improve the recognition accuracy of the road marking point cloud recognition model in road marking point clouds, the following improvements were made to the RandLA-Net model:

[0009] Step 2.1: Introduce angle information into the local space encoding module: By calculating the normalized direction vector of the relative coordinates and their dot product relationship, the inverse cosine function is used to obtain the angle value, and the relative distance between points is combined to construct a feature representation that integrates spatial and geometric relationships;

[0010] Step 2.2: Global feature enhancement of the attention pooling module: extract global feature information by averaging the local features and concatenate it with the local features as output;

[0011] Step 2.3: Design a contrast enhancement module: Calculate the maximum value of the joint features output by the attention pooling module and average , through the maximum value and average Subtraction to obtain contrast features , the maximum value and contrasting features Splice and use the softmax function to obtain its feature weights, perform weighted processing on the joint features, obtain the final feature expression with contrast enhancement, and then splice it with the original joint features to obtain the final feature , input into the shared multi-layer perceptron for aggregation;

[0012] Step 2.4: Multi-loss function fusion: Introduce the boundary loss function and fuse it with the weighted cross entropy loss function to enhance the model's perception of boundary areas;

[0013] Step 3: Output the road marking point cloud recognition results.

[0014] Furthermore, the step 1 is specifically as follows:

[0015] Based on the open source point cloud dataset Toronto_3D and the road scene area measured at the South Campus of the Polytechnic University of Madrid, Spain, road markings were divided and two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the drivable surface of the vehicle; "nature" includes a variety of environmental features, including trees, vertical barriers of fences, some low-rise and multi-story buildings, vehicles, and street lights; "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", and "parking space line" categories are specific road markings that convey key navigation information to drivers and autonomous driving systems.

[0016] Furthermore, the step 2.1 is specifically as follows:

[0017] Road markings are more regular than roads and nature, usually showing a certain directionality and linear structure. Therefore, introducing angle information helps to further identify the direction and relative position of the markings. By calculating the relative coordinates, relative distance, and angle relationship between the midpoint and neighboring points in the point cloud data, a richer local feature representation is generated. The specific steps are as follows:

[0018] ①Calculate relative coordinates : ,

[0019] in, , Yes and point The three-dimensional coordinates of

[0020] ②The relative coordinates Normalized to the direction vector, we get the unit vector ,

[0021] ,

[0022] ,

[0023] in, for 2-norm of ;

[0024] ③Calculate the angle between two unit vectors using the inverse cosine function ,

[0025] ,

[0026] in, for point and point The unit vector between ;

[0027] ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is:

[0028] ,

[0029] in, is the feature after the relative distance and angle information are spliced ​​together. It is the feature after being processed by the activation function ReLu.

[0030] Furthermore, the step 2.2 is specifically as follows:

[0031] The attention pooling module in the RandLA-Net model only considers local context information and ignores global features. Therefore, the global features are represented by averaging the local features and concatenated with the local features as the output. The expression is as follows:

[0032] ,

[0033] in, is a global feature; It is a local feature; is the joint feature after splicing.

[0034] Furthermore, the step 2.3 is specifically as follows:

[0035] Road markings are often affected by shadows, occlusions, or complex environments with similar backgrounds, and their local and global features may be affected. Therefore, by designing a contrast enhancement module, the recognition ability of the model at different scales and in different situations is effectively improved. Specifically, by combining the maximum value in the feature sequence , the model can better focus on key features; however, when the marking line is blocked or partially damaged, relying solely on the maximum value cannot fully express the full picture of the marking line; in order to make up for this deficiency, the contrast feature is introduced , contrast features By calculating the maximum value of the joint feature sequence and average The difference between the two can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect; finally, by and contrasting features Perform weighted processing and concatenate with the original joint features to obtain the final features , the model obtains a more robust and accurate line marking feature expression based on the integration of more information, which is expressed as follows:

[0036] ,

[0037] in, is the maximum value and contrasting features Features after splicing; It is the contrast enhancement feature after weighted summation.

[0038] Furthermore, the step 2.4 is specifically as follows:

[0039] Marking areas are usually small and subject to significant environmental interference, making it more difficult for the model to perceive and recognize boundary areas. The weighted cross entropy loss function can effectively address the class imbalance problem by assigning different weights to different categories. The boundary loss function, Dice Loss, can focus on the boundary areas of the markings, especially when the markings are partially occluded or damaged, helping the model to more accurately restore the boundary shape of the markings.

[0040] Therefore, by combining Dice Loss with weighted cross entropy loss, the advantages of both are fully considered during the training process. On the one hand, weighted cross entropy ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, Dice Loss focuses on strengthening the model's ability to recognize road marking boundaries, especially when the road markings are partially missing or disturbed. By designing a fusion loss function, the model can find a balance between overall classification performance and the ability to finely recognize boundary areas, ultimately improving the robustness and accuracy of road marking recognition. The expression of the fusion loss function is:

[0041] ,

[0042] in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Representation category The weight of is the weight factor.

[0043] Furthermore, step 3 is specifically as follows: after multi-layer processing and feature fusion, the network finally outputs the classification result and corresponding prediction probability of each point. The classification result constitutes the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

[0044] Furthermore, the method further includes an evaluation step for the road marking point cloud recognition model, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), mean intersection over union (mIoU), and overall accuracy (OA), which are calculated as follows:

[0045] ,

[0046] in, is the total number of semantic categories, is a semantic category, is the true proportion, For false proportions, is a false negative example, is the total number of point clouds.

[0047] The beneficial effects of the present invention are:

[0048] 1. This invention can be directly applied to large-scale road scenes, preserving complete spatial geometric information without relying on empirical thresholds. This significantly improves the model's adaptability and automation across diverse road scenarios. In particular, by introducing angular information through the local spatial encoding module, the model effectively improves its ability to recognize road marking point clouds, even when faced with directional variations and irregular shapes.

[0049] 2. This invention significantly improves the model's performance in complex scenes through the design of global feature enhancement and contrast enhancement modules. In particular, the contrast enhancement module constructs contrast features by calculating the difference between the maximum and average values ​​of joint features. This weighted processing, combined with local and global contextual information, further enhances the recognition of occluded, worn, or partially missing markings.

[0050] 3. The present invention introduces a fusion strategy of Dice Loss and weighted cross entropy loss function, which makes the model more sensitive to the boundary area of ​​the marking line, can effectively reduce the misjudgment and missed judgment problems in the boundary area, and improves the overall recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the present invention;

[0052] Figure 2 This is the semantic information graph of the validation set of the road marking point cloud dataset of the present invention.

[0053] in: Figure 2(a) is the semantic information of the validation set of Toronto_rdmk,

[0054] Figure 2 (b) is the semantic information of the validation set of UPM_rdmk;

[0055] Figure 3 This is the improved attention pooling module diagram of the present invention;

[0056] Figure 4 Diagram of the contrast enhancement module designed for the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0058] Combined with attachment Figure 1 It can be seen that the main steps of the present invention are:

[0059] Step 1: Construct a road marking point cloud dataset;

[0060] Step 2: Train the road marking point cloud recognition model;

[0061] Step 3: Output the road marking point cloud recognition results.

[0062] Combine Figure 2-4 , specifically describing the specific process and core improvements of the present invention:

[0063] The road marking recognition method based on vehicle-mounted LiDAR point cloud includes the following steps:

[0064] Step 1: Construct a road marking point cloud dataset;

[0065] The road marking point cloud dataset is divided based on the open source point cloud dataset Toronto_3D and the measured road scene area of ​​the South Campus of the Polytechnic University of Madrid, Spain. (1) The open source data was collected from the urban street scene of Toronto, Canada. The road is about 1 km long and 16 m wide, including information about buildings, trees, and street lights. The number of laser points is 78.3 million. (2) The measured data was collected using the Trimble MX50 vehicle-mounted mobile measurement system. The scanner of this system has an effective measurement frequency of up to 960 kHz, a ranging accuracy of 2 mm, and a scanning speed of 240 times / second. The obtained road is 1.5 km long and 26 m wide, including information about vehicles, street lights, and low vegetation. The number of laser points is 61.67 million.

[0066] Based on the open source point cloud dataset Toronto_3D and the measured road scene area of ​​the South Campus of the Polytechnic University of Madrid, Spain, two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Figure 2 Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the main drivable surface for vehicles; "nature" includes a variety of environmental features, including vertical barriers such as trees and fences, some low-rise and multi-story buildings, vehicles, street lights, etc.; categories such as "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", and "parking space line" are specific road markings that convey key navigation information to drivers and autonomous driving systems.

[0067] Step 2: Train the road marking point cloud recognition model;

[0068] Based on the existing point cloud semantic segmentation network RandLA-Net model, structural improvements suitable for road marking recognition are made.

[0069] (1) Road markings are more regular than roads and nature, and usually show certain directionality and linear structure. Therefore, introducing angle information helps to further identify the direction and relative position of the markings. By calculating the relative coordinates, relative distances, and angle relationships between the midpoint and neighboring points in the point cloud data, a richer local feature representation can be generated. The specific steps are as follows:

[0070] ①Calculate relative coordinates : ,

[0071] in, , Yes and point The three-dimensional coordinates of

[0072] ②The relative coordinates Normalized to the direction vector, we get the unit vector ,

[0073] ,

[0074] ,

[0075] in, for 2-norm of ;

[0076] ③Calculate the angle between two unit vectors using the inverse cosine function ,

[0077] ,

[0078] in, for point and point The unit vector between ;

[0079] ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is:

[0080] ,

[0081] in, is the feature after the relative distance and angle information are spliced ​​together. It is the feature after being processed by the activation function ReLu.

[0082] (2) The attention pooling module in the RandLA-Net model only considers local context information and ignores global features. Therefore, the global features are represented by averaging the local features and concatenated with the local features as the output, as shown in Figure 3 The expression is as follows:

[0083] ,

[0084] in, is a global feature; It is a local feature; is the joint feature after splicing.

[0085] (3) Road markings are often affected by the interference of complex environments (such as shadows, occlusions or similar backgrounds), and their local and global features may be affected. Therefore, by designing a contrast enhancement module (such as Figure 4 As shown in Figure 2), it can effectively improve the recognition ability of the model at different scales and in different situations. Specifically, by combining the maximum value in the feature sequence , the model can better focus on key features. However, relying solely on the maximum value may not fully express the full picture of the marking, especially when the marking is blocked or partially damaged. To compensate for this deficiency, the contrast feature is introduced By calculating the difference between the maximum and average values ​​of the joint feature sequence, it can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect. Finally, the final feature is obtained by weighting the two and concatenating them with the original joint feature. , the model can obtain a more robust and accurate line feature expression based on the integration of more information. The expression is as follows:

[0086] ,

[0087] in, is the maximum value and contrasting features Features after splicing; It is the contrast enhancement feature after weighted summation.

[0088] (4) The marking area is usually small and is more susceptible to environmental interference, which makes it more difficult for the model to perceive and recognize the boundary area. The weighted cross entropy loss function can effectively solve the problem of category imbalance by assigning different weights to different categories; while the boundary loss function Dice Loss can focus on the boundary area of ​​the marking, especially when the marking is partially blocked or damaged, which can help the model more accurately restore the boundary shape of the marking.

[0089] Therefore, by combining Dice Loss with weighted cross entropy loss, the advantages of both are fully considered during the training process. On the one hand, weighted cross entropy ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, Dice Loss focuses on strengthening the model's ability to recognize road marking boundaries, especially when the markings are partially missing or disturbed. Through the design of this fusion loss function, the model can find a balance between overall classification performance and the ability to fine-tune the boundary area, ultimately improving the robustness and accuracy of road marking recognition. The expression of the fusion loss function is:

[0090] ,

[0091] in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Representation category The weight of is the weight factor.

[0092] Step 3: Output road marking point cloud recognition results;

[0093] After multiple layers of processing and feature fusion, the network ultimately outputs the classification result and corresponding predicted probability for each point. These results constitute the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

[0094] This embodiment also includes an evaluation step for the road marking point cloud recognition model, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), mean intersection over union (mIoU), and overall accuracy (OA), which are calculated as follows:

[0095] ,

[0096] in, is the total number of semantic categories, is a semantic category, is the true proportion, For false proportions, is a false negative example, is the total number of point clouds.

[0097] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such, will not be interpreted in an idealized or overly formal sense.

[0098] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A road marking recognition method based on vehicle-mounted LiDAR point cloud, characterized in that: The steps include: Step 1: Construct a road marking point cloud dataset; Step 2: Train the road marking point cloud recognition model: The road marking point cloud recognition model is built based on the RandLA-Net model, a lightweight point cloud semantic segmentation network with an encoder-decoder structure consisting of four encoding layers and four decoding layers. The corresponding encoding and decoding layers stack features via skip connections. To improve the recognition accuracy of the road marking point cloud recognition model in road marking point clouds, the following improvements were made to the RandLA-Net model: Step 2.1: Introduce angle information into the local space encoding module: By calculating the normalized direction vector of the relative coordinates and their dot product relationship, the inverse cosine function is used to obtain the angle value, and the relative distance between points is combined to construct a feature representation that integrates spatial and geometric relationships; Step 2.2: Global feature enhancement of the attention pooling module: extract global feature information by averaging the local features and concatenate it with the local features as the output; Step 2.3: Design a contrast enhancement module: Calculate the maximum value of the joint features output by the attention pooling module and average , through the maximum value and average Subtraction to obtain contrast features , the maximum value and contrasting features Splice and use the softmax function to obtain its feature weights. The joint features are weighted to obtain the final feature expression with contrast enhancement, and then spliced ​​with the original joint features to obtain the final feature , input into the shared multi-layer perceptron for aggregation; Step 2.4: Multi-loss function fusion: Introduce the boundary loss function and fuse it with the weighted cross entropy loss function to enhance the model's perception of boundary areas; Step 3: Output the road marking point cloud recognition results.

2. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The step 1 is specifically as follows: Based on the open source point cloud dataset Toronto_3D and road scene areas measured at the South Campus of the Polytechnic University of Madrid, Spain, road markings were divided and two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the drivable surface of the vehicle; "nature" includes a variety of environmental features, including trees, vertical barriers of fences, some low-rise and multi-story buildings, vehicles, and street lights; "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", and "parking space line" categories are specific road markings that convey key navigation information to drivers and autonomous driving systems.

3. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The step 2.1 is specifically as follows: Introducing angle information helps further identify the direction and relative position of the markings. By calculating the relative coordinates, relative distance, and angle relationship between the midpoint and neighboring points in the point cloud data, a richer local feature representation is generated. The specific steps are as follows: ①Calculate relative coordinates : , in, , Yes and point The three-dimensional coordinates of ②The relative coordinates Normalized to the direction vector, we get the unit vector , ， ， in, for 2-norm of ; ③Calculate the angle between two unit vectors using the inverse cosine function , ， in, for point and point The unit vector between ; ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is: ， in, is the feature after the relative distance and angle information are spliced ​​together. It is the feature after being processed by the activation function ReLu.

4. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 3, characterized in that: The step 2.2 is specifically as follows: The global features are represented by averaging the local features and concatenated with the local features as the output. The expression is as follows: ， in, is a global feature; It is a local feature; is the joint feature after splicing.

5. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 4, characterized in that: The step 2.3 is specifically as follows: By designing a contrast enhancement module, the recognition ability of the model at different scales and in different situations is effectively improved, that is, by combining the maximum value in the feature sequence , the model focuses on key features; however, when the marking line is blocked or partially damaged, relying solely on the maximum value cannot fully express the full picture of the marking line; in order to make up for this deficiency, the contrast feature is introduced , contrast features By calculating the maximum value of the joint feature sequence and average The difference between the two can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect; finally, by and contrasting features Perform weighted processing and concatenate with the original joint features to obtain the final features , the model obtains a more robust and accurate line marking feature expression based on the integration of more information, which is expressed as follows: ， in, is the maximum value and contrasting features Features after splicing; It is the contrast enhancement feature after weighted summation.

6. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 5, characterized in that: The step 2.4 is specifically as follows: The marking area is small and subject to environmental interference, making it more difficult for the model to perceive and recognize the boundary area. The weighted cross entropy loss function addresses the class imbalance problem by assigning different weights to different categories. The boundary loss function can focus on the boundary area of ​​the marking, especially when the marking is partially obscured or damaged, and can help the model more accurately restore the boundary shape of the marking. Therefore, by combining the boundary loss function with the weighted cross entropy loss function, the advantages of both are fully considered during the training process. On the one hand, the weighted cross entropy loss function ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, the boundary loss function Dice Loss focuses on strengthening the model's ability to recognize the boundaries of road markings, especially when the road markings are partially missing or disturbed. By designing the fusion loss function, a balance can be found between the overall classification performance and the fine recognition ability of the boundary area, ultimately improving the robustness and accuracy of road marking recognition. The expression of the fusion loss function is: ， in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Representation category The weight of is the weight factor.

7. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: Specifically, step 3 is as follows: After multi-layer processing and feature fusion, the network finally outputs the classification result and corresponding prediction probability of each point. The classification results constitute the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

8. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The evaluation step of the road marking point cloud recognition model is also included, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), mean intersection over union (mIoU), and overall accuracy (OA), which are calculated as follows: ， in, is the total number of semantic categories, is a semantic category, is the true proportion, For false proportions, is a false negative example, is the total number of point clouds.

Images