Road marking identification method based on vehicle-mounted LiDAR point cloud

Abstract

The invention relates to a road marking identification method based on a vehicle-mounted LiDAR point cloud, and belongs to the technical field of intelligent traffic. Comprising the following steps: constructing a road marking point cloud data set; training a road marking point cloud recognition model, wherein the road marking point cloud recognition model is constructed based on a RandLA-Net model, and the RandLA-Net model is improved; and outputting a road marking point cloud recognition result. The method can be directly applied to a large-scale road scene, keeps complete space geometric information, does not need to depend on an empirical threshold, and greatly improves the adaptability and automation level of the model in different road scenes.

Description

Technical Field

[0001] The invention relates to a road marking recognition method based on vehicle-mounted LiDAR point cloud, belonging to the technical field of intelligent transportation. Background Art

[0002] Road markings provide road guidance information for pedestrians, vehicle drivers and other traffic participants, and are a core component of high-precision maps required for autonomous driving technology. Therefore, studying how to accurately, efficiently and completely identify road marking information from vehicle-mounted LiDAR point clouds is crucial to practical needs such as building high-precision maps.

[0003] At present, there are two main methods for identifying road marking point clouds: (1) Projecting the vehicle-mounted laser point cloud onto a two-dimensional plane to convert it into a feature image, extracting road marking information using methods such as threshold segmentation, semantic segmentation, and neural networks, and then optimizing it through methods such as morphology, and finally using template matching for classification. This type of method is relatively simple and easy to implement, and has a good extraction and classification effect when the road scene is relatively simple. However, the road environment is usually complex and diverse, and it is difficult to identify obscured or worn road marking point clouds. In addition, there are many problems such as multiple empirical thresholds that are difficult to determine, and projection and back-projection loss of accuracy. (2) Obtaining directly from the road point cloud. ① Extracting road marking point clouds using multi-threshold segmentation. This type of method faces the problem of scale selection when dividing the point cloud data into blocks, and when the intensity difference between the road markings and road points is small, it is easy to cause missed or incorrect extraction of the markings. ② Extracting road marking point clouds using the scanning line method. This type of method has a high degree of automation, but the extraction accuracy is easily affected by noise. ③ Extracting road marking point clouds using machine learning methods. This type of method requires large-scale point clouds for training, and cannot guarantee both segmentation accuracy and extraction efficiency.

[0004] In addition, many methods only consider the intensity information of point clouds and lack the effective use of semantic information, which limits the accuracy and robustness of recognition. Therefore, there is an urgent need for a vehicle-mounted LiDAR point cloud road marking recognition method with better recognition effect, higher degree of automation and higher segmentation accuracy. Summary of the invention

[0005] In order to solve the above problems, the present invention discloses a road marking recognition method based on vehicle-mounted LiDAR point cloud, and its specific technical solution is as follows: A road marking recognition method based on vehicle-mounted LiDAR point cloud comprises the following steps: Step 1: Construct a road marking point cloud dataset; Step 2: Train the road marking point cloud recognition model: The road marking point cloud recognition model is built based on the RandLA-Net model, which is a lightweight point cloud semantic segmentation network with an encoding-decoding structure. It contains 4 encoding layers and 4 decoding layers. The corresponding encoding layers and decoding layers are stacked with features through jump connections. In order to improve the recognition accuracy of the road marking point cloud recognition model in the road marking point cloud, the following improvements are made to the RandLA-Net model: Step 2.1: Introduce angle information into the local space encoding module: By calculating the normalized direction vector of the relative coordinates and their dot product relationship, the inverse cosine function is used to obtain the angle value, and the relative distance between points is combined to construct a feature representation that integrates space and geometric relationships; Step 2.2: Global feature enhancement of the attention pooling module: extract global feature information by averaging the local features and concatenate it with the local features as output; Step 2.3: Design a contrast enhancement module: Calculate the maximum value of the joint features output by the attention pooling module and average , through the maximum value and average Subtraction to obtain contrast features , the maximum value and contrast features The joint features are concatenated and the softmax function is used to obtain their feature weights. The joint features are weighted to obtain the final feature expression with contrast enhancement, and then concatenated with the original joint features to obtain the final feature. , input into the shared multi-layer perceptron for aggregation; Step 2.4: Fusion of multiple loss functions: Introduce the boundary loss function and fuse it with the weighted cross entropy loss function to enhance the model's perception of the boundary area; Step 3: Output the road marking point cloud recognition results.

[0006] Furthermore, the step 1 is specifically as follows: Based on the open source point cloud dataset Toronto_3D and the road scene area measured at the South Campus of the Polytechnic University of Madrid, Spain, road markings were divided and two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the drivable surface of the vehicle; "nature" includes a variety of environmental features, including trees, vertical barriers of fences, some low-rise and multi-story buildings, vehicles, and street lights; "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", and "parking line" categories are specific road markings that convey key navigation information to drivers and autonomous driving systems.

[0007] Furthermore, the step 2.1 is specifically as follows: Road markings are more regular than roads and nature, and usually present certain directionality and linear structure. Therefore, the introduction of angle information helps to further identify the direction and relative position of the markings. By calculating the relative coordinates, relative distance and angle relationship between the midpoint and the neighboring points in the point cloud data, a richer local feature representation is generated. The specific steps are as follows: ①Calculate relative coordinates : , in, , Yes and Point The three-dimensional coordinates of ②Replace the relative coordinates Normalized to the direction vector, we get the unit vector , , , in, for 2-norm of ; ③ Calculate the angle between two unit vectors using the inverse cosine function , , in, For point and Point The unit vector between ; ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is: , in, is the feature after the relative distance and angle information are stitched together. It is the feature after being processed by the activation function ReLu.

[0008] Furthermore, the step 2.2 is specifically as follows: The attention pooling module in the RandLA-Net model only considers local context information and ignores global features. Therefore, the global features are represented by averaging the local features and concatenated with the local features as output. The expression is as follows: , in, It is a global feature; It is a local feature; is the joint feature after splicing.

[0009] Furthermore, the step 2.3 is specifically as follows: Road markings are usually affected by shadows, occlusions, or complex environments with similar backgrounds, and their local and global features may be affected. Therefore, by designing a contrast enhancement module, the recognition ability of the model at different scales and in different situations is effectively improved. Specifically, by combining the maximum value in the feature sequence , the model can better focus on key features; however, when the marking line is blocked or partially damaged, relying solely on the maximum value cannot fully express the full picture of the marking line; in order to compensate for this deficiency, the contrast feature is introduced , contrast features By calculating the maximum value of the joint feature sequence and average The difference between the maximum and the maximum value can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect. and contrast features Perform weighted processing and concatenate with the original joint features to obtain the final features , the model obtains a more robust and accurate line feature expression based on the fusion of more information, and the expression is as follows: , in, is the maximum value and contrast features Features after splicing; It is the contrast enhancement feature after weighted summation.

[0010] Furthermore, the step 2.4 is specifically as follows: The marking area is usually small and is greatly disturbed by the environment, making it more difficult for the model to perceive and recognize the boundary area. The weighted cross entropy loss function can effectively solve the problem of category imbalance by assigning different weights to different categories. The boundary loss function Dice Loss can focus on the boundary area of ​​the marking, especially when the marking is partially blocked or damaged, which can help the model restore the boundary shape of the marking more accurately. Therefore, by combining Dice Loss with weighted cross entropy loss, the advantages of both are fully considered during the training process. On the one hand, weighted cross entropy ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, Dice Loss focuses on strengthening the model's ability to recognize the boundaries of road markings, especially when the road markings are partially missing or disturbed; through the design of the fusion loss function, the model can find a balance between the overall classification performance and the fine recognition ability of the boundary area, and ultimately improve the robustness and accuracy of road marking recognition. The expression of the fusion loss function is: , in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Indicates category The weight of is the weight factor.

[0011] Furthermore, the step 3 is specifically as follows: after multi-layer processing and feature fusion, the network finally outputs the classification result of each point and the corresponding prediction probability. The classification result constitutes the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

[0012] Furthermore, the method further includes an evaluation step for the road marking point cloud recognition model, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), average intersection over union (mIoU), and overall accuracy (OA), the calculation formula of which is as follows: , in, is the total number of semantic categories, is a semantic category, is the true proportion, is a false ratio, is a false negative example, is the total number of point clouds.

[0013] The beneficial effects of the present invention are: 1. The present invention can be directly applied to large-scale road scenes, retaining complete spatial geometric information without relying on empirical thresholds, greatly improving the adaptability and automation level of the model in different road scenes. In particular, by introducing angle information through the local spatial encoding module, the model can effectively improve the recognition ability of road marking point clouds when facing changes in marking direction and irregular shapes.

[0014] 2. The present invention significantly improves the performance of the model in complex scenes through the design of global feature enhancement and contrast enhancement modules. In particular, the contrast enhancement module calculates the difference between the maximum and average values ​​of the joint features, constructs contrast features, and combines local and global context information weighted processing to further enhance the recognition ability of occluded, worn or partially missing markings.

[0015] 3. The present invention introduces a fusion strategy of Dice Loss and weighted cross entropy loss function to make the model more sensitive to the boundary area of ​​the marking line, which can effectively reduce the misjudgment and missed judgment problems in the boundary area and improve the overall recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the present invention; Figure 2 This is the semantic information graph of the validation set of the road marking point cloud dataset of the present invention. in: Figure 2 (a) is the semantic information of the validation set of Toronto_rdmk. Figure 2 (b) is the semantic information of the validation set of UPM_rdmk; Figure 3 This is a diagram of the improved attention pooling module of the present invention; Figure 4 Diagram of the contrast enhancement module designed for the present invention. DETAILED DESCRIPTION

[0017] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0018] Combined with Figure 1 It can be seen that the main steps of the present invention are: Step 1: Construct a road marking point cloud dataset; Step 2: Train the road marking point cloud recognition model; Step 3: Output the road marking point cloud recognition results.

[0019] Combination Figure 2-4 , specifically describe the specific process and core improvements of the present invention: The road marking recognition method based on vehicle-mounted LiDAR point cloud includes the following steps: Step 1: Construct a road marking point cloud dataset; The road marking point cloud dataset is divided into areas based on the open source point cloud dataset Toronto_3D and the measured road scene areas of the South Campus of the Polytechnic University of Madrid, Spain. (1) The open source data is collected from the urban street scenes of Toronto, Canada. The road is about 1 km long and 16 m wide, including information about buildings, trees, and street lights. The number of laser points is 78.3 million. (2) The measured data is collected using the Trimble MX50 vehicle-mounted mobile measurement system. The scanner of this system has an effective measurement frequency of up to 960 kHz, a ranging accuracy of 2 mm, and a scanning speed of 240 times / second. The obtained road is 1.5 km long and 26 m wide, including information about vehicles, street lights, and low vegetation. The number of laser points is 61.67 million.

[0020] Based on the open source point cloud dataset Toronto_3D and the measured road scene area of ​​the South Campus of the Polytechnic University of Madrid, Spain, two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Figure 2 Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the main drivable surface of the vehicle; "nature" includes a variety of environmental features, including vertical barriers such as trees and fences, some low-rise and multi-story buildings, vehicles, street lights, etc.; "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", "parking space line" and other categories are specific road markings that convey key navigation information to drivers and autonomous driving systems.

[0021] Step 2: Train the road marking point cloud recognition model; Based on the existing point cloud semantic segmentation network RandLA-Net model, structural improvements suitable for road marking recognition are made.

[0022] (1) Road markings are more regular than roads and nature, and usually present certain directionality and linear structure. Therefore, introducing angle information helps to further identify the direction and relative position of the markings. By calculating the relative coordinates, relative distance, and angle relationship between the midpoint and the neighboring points in the point cloud data, a richer local feature representation can be generated. The specific steps are as follows: ①Calculate relative coordinates : , in, , Yes and Point The three-dimensional coordinates of ②Replace the relative coordinates Normalized to the direction vector, we get the unit vector , , , in, for 2-norm of ; ③ Calculate the angle between two unit vectors using the inverse cosine function , , in, For point and Point The unit vector between ; ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is: , in, is the feature after the relative distance and angle information are stitched together. It is the feature after being processed by the activation function ReLu.

[0023] (2) The attention pooling module in the RandLA-Net model only considers local context information and ignores global features. Therefore, the global features are represented by averaging the local features and concatenated with the local features as the output, as shown in Figure 3 The expression is as follows: , in, It is a global feature; It is a local feature; is the joint feature after splicing.

[0024] (3) Road markings are usually affected by complex environments (such as shadows, occlusions, or similar backgrounds), and their local and global features may be affected. Therefore, by designing a contrast enhancement module (such as Figure 4 As shown in Figure 2), it can effectively improve the recognition ability of the model at different scales and in different situations. Specifically, by combining the maximum value in the feature sequence , the model can better focus on key features. However, relying solely on the maximum value may not fully express the full picture of the reticle, especially when the reticle is blocked or partially damaged. To compensate for this deficiency, the contrast feature is introduced , which can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect by calculating the difference between the maximum value and the average value of the joint feature sequence. Finally, the final feature is obtained by weighting the two and concatenating them with the original joint feature. , the model can obtain a more robust and accurate line feature expression based on the integration of more information. The expression is as follows: , in, is the maximum value and contrast features Features after splicing; It is the contrast enhancement feature after weighted summation.

[0025] (4) The marking area is usually small and is more susceptible to environmental interference, which makes it more difficult for the model to perceive and recognize the boundary area. The weighted cross entropy loss function can effectively solve the problem of category imbalance by assigning different weights to different categories; while the boundary loss function Dice Loss can focus on the boundary area of ​​the marking, especially when the marking is partially blocked or damaged, which can help the model restore the boundary shape of the marking more accurately.

[0026] Therefore, by combining Dice Loss with weighted cross entropy loss, the advantages of both are fully considered during the training process. On the one hand, weighted cross entropy ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, Dice Loss focuses on strengthening the model's ability to recognize the boundaries of road markings, especially when the road markings are partially missing or disturbed. Through the design of this fusion loss function, the model can find a balance between the overall classification performance and the fine recognition ability of the boundary area, and ultimately improve the robustness and accuracy of road marking recognition. The expression of the fusion loss function is: , in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Indicates category The weight of is the weight factor.

[0027] Step 3: Output the road marking point cloud recognition result; After multi-layer processing and feature fusion, the network finally outputs the classification result of each point and the corresponding prediction probability. These results constitute the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

[0028] This embodiment also includes an evaluation step for the road marking point cloud recognition model, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), mean intersection over union (mIoU) and overall accuracy (OA), which are calculated as follows: , in, is the total number of semantic categories, is a semantic category, is the true proportion, is a false ratio, is a false negative example, is the total number of point clouds.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0030] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A road marking recognition method based on vehicle-mounted LiDAR point cloud, characterized in that: The steps include: Step 1: Construct a road marking point cloud dataset; Step 2: Train the road marking point cloud recognition model: The road marking point cloud recognition model is built based on the RandLA-Net model, which is a lightweight point cloud semantic segmentation network with an encoding-decoding structure. It contains 4 encoding layers and 4 decoding layers. The corresponding encoding layers and decoding layers are stacked with features through jump connections. In order to improve the recognition accuracy of the road marking point cloud recognition model in the road marking point cloud, the following improvements are made to the RandLA-Net model: Step 2.1: Introduce angle information into the local space encoding module: By calculating the normalized direction vector of the relative coordinates and their dot product relationship, the inverse cosine function is used to obtain the angle value, and the relative distance between points is combined to construct a feature representation that integrates space and geometric relationships; Step 2.2: Global feature enhancement of the attention pooling module: extract global feature information by averaging the local features and concatenate it with the local features as output; Step 2.3: Design a contrast enhancement module: Calculate the maximum value of the joint features output by the attention pooling module and average , through the maximum value and average Subtraction to obtain contrast features , the maximum value and contrast features Concatenate and use the softmax function to obtain its feature weights. The joint features are weighted to obtain the final feature expression with contrast enhancement, and then concatenated with the original joint features to obtain the final feature , input into the shared multi-layer perceptron for aggregation; Step 2.4: Fusion of multiple loss functions: Introduce the boundary loss function and fuse it with the weighted cross entropy loss function to enhance the model's perception of the boundary area; Step 3: Output the road marking point cloud recognition results.

2. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The step 1 is specifically as follows: Based on the open source point cloud dataset Toronto_3D and the road scene area measured at the South Campus of the Polytechnic University of Madrid, Spain, road markings were divided and two road marking point cloud datasets, Toronto_rdmk and UPM_rdmk, were constructed. Different colors represent different semantic categories in the dataset. The part marked as "road" refers to the drivable surface of the vehicle; "nature" includes a variety of environmental features, including trees, vertical barriers of fences, some low-rise and multi-story buildings, vehicles, and street lights; "arrow", "long solid line", "short solid line", "zebra crossing", "stop line", "guide line", "no parking line", and "parking line" categories are specific road markings that convey key navigation information to drivers and autonomous driving systems.

3. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The step 2.1 is specifically as follows: Introducing angle information helps to further identify the direction and relative position of the marking line. By calculating the relative coordinates, relative distance, and angle relationship between the midpoint and the neighboring points in the point cloud data, a richer local feature representation is generated. The specific steps are as follows: ①Calculate relative coordinates : , in, , Yes and Point The three-dimensional coordinates of ②Replace the relative coordinates Normalized to the direction vector, we get the unit vector , ， ， in, for 2-norm of ; ③ Calculate the angle between two unit vectors using the inverse cosine function , ， in, For point and Point The unit vector between ; ④Calculate relative distance , and concatenate it with the angle information, and then use the activation function ReLu to process the features to further improve the expressiveness of the features. The expression is: ， in, is the feature after the relative distance and angle information are stitched together. It is the feature after being processed by the activation function ReLu.

4. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 3 is characterized in that: The step 2.2 is specifically as follows: The global features are represented by averaging the local features and concatenated with the local features as the output. The expression is as follows: ， in, It is a global feature; It is a local feature; is the joint feature after splicing.

5. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 4, characterized in that: The step 2.3 is specifically as follows: By designing a contrast enhancement module, the recognition ability of the model at different scales and in different situations is effectively improved, that is, by combining the maximum value in the feature sequence , the model focuses on key features; however, when the marking line is blocked or partially damaged, relying solely on the maximum value cannot fully express the full picture of the marking line; in order to compensate for this deficiency, the contrast feature is introduced , contrast features By calculating the maximum value of the joint feature sequence and average The difference between the maximum and the maximum value can enhance the perception of these local changes and help the model identify the parts of the marking line that are not easy to detect. and contrast features Perform weighted processing and concatenate with the original joint features to obtain the final features , the model obtains a more robust and accurate line feature expression based on the fusion of more information, and the expression is as follows: ， in, is the maximum value and contrast features Features after splicing; It is the contrast enhancement feature after weighted summation.

6. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 5, characterized in that: The step 2.4 is specifically as follows: The marking area is small and affected by the environment, making it more difficult for the model to perceive and recognize the boundary area. The weighted cross entropy loss function solves the problem of category imbalance by assigning different weights to different categories. The boundary loss function can focus on the boundary area of ​​the marking, especially when the marking is partially blocked or damaged, which can help the model restore the boundary shape of the marking more accurately. Therefore, by combining the boundary loss function with the weighted cross entropy loss function, the advantages of both are fully considered during the training process. On the one hand, the weighted cross entropy loss function ensures that the model can correctly identify the labels of each point, especially the road marking category; on the other hand, the boundary loss function Dice Loss focuses on strengthening the model's ability to recognize the boundary of the road marking, especially when the road marking is partially missing or disturbed; through the design of the fusion loss function, it is possible to find a balance between the overall classification performance and the fine recognition ability of the boundary area, and ultimately improve the robustness and accuracy of road marking recognition. The expression of the fusion loss function is: ， in, is the fusion loss function, is the boundary loss function, is the weighted cross entropy loss function, It is The true category of each point; It is The predicted probability of a point, is a smoothing constant to avoid the denominator being 0; is the number of points in the point cloud; is the total number of categories; Indicates category The weight of is the weight factor.

7. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The step 3 is specifically as follows: after multi-layer processing and feature fusion, the network finally outputs the classification result of each point and the corresponding prediction probability. The classification result constitutes the final road marking point cloud recognition output, providing accurate road marking information for subsequent autonomous driving and map construction applications.

8. The road marking recognition method based on vehicle-mounted LiDAR point cloud according to claim 1, characterized in that: The method also includes an evaluation step for the road marking point cloud recognition model, including the following quantitative accuracy evaluation indicators: single-class intersection over union (IoU), average intersection over union (mIoU), and overall accuracy (OA), which are calculated as follows: ， in, is the total number of semantic categories, is a semantic category, is the true proportion, is a false ratio, is a false negative example, is the total number of point clouds.

Images