A method for extracting road marking lines in a lidar point cloud

Abstract

The application discloses a kind of methods for extracting road marking line in laser radar point cloud, comprising the following steps: S1, propose M-Transformer network, the original point cloud of road marking line is processed, the context information of original point cloud and the long-term dependence between different road marking lines are obtained;S2, based on parallel pooling attention mechanism, the feature relationship of the road marking line is aggregated;S3, the position dependence of all scales is integrated, the minimum value and average value of feature sequence are calculated, and they are connected to obtain the two-dimensional feature of road marking line point cloud data;S4, the feature weight of the two-dimensional feature is obtained using Softmax function, further enhance the semantic feature of road marking line point cloud data, obtain accurate road marking line distribution.The present application can quickly and accurately extract the road marking line of large-scale urban road scene.

Description

Technical Field

[0001] This invention relates to the fields of urban data science, digital twins, and intelligent transportation technologies, specifically a method for extracting road marking lines from lidar point clouds. Background Technology

[0002] As a crucial component of new intelligent transportation systems and urban digital twins, road markings require periodic monitoring and maintenance. Intelligent monitoring effectively ensures the accuracy of intelligent urban road inspections. Traditional road marking monitoring methods primarily rely on manual field surveying and digital photogrammetry. Manual methods involve assigning road maintenance personnel to manually inspect road surface conditions to ensure the absence of traffic accident hazards. However, this method is significantly affected by factors such as high workload and safety risks. Digital photogrammetry utilizes drone imagery and surveillance camera images to acquire urban road information; however, limitations imposed by factors such as rain, snow, object obstruction, and image resolution result in the extracted road marking accuracy not meeting the requirements of intelligent traffic network inspections.

[0003] As a rapidly developing high-tech surveying method, laser scanning technology can quickly and accurately collect three-dimensional information of large-scale urban road scenes by taking advantage of its characteristics such as fast data acquisition speed, low update cycle, high data accuracy, and active non-contact measurement. It has significant advantages in extracting road marking lines in complex scenes, and at the same time provides necessary data support for real-time monitoring of road health.

[0004] However, efficiently and accurately extracting diverse road marking lines from high-density, unordered, massive laser scanning point cloud data remains a significant challenge. Currently common road marking line extraction methods include morphological analysis and deep learning. However, these methods are limited by several factors: (1) the diversity and complexity of road marking line types and shapes; (2) interference and occlusion of road marking lines by other road surface objects; and (3) the limited quantity of high-quality road marking line point cloud data. These methods cannot yet meet the requirements for road marking line extraction in complex urban road scenarios. Therefore, there is an urgent need to develop a stable, accurate, and robust road marking line extraction method to overcome the shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting road marking lines from lidar point clouds, so as to solve the problems in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for extracting road marking lines from a lidar point cloud, comprising the following steps:

[0007] S1. Propose the M-Transformer network to process the original point cloud of road marking lines, and obtain the context information of the original point cloud and the long-term dependencies between different road marking lines.

[0008] S2. Based on the parallel pooling attention mechanism, the feature relationships of the road marking lines are aggregated;

[0009] S3. Integrate the location dependencies of all scales, calculate the minimum and average values ​​of the feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data;

[0010] S4. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

[0011] Preferably, step S1 specifically includes the following steps:

[0012] S11. Perform multiple linear transformations on the original point cloud data to obtain the query matrix, key matrix, and value matrix of the point cloud features, respectively.

[0013] S12. Based on the query vector and key vector output by S11, perform a dot product operation to obtain the energy function;

[0014] S13. Use the Softmax function to normalize the energy function, calculate the vector value weights, and obtain the deep features of the point cloud.

[0015] S14. Perform sequential operations on deep point cloud features using convolution, batch normalization, and ReLU activation functions to learn the attention features of point cloud data.

[0016] S15. The attention features of the above point cloud data are fused to obtain the context information and long-term dependencies of the road marking line point cloud data.

[0017] Preferably, step S2 specifically includes the following steps:

[0018] S21. Using max pooling and average pooling operations, the context information and long-term dependencies of the road mark line point cloud data obtained in S15 are used to perform feature aggregation.

[0019] S22. The aggregated features are fed into the Softmax function to obtain feature weights, and then the deep-level aggregated features of the road marking line point cloud data are obtained.

[0020] Preferably, step S3 specifically includes the following steps:

[0021] S31. Based on the deeper features of the point cloud data obtained in S22, calculate the minimum and average values ​​of its feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data.

[0022] Preferably, step S4 specifically includes the following steps:

[0023] S41. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

[0024] Compared with existing technologies, the advantages of this invention are: it can quickly and accurately extract road marking lines in large-scale urban road scenes; by utilizing a method based on feature extraction from raw point cloud data, point deformer neural networks, and point cloud feature pooling aggregation, this invention achieves accurate and stable extraction of road marking lines even in complex urban road scenes; combined with relevant road marking line design and construction standards, it effectively reduces the limitations of disordered arrangement of laser scanning point cloud data, occlusion by other objects, and uneven distribution of density and reflectance values, making the road marking line extraction results more robust and efficient; by developing a method based on data preprocessing and deep learning, this invention effectively extracts the inherent features of road marking line point cloud data, improves the efficiency of large-scale point cloud data processing, and greatly improves the effectiveness of intelligent operation and maintenance of traffic networks and the safety of urban road networks. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0026] Figure 1 This is a schematic diagram of the process of the present invention;

[0027] Figure 2 This is a schematic diagram of the road marking line extraction results of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1-2 In this embodiment of the invention, a method for extracting road marking lines from a lidar point cloud includes the following steps:

[0030] S1. Propose the M-Transformer network to process the original point cloud of road marking lines, and obtain the context information of the original point cloud and the long-term dependencies between different road marking lines.

[0031] S2. Based on the parallel pooling attention mechanism, the feature relationships of the road marking lines are aggregated;

[0032] S3. Integrate the location dependencies of all scales, calculate the minimum and average values ​​of the feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data;

[0033] S4. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

[0034] S1 specifically includes the following steps:

[0035] S11. Perform multiple linear transformations on the original point cloud data to obtain the query matrix, key matrix, and value matrix of the point cloud features, respectively.

[0036] S12. Based on the query vector and key vector output by S11, perform a dot product operation to obtain the energy function;

[0037] S13. Use the Softmax function to normalize the energy function, calculate the vector value weights, and obtain the deep features of the point cloud.

[0038] S14. Perform sequential operations on deep point cloud features using convolution, batch normalization, and ReLU activation functions to learn the attention features of point cloud data.

[0039] S15. The attention features of the above point cloud data are fused to obtain the context information and long-term dependencies of the road marking line point cloud data.

[0040] S2 specifically includes the following steps:

[0041] S21. Using max pooling and average pooling operations, the context information and long-term dependencies of the road mark line point cloud data obtained in S15 are used to perform feature aggregation.

[0042] S22. The aggregated features are fed into the Softmax function to obtain feature weights, and then the deep-level aggregated features of the road marking line point cloud data are obtained.

[0043] S3 specifically includes the following steps:

[0044] S31. Based on the deeper features of the point cloud data obtained in S22, calculate the minimum and average values ​​of its feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data.

[0045] S4 specifically includes the following steps:

[0046] S41. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

[0047] Specifically:

[0048] S1. Propose the M-Transformer network to process the original point cloud of road marking lines, and obtain the context information of the original point cloud and the long-term dependencies between different road marking lines.

[0049] S2. Based on the parallel pooling attention mechanism, the feature relationships of the road marking lines are aggregated;

[0050] S3. Integrate the location dependencies of all scales, calculate the minimum and average values ​​of the feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data;

[0051] S4. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

[0052] Step S1 specifically includes the following sub-steps:

[0053] S11. Perform feature encoding on the original point cloud data to obtain high-dimensional point cloud features. as follows:

[0054]

[0055] in The initial features of the road marking line point cloud data are represented by N, where N represents the number of points in the input point cloud, and d represents the feature dimension of each point. This represents the point feature encoding operation. Next, the features... After performing multiple linear transformations, the query matrix, key matrix, and value matrix of the point cloud features are obtained as follows:

[0056]

[0057] in It represents a linear transformation.

[0058] S12. Based on the above output query vector Q (t) and bond vector K (t) Performing a dot product operation, we obtain the energy function as follows:

[0059]

[0060] in Φ represents the energy function. dot This represents the dot product operation.

[0061] S13. Using the Softmax function to convert the energy function After normalization and calculation of vector weights, the deep features of the point cloud are obtained, as shown below:

[0062]

[0063]

[0064] Where S t For attention score, These are the weights of the value vector. Further, the value vector V... (t) Its weight Multiplying yields the deep features of the point cloud. as follows:

[0065]

[0066] S14. Apply convolution, batch normalization, and ReLU activation function to the above-mentioned deep point cloud features. Perform sequential operations to obtain the attention features of the point cloud data. The method is as follows:

[0067]

[0068] Where Conv, BN, and ReLU represent convolution operation, batch normalization, and ReLU activation function, respectively.

[0069] S15. Fuse the attention features of the above point cloud data to obtain the contextual information and long-term dependencies of the road marking line point cloud data. The method is as follows:

[0070]

[0071] Where Concat represents the aggregation operation.

[0072] Step S2 specifically includes the following sub-steps:

[0073] S21. Extract the context information and long-term dependencies of the road marking point cloud data obtained in S15. Using max pooling and average pooling operations, the method is as follows:

[0074]

[0075]

[0076] in and They represent The result after max pooling and average pooling operations. and Perform feature aggregation to obtain aggregated features. The method is as follows:

[0077]

[0078] S22, Aggregate the features The data is fed into the Softmax function to obtain feature weights, which are then compared with the contextual information and long-term dependencies of the road mark point cloud data. Multiplication yields deep aggregation features of the road marking line point cloud data. The method is as follows:

[0079]

[0080] Step S3 specifically includes the following sub-steps:

[0081] S31. Deep aggregation features of point cloud data obtained from S22 Calculate the minimum and average values ​​of its feature sequences, and concatenate them to obtain the two-dimensional features of the road marking line point cloud data. The method is as follows:

[0082]

[0083] Among them G min and G mean These represent the minimum and average value functions for calculating the characteristic sequence, respectively.

[0084] Step S4 specifically includes the following sub-steps:

[0085] S41. Obtain the two-dimensional features based on the Softmax function. The feature weights are then used to deeply aggregate the features of the road mark point cloud data. Multiply to output the predicted features of road markings. The method is as follows:

[0086]

[0087] The final result is the extraction of road marking lines.

[0088] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for extracting road marking lines from lidar point clouds, characterized in that: Includes the following steps: S1. Propose the M-Transformer network to process the original point cloud of road marking lines, and obtain the context information of the original point cloud and the long-term dependencies between different road marking lines. S2. Based on the parallel pooling attention mechanism, the feature relationships of the road marking lines are aggregated; S3. Integrate the location dependencies of all scales, calculate the minimum and average values ​​of the feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data; S4. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution; S1 specifically includes the following steps: S11. Perform multiple linear transformations on the original point cloud data to obtain the query matrix, key matrix, and value matrix of the point cloud features, respectively. S12. Based on the query vector and key vector output by S11, perform a dot product operation to obtain the energy function; S13. Use the Softmax function to normalize the energy function, calculate the vector value weights, and obtain the deep features of the point cloud. S14. Perform sequential operations on deep point cloud features using convolution, batch normalization, and ReLU activation functions to learn the attention features of point cloud data. S15. Fuse the attention features of the above point cloud data to obtain the contextual information and long-term dependencies of the road marking line point cloud data. The method is as follows: ； in This indicates an aggregation operation.

2. The method for extracting road marking lines from a lidar point cloud according to claim 1, characterized in that: S2 specifically includes the following steps: S21. Using max pooling and average pooling operations, the context information and long-term dependencies of the road mark line point cloud data obtained in S15 are used to perform feature aggregation. S22. The aggregated features are fed into the Softmax function to obtain feature weights, and then the deep-level aggregated features of the road marking line point cloud data are obtained.

3. The method for extracting road marking lines from a lidar point cloud according to claim 2, characterized in that: S3 specifically includes the following steps: S31. Based on the deeper features of the point cloud data obtained in S22, calculate the minimum and average values ​​of its feature sequences, and connect them to obtain the two-dimensional features of the road marking line point cloud data.

4. The method for extracting road marking lines from a lidar point cloud according to claim 3, characterized in that: S4 specifically includes the following steps: S41. Use the Softmax function to obtain the feature weights of the two-dimensional features, further enhance the semantic features of the road marking line point cloud data, and obtain an accurate road marking line distribution.

Images