Road boundary line tracking method and device, electronic equipment and storage medium

Abstract

The application belongs to the technical field of automatic driving, and provides a road boundary line tracking method and device, electronic equipment and a storage medium, the tracking method comprising: acquiring a road boundary key point set in real time; according to the road boundary key point set, a road boundary curve polynomial algorithm based on a random sampling consistent algorithm and a least square method is used to fit a road boundary line observation value, wherein the road boundary key point set of each frame corresponds to different road boundary line observation values; according to the real-time speed of the vehicle, the time difference between the last frame and the current frame, and the road boundary line observation value of the last frame, a Kalman filtering algorithm is used to predict a road boundary line prediction value of the current frame. The real-time speed of the vehicle is introduced, so that the obtained road boundary line prediction value is closer to the actual change model of the road boundary line in the vehicle coordinate system, and the accuracy of the correction result of the finally obtained road boundary line is improved.

Description

Road boundary line tracking methods, devices, electronic equipment and storage media Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to road boundary line tracking methods, devices, electronic equipment, and storage media. Background Technology

[0002] In the field of autonomous driving, the technology of using visual image sensors for lane detection and tracking is relatively mature. However, in determining road boundaries, LiDAR, as a sensor capable of real-time detection of three-dimensional spatial information, can obtain the spatial features of terrain or man-made structures at road boundaries. This gives LiDAR sensors a significant advantage over visual image sensors in detecting and tracking road boundaries. However, compared to using visual image sensors for lane tracking, the point cloud distribution acquired by LiDAR is irregular, and the distribution becomes sparser with distance. This makes it easy for the detection of key points at the road boundary to generate more false detections, resulting in certain errors when using LiDAR to track road boundary lines. Summary of the Invention

[0003] The purpose of this invention is to provide a road boundary line tracking method, apparatus, electronic device, and storage medium to address the problem mentioned in the background art: compared to lane line tracking using visual image sensors, the point cloud distribution acquired by lidar is irregular, and the point cloud distribution becomes sparser with distance. This makes it easy for the detection of key road boundary points to generate a large number of false detection points, resulting in a certain degree of error when using lidar to track road boundary lines.

[0004] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0005] Firstly, a method for tracing road boundary lines is provided, the method comprising:

[0006] Real-time acquisition of key point sets at road boundaries;

[0007] Based on the set of key points of the road boundary, the observation values ​​of the road boundary line are fitted by the random sampling consensus algorithm and the road boundary curve polynomial algorithm based on the least squares method. Each set of key points of the road boundary corresponds to a different observation value of the road boundary line.

[0008] Based on the vehicle's real-time speed, the time difference between the previous and current frames, and the road boundary line observations from the previous frame, the Kalman filter algorithm is used to predict the road boundary line values ​​for the current frame.

[0009] Based on the predicted road boundary line value and the observed road boundary line value of the current frame, the Kalman filter algorithm is used to update and obtain the corrected road boundary line value of the current frame.

[0010] In conjunction with the first aspect, the key points in the road boundary key point set have four attributes: x-coordinate, y-coordinate, z-coordinate, and intensity value.

[0011] In conjunction with the first aspect, the road boundary line observations are fitted using a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on least squares, based on the road boundary key point set. Each frame's road boundary key point set corresponds to different road boundary line observations, including:

[0012] A random sampling consensus algorithm is used to randomly select a preset proportion of key points from the set of key points at the road boundary to form a subset of key points.

[0013] Construct a cubic polynomial model to fit the road boundary line, as shown in the following formula:

[0014] y = c3x 3 +c2x 2 +c1x+c0

[0015] Among them, c0, c1, c2 and c3 are four coefficients that determine the specific result of the road boundary line;

[0016] The least squares method is used, and the specific values ​​of the four coefficients c0, c1, c2 and c3 in the cubic polynomial model are obtained based on the key points in the sub-point set.

[0017] By substituting the specific values ​​of the four coefficients c0, c1, c2, and c3 into the cubic polynomial model, the observed values ​​of the road boundary line are obtained.

[0018] In conjunction with the first aspect, in the step of using a random sampling consensus algorithm to randomly select a preset proportion of key points from the set of key points at the road boundary to form a subset of key points, the formula is defined as follows:

[0019]

[0020] Where |S| is the number of key points in the sub-point set, |M| is the number of key points in the road boundary key point set, r is the preset ratio, and r is a custom parameter.

[0021] In conjunction with the first aspect, the least squares method is used, and based on the key points in the sub-point set, the specific values ​​of the four coefficients c0, c1, c2, and c3 in the cubic polynomial model are obtained, including:

[0022] Construct the Vandermonde matrix as follows:

[0023]

[0024] Where Φ is the Vandermonde matrix, and x represents the x-coordinate of the key point;

[0025] Construct the coefficient vector as follows:

[0026]

[0027] Where c is the coefficient vector;

[0028] Based on the Vandermonde matrix and coefficient vector, the following equation is generated:

[0029] Φ T Φc=Φ T y

[0030] Where y is an m-dimensional vector, represented by the y-coordinates of the key points;

[0031] Based on the key points in the sub-point set, the coefficient vector c in the equation is calculated to obtain the specific values ​​of the four coefficients c0, c1, c2 and c3.

[0032] In conjunction with the first aspect, the method of predicting the predicted road boundary line value for the current frame using a Kalman filter algorithm based on the vehicle's real-time speed, the time difference between the previous and current frames, and the road boundary line observation value from the previous frame includes:

[0033] The state variables for the road boundary line observations are constructed based on the fitted cubic polynomial model of the road boundary line, as follows:

[0034]

[0035] Where s is the state variable, v is the real-time speed of the vehicle, and w is the angular velocity of the vehicle;

[0036] Construct the state transition matrix as follows:

[0037]

[0038] dx 2 =dx*dx

[0039] dx 3 = dx*dx*dx

[0040] Where T is the state transition matrix and dt is the time difference between the previous frame and the current frame;

[0041] The state variables for calculating the predicted road boundary line value in the current frame are given by the following formula:

[0042]

[0043] in, s is the state variable for the predicted road boundary line value in the current frame. k-1 The state variable is the road boundary line observation value from the previous frame;

[0044] From the state variables of the predicted road boundary line value in the current frame, obtain the specific values ​​of the four coefficients c0, c1, c2, and c3 corresponding to the predicted road boundary line value in the current frame;

[0045] By substituting the specific values ​​of the four coefficients c0, c1, c2, and c3 of the cubic polynomial model into the cubic polynomial model, the predicted value of the road boundary line for the current frame is obtained.

[0046] Secondly, a road boundary line tracking device is provided, the tracking device comprising:

[0047] The acquisition module is used to acquire the set of key points of the road boundary in real time;

[0048] The fitting module is used to fit the road boundary line observations based on the road boundary key point set using a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on the least squares method. Each frame of the road boundary key point set corresponds to a different road boundary line observation.

[0049] The prediction module is used to predict the road boundary line value of the current frame based on the vehicle's real-time speed, the time difference between the previous frame and the current frame, and the road boundary line observation value of the previous frame, using the Kalman filter algorithm.

[0050] The update module is used to update the corrected road boundary line of the current frame based on the predicted road boundary line value and the observed road boundary line value of the current frame, using the Kalman filter algorithm.

[0051] Thirdly, an electronic device is provided, the electronic device including a controller and a memory, the memory storing a computer program that, when executed by the controller, causes the electronic device to perform the tracking method as described in any one of the first aspects.

[0052] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is run on a computer, the computer causes the computer to perform the tracking method as described in any one of the first aspects.

[0053] The invention employing the above technical solution has the following advantages:

[0054] The road boundary line tracking method of this application employs a random sampling consensus algorithm and the least squares method during the road boundary line fitting stage. This enables the handling of key point sets with a certain degree of noise, allowing irregular and sparse key points to form a stable curve, thereby improving the accuracy of the final corrected road boundary line. Furthermore, by incorporating the real-time speed of the vehicle, the method makes the obtained predicted road boundary line values ​​closer to the actual road boundary line variation model in the vehicle's coordinate system, further enhancing the accuracy of the final corrected road boundary line. Attached Figure Description

[0055] The present invention can be further illustrated by the non-limiting embodiments given in the accompanying drawings;

[0056] Figure 1 is a flowchart illustrating the road boundary line tracking method in an embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of the road boundary line tracking device in an embodiment of the present invention;

[0058] Among them, 10 is the acquisition module; 20 is the fitting module; 30 is the prediction module; and 40 is the update module. Detailed Implementation

[0059] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In addition, directional terms mentioned in the embodiments, such as "up," "down," "top," "bottom," "left," "right," "front," and "back," are only for reference to the directions in the drawings and are not intended to limit the scope of protection of the present invention.

[0060] This application provides a road boundary line tracking method, mainly applied to vehicle-mounted LiDAR. The LiDAR can acquire key points of the road boundary, forming a road boundary key point set, which enables the tracking of the road boundary line. Specifically, the tracking method may include the following steps:

[0061] S100: Real-time acquisition of key point set of road boundary;

[0062] S200. Based on the set of key points of the road boundary, the observation values ​​of the road boundary line are fitted by the random sampling consensus algorithm and the road boundary curve polynomial algorithm based on the least squares method. Each set of key points of the road boundary corresponds to a different observation value of the road boundary line.

[0063] S300. Based on the vehicle's real-time speed, the time difference between the previous frame and the current frame, and the road boundary line observation value of the previous frame, the Kalman filter algorithm is used to predict the predicted value of the road boundary line in the current frame.

[0064] S400. Based on the predicted road boundary line value and the observed road boundary line value of the current frame, the Kalman filter algorithm is used to update and obtain the corrected road boundary line value of the current frame.

[0065] By employing the above implementation method, the random sampling consensus algorithm and least squares method are used in the road boundary line fitting stage. This can handle key point sets with a certain degree of noise, allowing irregular and sparse key points to form a stable curve, thus improving the accuracy of the final corrected road boundary line. Furthermore, by incorporating the vehicle's real-time speed, the predicted road boundary line value more closely resembles the actual road boundary line variation model in the vehicle's coordinate system, further enhancing the accuracy of the final corrected road boundary line.

[0066] As an optional implementation, the key points in the road boundary key point set have four attributes: x-coordinate, y-coordinate, z-coordinate, and intensity value. It is understood that key points acquired by LiDAR can be presented in three-dimensional coordinates; therefore, key points can have x-coordinate, y-coordinate, and z-coordinate. Furthermore, the intensity value is an inherent attribute of the key point itself. The specific values ​​of the x-coordinate and y-coordinate can be used to construct the road boundary line model.

[0067] As an optional implementation, based on the road boundary key point set, the observed values ​​of the road boundary line are fitted using a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on least squares. Each frame's road boundary key point set corresponds to different observed values ​​of the road boundary line, including:

[0068] S210. Using a random sampling consensus algorithm, a preset proportion of key points are randomly selected from the set of key points at the road boundary to form a subset of key points.

[0069] S220. Construct a cubic polynomial model to fit the road boundary line, as shown in the following formula:

[0070] y = c3x 3 +c2x 2 +c1x+c0

[0071] Among them, c0, c1, c2 and c3 are four coefficients that determine the specific result of the road boundary line;

[0072] S230. Using the least squares method and based on the key points in the sub-point set, the specific values ​​of the four coefficients c0, c1, c2 and c3 in the cubic polynomial model are obtained.

[0073] S240. Substitute the specific values ​​of the four coefficients c0, c1, c2 and c3 into the cubic polynomial model to obtain the observed values ​​of the road boundary line.

[0074] By implementing the above methods, combining the random sampling consensus algorithm with the least squares method, the road boundary line observations of each frame can be fitted as accurately as possible, resulting in more accurate correction results for the final road boundary lines.

[0075] As an optional implementation, in the step of randomly selecting a preset proportion of key points from the set of key points at the road boundary using a random sampling consensus algorithm to form a subset of key points, the formula is defined as follows:

[0076]

[0077] Where |S| is the number of key points in the sub-point set, |M| is the number of key points in the road boundary key point set, r is the preset ratio, and r is a custom parameter.

[0078] As an optional implementation, the least squares method is used, and based on the key points in the sub-point set, the specific values ​​of the four coefficients c0, c1, c2, and c3 in the cubic polynomial model are obtained, including:

[0079] S231. Construct the Vandermonde matrix as follows:

[0080]

[0081] Where Φ is the Vandermonde matrix, and x represents the x-coordinate of the key point;

[0082] S232. Construct the coefficient vector as follows:

[0083]

[0084] Where c is the coefficient vector;

[0085] S233. Based on the Vandermonde matrix and coefficient vector, generate the following equation:

[0086] Φ T Φc=Φ T y

[0087] Where y is an m-dimensional vector, represented by the y-coordinates of the key points;

[0088] S234. Based on the key points in the sub-point set, calculate the coefficient vector c in the equation to obtain the specific values ​​of the four coefficients c0, c1, c2 and c3.

[0089] As an optional implementation, the step of predicting the predicted road boundary line value for the current frame using a Kalman filter algorithm based on the vehicle's real-time speed, the time difference between the previous and current frames, and the road boundary line observation value of the previous frame includes:

[0090] S310. Construct the state variables of the road boundary line observations based on the fitted cubic polynomial model of the road boundary line, as follows:

[0091]

[0092] Where s is the state variable, v is the real-time speed of the vehicle, and w is the angular velocity of the vehicle;

[0093] Construct the state transition matrix as follows:

[0094]

[0095] dx 2 =dx*dx

[0096] dx 3 = dx*dx*dx

[0097] Where T is the state transition matrix and dt is the time difference between the previous frame and the current frame;

[0098] S320. Calculate the state variables for the predicted road boundary line value in the current frame, using the following formula:

[0099]

[0100] in, s is the state variable for the predicted road boundary line value in the current frame. k-1 The state variable is the road boundary line observation value from the previous frame;

[0101] S330. Obtain the specific values ​​of the four coefficients c0, c1, c2 and c3 corresponding to the predicted road boundary line value of the current frame from the state variables of the predicted road boundary line value of the current frame.

[0102] S340. Substitute the specific values ​​of the four coefficients c0, c1, c2 and c3 of the cubic polynomial model into the cubic polynomial model to obtain the predicted value of the road boundary line in the current frame.

[0103] This application embodiment also provides a road boundary line tracking device, which includes an acquisition module 10, a fitting module 20, a prediction module 30, and an update module 40, wherein the functions of each module are as follows:

[0104] Module 10 is used to acquire the set of key points of the road boundary in real time;

[0105] The fitting module 20 is used to fit the road boundary line observations based on the road boundary key point set using a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on the least squares method. Each frame of the road boundary key point set corresponds to a different road boundary line observation.

[0106] The prediction module 30 is used to predict the road boundary line prediction value of the current frame based on the real-time speed of the vehicle, the time difference between the previous frame and the current frame, and the road boundary line observation value of the previous frame, using the Kalman filter algorithm.

[0107] The update module 40 is used to update the corrected road boundary line of the current frame based on the predicted road boundary line value and the observed road boundary line value of the current frame, using the Kalman filter algorithm.

[0108] This application also provides an electronic device, which includes a controller and a memory. The memory stores a computer program, and when the computer program is executed by the controller, the electronic device performs the tracking method described above.

[0109] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the tracing method described above.

[0110] Based on the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, control device, or network device, etc.) to execute the methods described in the various implementation scenarios of this application.

[0111] The road boundary line tracking method, apparatus, electronic device, and storage medium provided by the present invention have been described in detail above. The specific embodiments are provided only to aid in understanding the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for tracing road boundary lines, characterized in that, The tracking method includes: acquiring a set of key points on the road boundary in real time, wherein the key points in the set of key points on the road boundary have... coordinate, The system uses four attributes: coordinate, z-coordinate, and intensity value. Based on the road boundary key point set, it employs a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on least squares to fit the observed road boundary lines. Each frame's road boundary key point set corresponds to a different road boundary line observation value. The process includes: using the random sampling consensus algorithm to randomly select a preset proportion of key points from the road boundary key point set to form a subset; and constructing a cubic polynomial model to fit the road boundary line, as shown in the following formula: ;in, To determine the specific results of the road boundary line, four coefficients are used; the least squares method is employed, and based on the key points in the sub-point set, the coefficients in the cubic polynomial model are calculated. The specific values ​​of these four coefficients; will The specific values ​​of these four coefficients are substituted into a cubic polynomial model to obtain the observed values ​​of the road boundary line. Based on the vehicle's real-time speed, the time difference between the previous and current frames, and the observed values ​​of the road boundary line in the previous frame, a Kalman filter algorithm is used to predict the predicted value of the road boundary line in the current frame. Based on the predicted value of the road boundary line in the current frame and the observed values ​​of the road boundary line in the current frame, a Kalman filter algorithm is used to update the corrected result of the road boundary line in the current frame. The step of using a Kalman filter algorithm to predict the predicted value of the road boundary line in the current frame, based on the vehicle's real-time speed, the time difference between the previous and current frames, and the observed values ​​of the road boundary line in the previous frame, includes: constructing the state variables of the observed road boundary line values ​​based on a cubic polynomial model fitted to the road boundary line, as follows: ;in, For state variables, This is the vehicle's real-time speed. Let be the angular velocity of the vehicle; construct the state transition matrix as follows: ;in, Here is the state transition matrix. Let be the time difference between the previous frame and the current frame; calculate the state variable for the predicted road boundary line value in the current frame using the following formula: ;in, The state variable is the predicted value of the road boundary line in the current frame. The state variable represents the road boundary line observation value from the previous frame; the state variable represents the predicted road boundary line value for the current frame. The specific values ​​of these four coefficients; the cubic polynomial model. The specific values ​​of these four coefficients are substituted into the cubic polynomial model to obtain the predicted value of the road boundary line for the current frame.

2. The road boundary line tracking method according to claim 1, characterized in that, In the step of using a random sampling consensus algorithm to randomly select a preset proportion of key points from the set of key points at the road boundary to form a subset of key points, the formula is defined as follows: ;in, The number of key points in the sub-point set. The number of key points concentrated at the road boundary. For the preset ratio, For custom parameters.

3. The road boundary line tracking method according to claim 2, characterized in that, The least squares method is used, and the cubic polynomial model is obtained based on the key points in the sub-point set. The specific values ​​of these four coefficients include: constructing the Vandermonde matrix, as follows: ;in, For the Vandermonde matrix, Key points Coordinates; construct the coefficient vector as follows: ;in, Let Vandermonde matrix be the coefficient vector. Based on the Vandermonde matrix and the coefficient vector, the following equation is generated: Where y is an m-dimensional vector, represented by the y-coordinates of the keypoints; based on the keypoints in the sub-point set, the coefficient vector in the equation is calculated. to obtain The specific values ​​of these four coefficients.

4. A road boundary line tracking device, characterized in that, The tracking device employs the method described in any one of claims 1 to 3, comprising: an acquisition module for acquiring a set of key points of the road boundary in real time; a fitting module for fitting the observed values ​​of the road boundary line based on the set of key points of the road boundary using a random sampling consensus algorithm and a road boundary curve polynomial algorithm based on least squares, wherein the set of key points of the road boundary in each frame corresponds to different observed values ​​of the road boundary line; a prediction module for predicting the predicted value of the road boundary line of the current frame based on the real-time speed of the vehicle, the time difference between the previous frame and the current frame, and the observed value of the road boundary line of the previous frame using a Kalman filter algorithm; and an update module for updating the corrected result of the road boundary line of the current frame based on the predicted value of the road boundary line of the current frame and the observed value of the road boundary line of the current frame using a Kalman filter algorithm.

5. An electronic device, characterized in that, The electronic device includes a controller and a memory, the memory storing a computer program that, when executed by the controller, causes the electronic device to perform the tracking method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the tracking method as described in any one of claims 1 to 3.

Images