A method, system, vehicle, and storage medium for completing missing lane lines.

Abstract

This invention relates to the field of intelligent driving technology for vehicles, and provides a method, system, vehicle, and storage medium for missing lane line completion. The method includes the following steps: collecting lane information to form a dataset; importing the dataset into a deep learning network model to train the network, enabling the network to predict the future trajectory of a vehicle based on the historical trajectories of surrounding vehicles; fitting the predicted trajectory of the vehicle and the lane line trajectory output by the network; performing regression analysis and matching the fitted curve of the lane line trajectory near the predicted trajectory to obtain a mathematical model of the fitted curve; obtaining the relationship between the fitted curve of the lane line and the vehicle's driving trajectory through the mathematical model; and completing the lane line based on the fitted lane line trajectory and the vehicle's position. This invention can achieve high-precision lane line completion in scenarios where lane lines are missing or blurred.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving of vehicles, in particular to a missing lane line completion method and system, a vehicle and a storage medium. BACKGROUND

[0002] In the field of intelligent driving, the motion control of vehicles needs information about the surrounding scene, and lane lines are relatively important information. On the one hand, when driving a vehicle, a human driver strongly relies on lane lines, and an intelligent driving vehicle, when providing driving assistance functions, is similar to a human driver and may use information about lane lines to provide information for wheel steering, vehicle speed control, etc.; on the other hand, current traffic rules also have corresponding requirements for vehicles to drive in lanes on the road, and intelligent driving vehicles also need to comply with traffic rules.

[0003] There are some methods for determining road edges and lane lines, such as the patent with publication number CN202011547621.3, which discloses a lane line image enhancement and completion method based on a generative adversarial network. This method uses the Pix2Pix network for lane line generation to convert an RGB color lane image with unclear lane lines into an RGB color lane image with lane lines. After obtaining the RGB image with lane lines, difference processing is performed with the original image to obtain a difference image, which is processed by an algorithm threshold to obtain a lane line image with good purity, thereby realizing lane line enhancement and completion. This method uses the strong robustness of the adversarial network to achieve anti-interference ability to external noise, and when deployed on a vehicle, it enhances the enhancement and completion ability of lane lines to a certain extent.

[0004] The patent with publication number CN201710196345.2 discloses a device and method for determining road edges, which includes a radar detector installed on a vehicle that can detect at least a stationary target beside the road edge of the road where the vehicle is located, and a processing component configured to receive the stationary target detected by the radar detector and extract arrangement information of the stationary targets arranged in a relatively regular manner relative to the road, thereby obtaining road edge information based on the arrangement information. This method is suitable for obtaining road edge information in unstructured roads, and can also obtain relatively accurate road edge information at a long distance, but the research object is the road edge.

[0005] Therefore, the above-mentioned prior art can enhance and complete lane line images to a certain extent

[0006] However, if the lane line is missing or the lane line is blurred, the existing method cannot complete the lane line with high accuracy. SUMMARY

[0007] In view of this, the embodiments of the present application aim to provide a missing lane line completion method, system, vehicle and storage medium, which can complete the lane line with high accuracy for the scene of missing lane line or blurred lane line.

[0008] To achieve the above technical purpose, the technical solution adopted by the present application is as follows:

[0009] In a first aspect, the present application provides a missing lane line completion method, comprising the following steps:

[0010] S1, collecting lane information to form a data set, importing the data set into a deep learning network model, training the network so that the network can predict the future trajectory of the vehicle according to the historical trajectory of the vehicle;

[0011] S2, fitting the predicted trajectory of the vehicle and the lane line trajectory output by the network;

[0012] S3, performing regression analysis and matching the fitting curve of the lane line trajectory near the predicted trajectory to obtain a mathematical model of the matching fitting curve, and obtaining the relationship between the lane line and the fitting curve of the vehicle driving trajectory through the mathematical model;

[0013] S4, completing the lane line based on the fitted lane line trajectory and the position of the ego vehicle.

[0014] Further, the deep learning network model is an LSTM encoder-decoder model, the encoder encodes the input information into an intermediate semantic representation, and the decoder generates the current output according to the intermediate semantic and the generated information. The LSTM encoder-decoder model has better effect and can cope with the case that the input sequence and the output sequence are not equal in length.

[0015] Further, the fitting of the predicted trajectory of the vehicle and the lane line trajectory output by the network comprises predicted trajectory fitting and lane line trajectory fitting, the predicted trajectory fitting comprises adopting a cubic fitting method to obtain the sequence of the future trajectory of the vehicle through network prediction, and the lane line trajectory fitting comprises fitting a cubic lane line trajectory y=a`x 3 +b`x 2 +c`x+d` using the collected lane line data.

[0016] Further, the cubic fitting method comprises setting a general cubic polynomial form y=ax 3 +bx 2 +cx+d, and determining four coefficients a, b, c and d by using the least square method.

[0017] Further, the regression analysis further comprises: based on the assumption that the driving track of the vehicle in most scenarios is parallel to the lane line, the distribution relationship and the probability density of the coefficients of the two fitting curves are counted; based on the distribution relationship of the coefficients, a threshold value delta is set, in the interval with a confidence greater than the threshold value, the future track of the vehicle is predicted through the historical track of the vehicle, and the track of the missing lane line is obtained through the predicted future track of the vehicle.

[0018] Further, the training network further comprises formulating evaluation indexes including ADE and FDE, selecting an optimizer and training parameters, and improving network precision.

[0019] In a second aspect, the present application further discloses an automatic driving system, which uses the lane line completion method.

[0020] In a third aspect, the present application further discloses a vehicle, which comprises a vehicle body and the automatic driving system.

[0021] In a fourth aspect, the present application further discloses a computer readable storage medium, which stores a computer program, and when the computer program runs on a computer, the computer executes the method.

[0022] The application with the above technical solution has the following advantages:

[0023] 1. The application first imports a data set into an LSTM encoder-decoder model, trains a network, so that the network can predict a future track of a vehicle according to a historical track of the vehicle, then uses a mathematical method to perform regression analysis, fits a vehicle curve, and then combines the premise that a human driver's driving path is strongly dependent on a lane line, and combines a missing lane line to complete the lane line; the whole process can realize lane line completion with higher precision in the scene of missing lane lines or blurred lane lines.

[0024] 2. The application has strong portability, and after training and fitting, it can be directly deployed to a vehicle end for application without the need for further training. DETAILED DESCRIPTION

[0025] The application can be further illustrated by the non-limiting embodiments shown in the accompanying drawings. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be considered as a limitation on the scope, and other related drawings can be obtained by those skilled in the art without creative labor.

[0026] Figure 1This is a flowchart of a method for completing missing lane lines according to the present invention.

[0027] Figure 2 This is a schematic diagram illustrating the specific implementation steps of a method for completing missing lane lines according to the present invention. Detailed Implementation

[0028] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] Example 1

[0030] This embodiment is a method for completing missing lane lines, including the following steps:

[0031] First, lane information is collected using a test vehicle to form a dataset. This dataset is then imported into an LSTMencoder-decoder model to train the network. This enables the network to predict future trajectories based on the historical trajectories of vehicles. Furthermore, relevant evaluation metrics, including but not limited to ADE and FDE, are established, and appropriate optimizers and training parameters are selected to ensure that the network meets the required accuracy.

[0032] Then, the data containing lane lines will be extracted, the predicted vehicle trajectory and the lane line trajectory will be fitted, regression analysis will be performed, a mathematical model matching the fitted curves of the two will be obtained, and then merged with the network model above.

[0033] The specific method consists of four steps:

[0034] 1. Predicted Trajectory Fitting. A sequence of future vehicle trajectories can be obtained through network prediction. To match common lane line fitting methods, this embodiment uses the same cubic fitting method. First, a general cubic polynomial form y = ax² is established. 3 +bx 2 +cx+d, and then use the least squares method to determine the four coefficients a, b, c, and d.

[0035] 2. Lane Trajectory Fitting. This step is similar to step 1. Using the collected lane line data, a cubic lane trajectory is fitted, y = a`x. 3 +b`x 2 +c`x+d`.

[0036] 3. Regression analysis of vehicle predicted trajectory and lane line trajectory. Based on the assumption that the vehicle's trajectory is parallel to the lane line in most scenarios, the distribution relationship and probability density of the coefficients of the two fitted curves are statistically analyzed.

[0037] 4. Predict the trajectory of the missing lane. Based on the coefficient distribution relationship obtained in step 3, a threshold δ is manually set. In the interval where the confidence level is greater than the threshold, the future trajectory of the vehicle is predicted by the vehicle's historical trajectory. The trajectory of the missing lane is then obtained by using the predicted future trajectory of the vehicle.

[0038] Finally, based on the fitted lane line trajectory and the vehicle's position, the lane lines are completed.

[0039] This embodiment can achieve lane line completion with higher accuracy in scenarios where lane lines are missing or blurred.

[0040] Example 2

[0041] This embodiment describes an autonomous driving system that utilizes a missing lane line completion method from Embodiment 1 described above. Such an autonomous driving system is highly portable; once training and fitting are complete, it can be directly deployed to the vehicle for application without requiring further training.

[0042] Example 3

[0043] This embodiment is a vehicle, which includes a vehicle body and the autonomous driving system of Embodiment 2, and the autonomous driving system is mounted on the vehicle.

[0044] Example 4

[0045] This embodiment is a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to execute the method described in Embodiment 1 above. Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented in hardware or by using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to cause a computer device (such as a personal computer, braking device, or network device, etc.) to execute the methods described in the various implementation scenarios of this application.

[0046] In the embodiments provided in this application, it should be understood that the disclosed apparatus, systems, and methods can also be implemented in other ways. The apparatus, systems, and methods embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0047] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for completing missing lane lines, characterized by, The method comprises the following steps: S1, collecting lane information to form a data set, importing the data set into a deep learning network model, and training the network so that the network can predict the future trajectory of the vehicle according to the historical trajectory of the vehicle; S2, fitting the predicted trajectory of the vehicle and the lane line trajectory output by the network; the fitting of the predicted trajectory of the vehicle and the lane line trajectory output by the network comprises: predicted trajectory fitting and lane line trajectory fitting, the predicted trajectory fitting comprises: using a cubic fitting method to obtain a sequence of future trajectories of the vehicle through network prediction; the lane line trajectory fitting comprises: using the collected lane line data to fit a cubic lane line trajectory y=a`x³+b`x²+c`x+d`; The cubic fitting method of the predicted trajectory of the vehicle comprises: setting a general cubic polynomial form y=ax³+bx²+cx+d, and then determining the four coefficients a, b, c and d by using the least square method; S3, regression analysis is performed to obtain a mathematical model of the matching fitting curve, and the relationship between the lane line and the fitting curve of the vehicle trajectory is obtained through the mathematical model; the regression analysis comprises: based on the assumption that the driving trajectory of the vehicle in most scenarios is parallel to the lane line, the distribution relationship and the probability density of the coefficients of the two fitting curves are counted; based on the distribution relationship of the coefficients, a threshold δ is set, in the interval with a confidence greater than the threshold, the future trajectory of the vehicle is predicted through the historical trajectory of the vehicle, and the trajectory of the missing lane line is obtained through the predicted future trajectory of the vehicle; S4, based on the fitted lane line trajectory, the lane line is completed in combination with the position of the ego vehicle.

2. The method of claim 1, wherein: The deep learning network model is an LSTM encoder-decoder model, the encoder encodes the input information into an intermediate semantic representation, and the decoder generates the current output according to the intermediate semantic and the generated information.

3. The method of claim 2, wherein: The training network further comprises formulating evaluation indexes including ADE and FDE, selecting an optimizer and training parameters, and improving the precision of the network.

4. An autonomous driving system, characterized by: The automatic driving system uses the missing lane line completion method of any one of claims 1-3.

5. A vehicle characterized by: The vehicle comprises a vehicle body and the automatic driving system of claim 4, and the automatic driving system is mounted on the vehicle body.

6. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and when the computer program runs on the computer, the computer executes the missing lane line completion method of any one of claims 1-3.

Images