Road lane line prediction method and device, vehicle and storage medium

Abstract

This application relates to a method, device, vehicle, and storage medium for predicting road lane lines. The method includes: collecting the current vehicle's location information and lane line information; determining a target boundary point based on the location and lane line information, and predicting the trajectory of a first lane line within the target boundary point using a preset Kalman filter model, and predicting the trajectory of a second lane line outside the target boundary point using a preset data-driven neural network model; using the first lane line trajectory as a reference, identifying when the deviation between the curvature of the trajectory points of the target boundary point and the second lane line trajectory and the curvature of the trajectory points of the first lane line trajectory exceeds a preset threshold, correcting the trajectory points of the second lane line trajectory, and obtaining the final predicted lane line based on the corrected second lane line trajectory and the first lane line trajectory. Therefore, by using different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a method, device, vehicle, and storage medium for predicting road lane lines. Background Technology

[0002] In the field of intelligent driving, vehicle motion control requires information about the surrounding environment, with lane markings being a crucial element. However, on older roads, lane markings are often severely worn, making them indistinct or even broken. When providing driver assistance functions, intelligent driving vehicles rely on lane marking information for lateral and longitudinal control. Furthermore, lane markings also provide constraints for the formulation of related strategies to ensure compliance with traffic regulations.

[0003] One proposed technique is a multi-lane line tracking method based on a Kalman filter bank. This method detects lane lines using images and then uses a Kalman filter model to output the optimal value for lane line estimation. The estimation is considered valid when the same lane line is detected in more than a certain number N images. Another approach considers left and right lane line constraints, tracks the width of the current lane, uses discrete points as observations to avoid the influence of outliers on the fitting results, and determines the coefficients through dynamic weight fitting.

[0004] However, the first method has the drawback that the value of N is difficult to determine, relies on experience, and ignores the relationship between lane lines. The second method also has certain disadvantages in multi-lane scenarios. Summary of the Invention

[0005] This application provides a road lane line prediction method, device, vehicle, and storage medium to solve the problems of related technologies ignoring the relationship between lane lines, relying too much on experience, and being unsuitable for multi-lane scenarios. By using different processing methods for lane lines in different areas, it saves computing resources and improves algorithm efficiency.

[0006] The first aspect of this application provides a method for predicting road lane lines, comprising the following steps: collecting the current location information of a vehicle and lane line information; determining a target boundary point based on the location information and the lane line information, and predicting the trajectory of a first lane line within the target boundary point using a preset Kalman filter model, and predicting the trajectory of a second lane line outside the target boundary point using a preset data-driven neural network model, wherein the preset data-driven neural network model is trained by a long short-term memory neural network and a preset Encoder-Decoder model; using the first lane line trajectory as a reference, identifying when the deviation between the curvature of the trajectory points of the target boundary point and the second lane line trajectory and the curvature of the trajectory points of the first lane line trajectory is greater than a preset threshold, correcting the trajectory points of the second lane line trajectory, and obtaining the final predicted lane line based on the corrected second lane line trajectory and the first lane line trajectory.

[0007] Based on the above-mentioned technical means, this application can solve the problems of related technologies ignoring the relationship between lane lines, relying too much on experience, and not being applicable to multi-lane scenarios. By adopting different processing methods for lane lines in different areas, it saves computing resources and improves algorithm efficiency.

[0008] Optionally, in some embodiments, predicting the trajectory of the first lane line within the target boundary point using a preset Kalman filter model includes: acquiring target lane line information that meets preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficients and coefficient change rate of the lane line; initializing the state vector of the preset Kalman filter model using the cubic curve coefficients and coefficient change rate of the lane line to obtain an initialized state vector; iteratively updating the initialized state vector and covariance matrix according to the lane line observations, observation matrix, and observation noise matrix until a preset iteration condition is met to obtain the trajectory of the first lane line within the target boundary point, wherein the initial value of the covariance matrix is ​​an identity matrix.

[0009] Based on the aforementioned technical means, this application can achieve high-precision processing of lane lines on old road surfaces through a data-driven approach combined with Kalman filtering.

[0010] Optionally, in some embodiments, predicting the trajectory of the second lane outside the target boundary point using a preset data-driven neural network model includes: acquiring historical lane trajectories and inputting the historical lane trajectories into the encoder of the preset Encoder-Decoder model to compress them into an intermediate quantity of a preset dimension; based on a preset weight sequence, predicting the trajectory of the second lane outside the target boundary point at the Decoder end of the preset Encoder-Decoder model according to the intermediate quantity and the output of the previous time step.

[0011] Based on the aforementioned technical means, this application can achieve high-precision processing of lane lines on old road surfaces through a data-driven approach combined with Kalman filtering.

[0012] Optionally, in some embodiments, determining the target boundary point based on the location information and the lane line information includes: determining the visible road segment of the lane line based on the lane line information; in the visible road segment of the lane line, using the location information as a base point, selecting a preset number of points from the Kalman-predicted lane line trajectory and the actual trajectory at a preset sampling frequency, and calculating a loss function based on the preset number of points; drawing a change curve based on the preset number of points and the loss function, and taking the number of points corresponding to the lowest point of the loss function as the target boundary point.

[0013] Based on the above technical means, this application can use the dataset collected from real vehicle roads to train the network, and formulate corresponding rules at the boundary points to constrain it, so that the network can output reasonable lane line information.

[0014] Optionally, in some embodiments, the loss function is:

[0015]

[0016] Where, x i To predict the lane line trajectory point, x j denoted as the actual trajectory point, and n as the number of points corresponding to the point with the lowest loss.

[0017] Based on the above technical means, this application can use a Kalman filter model to output the optimal value for lane line estimation.

[0018] A second aspect of this application provides a road lane line prediction device, comprising: a data acquisition module for acquiring the current location information of a vehicle and lane line information; a prediction module for determining a target boundary point based on the location information and the lane line information, predicting a first lane line trajectory within the target boundary point using a preset Kalman filter model, and predicting a second lane line trajectory outside the target boundary point using a preset data-driven neural network model, wherein the preset data-driven neural network model is trained by a long short-term memory neural network and a preset Encoder-Decoder model; and a correction module for, based on the first lane line trajectory, identifying when the deviation between the curvature of the trajectory points of the target boundary point and the second lane line trajectory and the curvature of the trajectory points of the first lane line trajectory is greater than a preset threshold, correcting the trajectory points of the second lane line trajectory, and obtaining the final predicted lane line based on the corrected second lane line trajectory and the first lane line trajectory.

[0019] Optionally, in some embodiments, the prediction module is further configured to: acquire target lane line information that meets preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficients and coefficient change rate of the lane line; initialize the state vector of the preset Kalman filter model using the cubic curve coefficients and coefficient change rate of the lane line to obtain an initialized state vector; iteratively update the initialized state vector and covariance matrix according to the lane line observations, observation matrix and observation noise matrix until the preset iteration conditions are met to obtain the first lane line trajectory within the target boundary point, wherein the initial value of the covariance matrix is ​​an identity matrix.

[0020] Optionally, in some embodiments, the correction module is further configured to: acquire historical lane line trajectories and input the historical lane line trajectories into the encoder of the preset Encoder-Decoder model to compress them into an intermediate quantity of a preset dimension; based on a preset weight sequence, predict the second lane line trajectory outside the target boundary point at the Decoder end of the preset Encoder-Decoder model according to the intermediate quantity and the output of the previous time step.

[0021] Optionally, in some embodiments, determining the target boundary point based on the location information and the lane line information includes: determining the visible road segment of the lane line based on the lane line information; in the visible road segment of the lane line, using the location information as a base point, selecting a preset number of points from the Kalman-predicted lane line trajectory and the actual trajectory at a preset sampling frequency, and calculating a loss function based on the preset number of points; drawing a change curve based on the preset number of points and the loss function, and taking the number of points corresponding to the lowest point of the loss function as the target boundary point.

[0022] Optionally, in some embodiments, the loss function is:

[0023]

[0024] Where, x i To predict the lane line trajectory point, x j denoted as the actual trajectory point, and n as the number of points corresponding to the point with the lowest loss.

[0025] A third aspect of this application provides a vehicle including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the road lane line prediction method as described in the above embodiments.

[0026] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the road lane line prediction method as described in the above embodiments.

[0027] Therefore, by collecting data from real-vehicle roads and defining boundaries, Kalman filtering is applied to lane lines in the foreground, while a data-driven method is used to predict lane lines in the background. Constraints are set at connection points to determine the rationality of the trajectory. This solves the problems of related technologies ignoring the relationships between lane lines, relying heavily on experience, and being unsuitable for multi-lane scenarios. By using different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

[0028] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0029] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0030] Figure 1 This is a flowchart of a road lane line prediction method provided according to an embodiment of this application;

[0031] Figure 2 A schematic diagram of an Encoder-Decoder framework in a network model is provided according to a specific embodiment of this application;

[0032] Figure 3 This is a flowchart of a road lane line prediction method according to a specific embodiment of this application;

[0033] Figure 4 This is a block diagram of a road lane prediction device provided according to an embodiment of this application;

[0034] Figure 5 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application.

[0035] Explanation of reference numerals in the attached drawings: 10-Road lane line prediction device, 100-Acquisition module, 200-Prediction module, and 300-Correction module. Detailed Implementation

[0036] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0037] The following description, with reference to the accompanying drawings, outlines a road lane prediction method, apparatus, vehicle, and storage medium according to embodiments of this application. Addressing the issues mentioned in the background art, such as neglecting the relationships between lane lines, relying heavily on experience, and being unsuitable for multi-lane scenarios, this application provides a road lane prediction method. In this method, a dataset is collected from real-vehicle road data, boundaries are defined, Kalman filtering is applied to nearby lane lines, and a data-driven method is used to predict distant lane lines. Constraints are set at connection points to determine the rationality of the trajectory. This solves the problems of related technologies neglecting the relationships between lane lines, relying heavily on experience, and being unsuitable for multi-lane scenarios. By employing different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

[0038] Specifically, Figure 1 This is a flowchart illustrating a road lane line prediction method provided in an embodiment of this application.

[0039] like Figure 1 As shown, the road lane line prediction method includes the following steps:

[0040] In step S101, the current location information and lane line information of the vehicle are collected.

[0041] Specifically, in this embodiment of the application, a camera can be used to collect the current location information of the vehicle and lane line information.

[0042] In step S102, the target boundary point is determined based on the location information and lane line information. The trajectory of the first lane line within the target boundary point is predicted by a preset Kalman filter model, and the trajectory of the second lane line outside the target boundary point is predicted by a preset data-driven neural network model. The preset data-driven neural network model is trained by a long short-term memory neural network and a preset Encoder-Decoder model.

[0043] Specifically, in this embodiment, based on the ability to obtain lane line information of the vehicle's location, reasonable evaluation indicators are specified, boundaries are defined, and the prediction of the nearby lane line, i.e. the trajectory of the first lane, is performed using the Kalman filter method, while the prediction of the slightly farther lane line, i.e. the trajectory of the second lane line, is performed using a data-driven method. By using different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

[0044] Optionally, in some embodiments, determining the target boundary point based on location information and lane line information includes: determining the visible road segment of the lane line based on the lane line information; in the visible road segment of the lane line, using the location information as a base point, selecting a preset number of points from the Kalman-predicted lane line trajectory and the actual trajectory according to a preset sampling frequency, and calculating a loss function based on the preset number of points; drawing a change curve based on the preset number of points and the loss function, and taking the number of points corresponding to the lowest point of the loss function as the target boundary point.

[0045] Specifically, this application embodiment uses road data collected from actual vehicles to formulate indicators and determine boundary points. In road sections where lane lines are visible, using the current vehicle position as the base point, points are sampled at a certain sampling frequency from the Kalman-predicted lane line trajectory and the actual trajectory to calculate the loss function. A curve showing the change between the number of selected points (n) and the loss is plotted, and the point n corresponding to the lowest point of loss is taken as the dividing point.

[0046]

[0047] Where, x i To predict the lane line trajectory point, x j denoted as the actual trajectory point, and n as the number of points corresponding to the point with the lowest loss.

[0048] Optionally, in some embodiments, predicting the trajectory of the first lane line within the target boundary point using a preset Kalman filter model includes: acquiring target lane line information that meets preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficients and coefficient change rates of the lane line; initializing the state vector of the preset Kalman filter model using the cubic curve coefficients and coefficient change rates of the lane line to obtain an initialized state vector; iteratively updating the initialized state vector and covariance matrix based on lane line observations, observation matrix, and observation noise matrix until preset iteration conditions are met to obtain the trajectory of the first lane line within the target boundary point, wherein the initial value of the covariance matrix is ​​the identity matrix.

[0049] Those skilled in the art will understand that determining the specific implementation of the Kalman filter model can be achieved by acquiring lane line information, determining its effectiveness, acquiring lane line information of the current frame, initializing the Kalman filter state vector, acquiring observations such as the observation matrix and observation matrix noise, updating the state, and generating lane lines.

[0050] Specifically, the trajectory of the first lane within the boundary point is predicted using a Kalman filter model. Valid lane information from the previous frame is obtained, including the cubic curve coefficients representing the lane lines and the rate of change of these coefficients as state variables, and the state vector X is initialized accordingly. k Based on lane line observations, observation matrix, and observation noise matrix, the state vector X is... k The covariance matrix P k The update is performed, where the initial covariance matrix P is... k Set as an identity matrix. By continuously updating the state variables in this way, the lane line trajectory points within the boundary can be obtained.

[0051] X k =[A k A′ k B k B′ k C k D k ] T

[0052] Among them, A k B k C k D k X represents the curve equation coefficients of the lane line at the current moment, which are derived from the state at the previous moment combined with the uniform motion model. k =F*X k-1 .

[0053] Optionally, in some embodiments, predicting the trajectory of the second lane outside the target boundary point using a preset data-driven neural network model includes: acquiring historical lane trajectories and inputting the historical lane trajectories into the encoder of a preset Encoder-Decoder model to compress them into an intermediate quantity of a preset dimension; based on a preset weight sequence, predicting the trajectory of the second lane outside the target boundary point at the Decoder end of the preset Encoder-Decoder model according to the intermediate quantity and the output of the previous time step.

[0054] It should be noted that, in order to predict vehicle trajectories, the LSTM encoder-decoder model, which has shown good performance, was selected. For example... Figure 2As shown, the Encoder-Decoder framework can handle situations where the input and output sequences are of unequal length, corresponding to historical lane lines and predicted lane lines, respectively, in this embodiment. The Encoder encodes the input information, converting it into an intermediate semantic representation, and the Decoder generates the current output based on the intermediate semantics and the already generated information.

[0055] Specifically, a neural network model is trained, and the trained model outputs the lane line trajectory points outside the boundary points. Both the input and output trajectories are serialized data. An Encoder-Decoder model architecture is selected. The historical lane line trajectories input to the Encoder are compressed into fixed-dimensional intermediate quantities for output. At the Decoder, the intermediate quantities and the output from the previous time step are transformed into the current output. LSTM is chosen as the specific algorithm for both the Encoder and Decoder. Furthermore, an attention mechanism is introduced. In lane line prediction, different historical frames have different impacts on future frames. Here, an assumption is made that information closer to the current frame has a greater impact on the future direction of the lane line, meaning it should have a larger weight in the intermediate vector. Through continuous training, the overall weight sequence w = [w1, w2, w3…] is determined, transforming the fixed-length intermediate semantic quantities into different semantic quantities. This allows the network to receive more recent lane line information during Decoder processing, improving the overall prediction accuracy.

[0056] In step S103, based on the first lane line trajectory, when the deviation between the curvature of the target dividing point and the curvature of the second lane line trajectory and the curvature of the first lane line trajectory exceeds a preset threshold, the trajectory points of the second lane line trajectory are corrected, and the final predicted lane line is obtained based on the corrected second lane line trajectory and the first lane line trajectory.

[0057] It should be noted that in order for the network to output reasonable lane line information, the network needs to be trained after predicting the lane line trajectory, and corresponding rules need to be formulated at the dividing point for constraint and post-processing.

[0058] Specifically, the predicted trajectory in this embodiment comes from two algorithms. After obtaining their respective prediction results, post-processing is required to ensure that the two can form a reasonable and accurate lane line. The specific implementation involves using the boundary point trajectory predicted by the Kalman filter as a benchmark, examining the curvature of the boundary point and the trajectory points in the subsequent t frames. When the curvature deviates from the boundary point by a set limit, the trajectory points in the subsequent t frames are corrected using a five-point cubic smoothing method, ultimately obtaining the predicted lane line.

[0059] To enable those skilled in the art to further understand the road lane line prediction method of the embodiments of this application, the following detailed description is provided in conjunction with specific embodiments.

[0060] like Figure 3 As shown, Figure 3 This is a flowchart of a road lane line prediction method proposed according to an embodiment of this application.

[0061] S301, real vehicle-based lane line data collection.

[0062] S302, determine whether it is inside the boundary point. If yes, proceed to step S303; otherwise, proceed to step S305.

[0063] S303 uses a Kalman filter to predict the lane line trajectory within the boundary points.

[0064] S304, predicts the lane line trajectory within the boundary point.

[0065] S305 uses a pre-built data-driven neural network model to predict the trajectory of lane lines outside the boundary points.

[0066] S306 predicts the final lane trajectory based on the lane trajectory inside and outside the boundary point.

[0067] In summary, this application's embodiments introduce two different methods—data-driven and Kalman prediction—to define boundaries and select different methods through an attention mechanism. Based on historical trajectories, it predicts future trajectories, thus achieving lane line processing in this scenario. By combining data-driven and Kalman filtering methods, it achieves high-precision processing of lane lines on aging road surfaces. It also boasts strong portability; after model training and fitting, it can be directly deployed to the vehicle without recalibration.

[0068] The lane line prediction method proposed in this application uses a real-vehicle road data set to define boundaries. Kalman filtering is applied to lane lines in the vicinity, while a data-driven method is used to predict lane lines in the distance. Constraints are set at connection points to determine the rationality of the trajectory. This solves the problems of related technologies neglecting the relationships between lane lines, relying heavily on experience, and being unsuitable for multi-lane scenarios. By using different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

[0069] Next, the road lane prediction device according to the embodiments of this application is described with reference to the accompanying drawings.

[0070] Figure 4 This is a block diagram of a road lane line prediction device according to an embodiment of this application.

[0071] like Figure 4 As shown, the road lane line prediction device 10 includes: a data acquisition module 100, a prediction module 200, and a correction module 300.

[0072] The system includes: a data acquisition module 100 for acquiring the current vehicle's location and lane information; a prediction module 200 for determining the target boundary point based on the location and lane information, predicting the trajectory of the first lane within the target boundary point using a preset Kalman filter model, and predicting the trajectory of the second lane outside the target boundary point using a preset data-driven neural network model (trained from a long short-term memory neural network and a preset Encoder-Decoder model); and a correction module 300 for correcting the trajectory points of the second lane based on the first lane trajectory, identifying when the deviation between the curvature of the trajectory points of the target boundary point and the second lane trajectory exceeds a preset threshold, and obtaining the final predicted lane line based on the corrected second lane trajectory and the first lane trajectory.

[0073] Optionally, in some embodiments, the prediction module 200 is further configured to: acquire target lane line information that meets preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficients and coefficient change rate of the lane line; initialize the state vector of a preset Kalman filter model using the cubic curve coefficients and coefficient change rate of the lane line to obtain an initialized state vector; iteratively update the initialized state vector and covariance matrix according to the lane line observations, observation matrix and observation noise matrix until the preset iteration conditions are met to obtain the trajectory of the first lane line within the target boundary point, wherein the initial value of the covariance matrix is ​​the identity matrix.

[0074] Optionally, in some embodiments, the correction module 300 is further configured to: acquire historical lane line trajectories and input the historical lane line trajectories into the encoder of a preset Encoder-Decoder model to compress them into intermediate quantities of a preset dimension; based on a preset weight sequence, predict the second lane line trajectory outside the target boundary point at the Decoder end of the preset Encoder-Decoder model according to the intermediate quantities and the output of the previous time step.

[0075] Optionally, in some embodiments, the prediction module 200 is further configured to: determine the visible road segment of the lane line based on the lane line information; in the visible road segment of the lane line, select a preset number of points from the Kalman predicted lane line trajectory and the actual trajectory according to a preset sampling frequency, using the location information as the base point, and calculate the loss function based on the preset number of points; draw a change curve based on the preset number of points and the loss function, and take the number of points corresponding to the lowest point of the loss function as the target dividing point.

[0076] Optionally, in some embodiments, the loss function is:

[0077]

[0078] Where, x i To predict the lane line trajectory point, x j denoted as the actual trajectory point, and n as the number of points corresponding to the point with the lowest loss.

[0079] It should be noted that the foregoing explanation of the road lane line prediction method embodiment also applies to the road lane line prediction device of this embodiment, and will not be repeated here.

[0080] The road lane prediction device proposed in this application collects data from real-vehicle roads, divides boundaries, uses Kalman filtering for nearby lane lines, and a data-driven method for predicting distant lane lines. Constraints are set at connection points to determine the rationality of the trajectory. This solves the problems of related technologies ignoring the relationships between lane lines, relying heavily on experience, and being unsuitable for multi-lane scenarios. By using different processing methods for lane lines in different areas, computational resources are saved and algorithm efficiency is improved.

[0081] Figure 5 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0082] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0083] When the processor 502 executes the program, it implements the road lane line prediction method provided in the above embodiments.

[0084] Furthermore, the vehicle also includes:

[0085] Communication interface 503 is used for communication between memory 501 and processor 502.

[0086] The memory 501 is used to store computer programs that can run on the processor 502.

[0087] The memory 501 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0088] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0089] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0090] Processor 502 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of this application.

[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the road lane line prediction method described above.

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0094] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0095] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0096] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0097] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for predicting road lane lines, characterized in that, Includes the following steps: Collect the current vehicle's location and lane information; The target boundary point is determined based on the location information and the lane line information. The trajectory of the first lane line within the target boundary point is predicted by a preset Kalman filter model, and the trajectory of the second lane line outside the target boundary point is predicted by a preset data-driven neural network model. The preset data-driven neural network model is trained by a long short-term memory neural network and a preset Encoder-Decoder model. Based on the first lane line trajectory, when the curvature of the target dividing point and the trajectory points of the second lane line trajectory deviates from the curvature of the trajectory points of the first lane line trajectory by a preset threshold, the trajectory points of the second lane line trajectory are corrected, and the final predicted lane line is obtained based on the corrected second lane line trajectory and the first lane line trajectory. Determining the target boundary point based on the location information and the lane line information includes: The visible road segments of the lane lines are determined based on the lane line information; In the visible section of the lane line, using the location information as the base point, a preset number of points are selected from the Kalman-predicted lane line trajectory and the actual trajectory according to a preset sampling frequency, and a loss function is calculated based on the preset number of points; A change curve is plotted based on the preset number of points and the loss function, and the number of points corresponding to the lowest point of the loss function is taken as the target dividing point.

2. The method according to claim 1, characterized in that, The step of predicting the trajectory of the first lane line within the target boundary point using a preset Kalman filter model includes: Obtain the target lane line information that meets the preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficient and the coefficient change rate of the lane line; Using the cubic curve coefficients of the lane lines and the rate of change of the coefficients, the state vector of the preset Kalman filter model is initialized to obtain the initialized state vector; The initial state vector and covariance matrix are iteratively updated based on lane line observations, observation matrix, and observation noise matrix until a preset iteration condition is met, thereby obtaining the trajectory of the first lane line within the target boundary point, wherein the initial value of the covariance matrix is ​​an identity matrix.

3. The method according to claim 2, characterized in that, The step of predicting the trajectory of the second lane outside the target boundary point using a preset data-driven neural network model includes: Obtain historical lane line trajectories and input the historical lane line trajectories into the Encoder encoder in the preset Encoder-Decoder model to compress them into intermediate quantities of a preset dimension; Based on a preset weight sequence, the Decoder of the preset Encoder-Decoder model predicts the trajectory of the second lane outside the target boundary point according to the intermediate quantity and the output of the previous time step.

4. The method according to claim 1, characterized in that, The loss function is: ； in, x i To predict lane line trajectory points, x j denoted as the actual trajectory point, and n as the number of points corresponding to the point with the lowest loss.

5. A road lane line prediction device, characterized in that, Includes the following steps: The data acquisition module is used to collect the current location information and lane line information of the vehicle. The prediction module is used to determine the target boundary point based on the location information and the lane line information, and predict the trajectory of the first lane line within the target boundary point through a preset Kalman filter model, and predict the trajectory of the second lane line outside the target boundary point through a preset data-driven neural network model, wherein the preset data-driven neural network model is trained by a long short-term memory neural network and a preset Encoder-Decoder model; The correction module, using the first lane line trajectory as a reference, identifies when the deviation between the curvature of the target dividing point and the trajectory points of the second lane line trajectory and the curvature of the trajectory points of the first lane line trajectory is greater than a preset threshold, corrects the trajectory points of the second lane line trajectory, and obtains the final predicted lane line based on the corrected second lane line trajectory and the first lane line trajectory. The prediction module is also used for: The visible road segments of the lane lines are determined based on the lane line information; In the visible section of the lane line, using the location information as the base point, a preset number of points are selected from the Kalman-predicted lane line trajectory and the actual trajectory according to a preset sampling frequency, and a loss function is calculated based on the preset number of points; A change curve is plotted based on the preset number of points and the loss function, and the number of points corresponding to the lowest point of the loss function is taken as the target dividing point.

6. The apparatus according to claim 5, characterized in that, The prediction module is also used for: Obtain the target lane line information that meets the preset conditions in the previous frame, wherein the target lane line information includes the cubic curve coefficient and the coefficient change rate of the lane line; Using the cubic curve coefficients of the lane lines and the rate of change of the coefficients, the state vector of the preset Kalman filter model is initialized to obtain the initialized state vector; The initial state vector and covariance matrix are iteratively updated based on lane line observations, observation matrix, and observation noise matrix until a preset iteration condition is met, thereby obtaining the trajectory of the first lane line within the target boundary point, wherein the initial value of the covariance matrix is ​​an identity matrix.

7. The apparatus according to claim 6, characterized in that, The correction module is also used for: Obtain historical lane line trajectories and input the historical lane line trajectories into the Encoder encoder in the preset Encoder-Decoder model to compress them into intermediate quantities of a preset dimension; Based on a preset weight sequence, the Decoder g of the preset Encoder-Decoder model predicts the trajectory of the second lane outside the target boundary point according to the intermediate quantity and the output of the previous time step.

8. A vehicle, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the road lane line prediction method as described in any one of claims 1-4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the road lane line prediction method as described in any one of claims 1-4.

Images