Method and device for predicting track of automatic driving vehicle, electronic equipment and medium

Abstract

The invention relates to an automatic driving vehicle trajectory prediction method and device, electronic equipment and a medium, and belongs to the technical field of intelligent driving, and the method comprises the steps: obtaining historical trajectory data of a plurality of drivers in a target scene and preference trajectory data at the current moment; inputting the historical trajectory data into the completely trained LSTM model to obtain a candidate trajectory set; cleaning the candidate trajectory set to obtain non-preference trajectory data; combining the historical trajectory data, the preference trajectory data and the non-preference trajectory data to obtain a driving preference data set; taking the LSTM model as a strategy model, and optimizing the strategy model by adopting a direct preference optimization algorithm based on the driving preference data set to obtain an optimized strategy model; and inputting the to-be-predicted data of the target vehicle into the optimized strategy model, and predicting the trajectory of the target vehicle at the next moment. According to the method, the smoothness of the predicted trajectory driving path is better, the driving comfort is better, and the human driving preference is met.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method, apparatus, electronic device, and medium for predicting the trajectory of autonomous vehicles. Background Technology

[0002] With the development of intelligent transportation systems and autonomous driving technology, road traffic environments will become increasingly complex. The diversity of traffic participants' actions, the complex interactions between traffic participants and the environment, and the uncertainty of perceived information will all become key factors in achieving safe decision-making in autonomous driving. Human drivers can typically quickly identify scene information, observe and capture the relationship between the vehicle and surrounding traffic participants, judge the current driving state of the vehicle, and accurately predict the future state of surrounding traffic participants, thus detecting potential dangers in advance and reacting accordingly. Although autonomous driving technology enables intelligent decision-making and control of vehicles, its decision-making models cannot accurately capture the interaction relationships between vehicles when predicting vehicle states. They also lack matching accuracy when predicting different driving preferences for the movement of surrounding vehicles, and their decision results often tend towards deterministic outputs. In some scenarios, the prediction effect and recognition accuracy are degraded to varying degrees. Therefore, autonomous vehicles need to possess judgment and predictive driving capabilities similar to human drivers, capable of predicting the future state of traffic participants and the environment near the vehicle, providing a scientific and reliable basis for vehicle decision-making.

[0003] Intelligent connected vehicles can acquire a large amount of sensor data from multiple sensors through onboard and roadside sensing devices, providing support for understanding the spatiotemporal motion status of surrounding vehicles and road environment conditions, and is also a prerequisite for vehicle trajectory prediction. By processing the collected driving environment information, autonomous vehicles can perform vehicle group trajectory prediction. Previous work on trajectory prediction has mainly focused on methods from physics, classical machine learning, deep learning, and reinforcement learning, enabling the prediction of vehicle driving behavior intentions, as well as single-trajectory and multi-modal trajectory prediction.

[0004] Current trajectory prediction models often extract motion trajectory features based on the target vehicle's historical trajectory, thus enabling relatively accurate predictions of its future path. However, existing methods primarily fit the vehicle's historical motion trajectory on a time-frame basis, failing to comprehensively analyze vehicle motion patterns and accurately capture the interaction relationships between traffic participants. Secondly, while the prediction models identify and predict driving intentions, they only predict driving preferences from a macroscopic perspective, neglecting the crucial element of driver specificity and failing to consider the motion differences between different vehicles. The vehicle trajectory does not fully understand human driving preferences. This prediction approach inevitably weakens the model's universality and the reliability of the trajectory prediction results, and also leads to a decrease in prediction accuracy.

[0005] In summary, existing technologies lack a method that incorporates the interactions between traffic participants and the driving preferences of human drivers into trajectory prediction to better align with more nuanced driving behavior patterns, thereby achieving accurate prediction of future vehicle trajectories and providing more reliable and precise trajectory prediction data support for autonomous driving. Summary of the Invention

[0006] In view of this, it is necessary to provide a method, device, electronic device and medium for predicting the trajectory of autonomous vehicles, in order to solve the problem that the existing technology does not take into account the interaction between traffic participants and the driving choice preferences of human drivers into the scope of trajectory prediction, resulting in low trajectory prediction accuracy.

[0007] To address the aforementioned problems, in a first aspect, the present invention provides a method for predicting the trajectory of an autonomous vehicle, comprising: Acquire historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario; Historical trajectory data is input into a fully trained LSTM model to obtain a candidate trajectory set. The improved LSTM model adds the influence coefficients of surrounding vehicles on the target vehicle to the hidden state based on the LSTM model. The displacement error is obtained based on the candidate trajectory set and the corresponding real trajectory data. The candidate trajectory set is cleaned based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data; Historical trajectory data, preferred trajectory data, and non-preferred trajectory data are combined to obtain a driving preference dataset; The improved LSTM model is used as the policy model, and the direct preference optimization algorithm is used to optimize the policy model based on the driving preference dataset to obtain the optimized policy model. The trajectory data of the target vehicle in the target scene at the previous moment, as well as the trajectory data of the vehicles interacting around the target vehicle at the previous moment, are input into the optimized strategy model to predict the trajectory of the target vehicle at the next moment.

[0008] In one possible implementation, the expression for the hidden state vector of the improved LSTM model is:

[0009] In the formula, This represents the hidden state of vehicle j, which is adjacent to target vehicle i in the target scene. This represents the interaction depth coefficient between target vehicle j and vehicle i. This represents the hidden state of vehicle i in the target scene. , This represents the weights of the LSTM model.

[0010] In one possible implementation, the process of cleaning the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data includes: Trajectories whose critical values ​​for the direction of motion in the candidate trajectory set are not less than the corresponding displacement error are selected to obtain a transition trajectory set. Trajectories whose driving habit scoring function values ​​are greater than a preset value in the transition trajectory set are filtered out to obtain non-preferred trajectory data.

[0011] In one possible implementation, the expression for the preset driving habit scoring function is:

[0012] In the formula: Let represent the score of the m candidate trajectories. Indicates the degree of impact. Indicates lateral acceleration. This represents the total distance traveled on the trajectory. The weighting coefficients representing the impact level. The weighting coefficient represents the lateral acceleration. The weighting coefficient represents the total distance traveled on the trajectory.

[0013] In one possible implementation, the optimization of the policy model based on the driving preference dataset using the direct preference optimization algorithm to obtain the optimized policy model includes: The weighted average is obtained by taking a weighted average of the probability densities of a predetermined number of mixture Gaussian distributions for each trajectory in the unpreferred trajectory data; The first conditional probability of the preferred trajectory data and the second conditional probability of the non-preferred trajectory data are obtained by using the chain method based on a weighted average. Based on the first conditional probability and the second conditional probability, the difference in response values ​​between the policy model and the reference policy model to preferred trajectory data and non-preferred trajectory data is obtained. The strategy model's response to human preferences is determined based on the response difference value; The loss function of the policy model is obtained based on the probability of the policy model responding to human preferences. The optimized strategy model is determined based on the loss function of the strategy model.

[0014] In one possible implementation, the probability density of each trajectory in the unpreferred trajectory data is expressed as:

[0015] In the formula, , These are the coordinates of the point on the trajectory. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the correlation coefficient of the Gaussian distribution. This represents the nth vehicle. Represents probability density, Let represent the k-th Gaussian distribution.

[0016] In one possible implementation, the expression for the loss function of the optimized policy model is:

[0017] In the formula, Represents the loss function. This indicates a deviation from the reference strategy. This represents the LSTM model to be trained. Indicates hyperparameters, Indicates historical trajectory, Let represent the preferred trajectory of vehicle n. Let represent the non-preference trajectory of vehicle n. ， The coordinate axes of vehicle n at time t are represented. , This represents the coordinates of the point on the trajectory. Hyperparameters are used to characterize the control deviation from the reference strategy. The degree; This represents the Sigmoid function.

[0018] In a second aspect, the present invention also provides an apparatus for predicting the trajectory of an autonomous vehicle, comprising: The trajectory data acquisition module is used to acquire historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario. The candidate trajectory set acquisition module is used to input historical trajectory data into the fully trained improved LSTM model to obtain the candidate trajectory set. The improved LSTM model adds the influence coefficient of surrounding vehicles on the target vehicle to the hidden state based on the LSTM model. The displacement error determination module is used to obtain the displacement error based on the candidate trajectory set and the corresponding real trajectory data. The non-preferred trajectory data determination module is used to clean the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data; The driving preference dataset determination module is used to combine historical trajectory data, preferred trajectory data, and non-preferred trajectory data to obtain a driving preference dataset. The optimized policy model determination module is used to use the improved LSTM model as the policy model, and to optimize the policy model using the direct preference optimization algorithm based on the driving preference dataset to obtain the optimized policy model. The vehicle trajectory prediction module is used to input the trajectory data of the target vehicle in the target scene at the previous moment and the trajectory data of the vehicles interacting around the target vehicle at the previous moment into the optimized strategy model to predict the trajectory of the target vehicle at the next moment. Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the method for predicting the trajectory of an autonomous vehicle as described in any of the above implementations.

[0019] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps of the method for predicting the trajectory of an autonomous vehicle as described in any of the above implementations.

[0020] The beneficial effects of this invention are as follows: This invention provides a method for predicting the trajectory of an autonomous vehicle, which acquires historical trajectory data and current preferred trajectory data of multiple drivers in a target scene. The historical trajectory data is input into a fully trained LSTM model to obtain a candidate trajectory set, taking into account the influence of surrounding vehicles on the target vehicle. Displacement error is obtained based on the candidate trajectory set and the corresponding real trajectory data. The candidate trajectory set is cleaned based on the displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data, removing trajectory data that does not conform to human driving habits. The historical trajectory data, preferred trajectory data, and non-preferred trajectory data are combined to obtain a driving preference dataset, thereby incorporating human driving preferences. Using the LSTM model as a policy model, the direct preference optimization algorithm is applied to optimize the policy model based on the driving preference dataset, resulting in an optimized policy model that achieves a deeper and better capture of human driving preferences. The trajectory data of the target vehicle at the previous moment and the trajectory data of the vehicles interacting with the target vehicle at the previous moment are input into the optimized policy model to predict the trajectory of the target vehicle at the next moment. Based on the policy model, a joint prediction trajectory considering human driving preferences is achieved. This invention incorporates the influence of other vehicles into the LSTM model, introduces human driving preferences into the traditional trajectory prediction method, and achieves a better capture of deeper human driving preferences through human-like optimization of trajectory prediction using the direct preference optimization algorithm. This aligns with human driving habits, improves driving comfort, and thereby enhances trajectory prediction accuracy. Attached Figure Description

[0021] Figure 1 A flowchart illustrating an embodiment of a method for predicting the trajectory of an autonomous vehicle provided by the present invention; Figure 2 A schematic diagram illustrating human-like trajectory prediction is provided as an embodiment of the method for predicting the trajectory of an autonomous vehicle according to the present invention. Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 4 A flowchart illustrating the generation of a human driving preference dataset, which is an embodiment of a method for predicting the trajectory of autonomous vehicles provided by the present invention; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S106; Figure 6 A schematic flowchart of an embodiment of the device for predicting the trajectory of an autonomous vehicle provided by the present invention; Figure 7 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0024] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] This invention provides a method, apparatus, electronic device, and medium for predicting the trajectory of autonomous vehicles, which are described below.

[0027] Figure 1 A schematic flowchart of an embodiment of the autonomous vehicle trajectory prediction method provided by the present invention is shown below. Figure 1 As shown, the method for predicting the trajectory of autonomous vehicles includes: S101. Obtain historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario; In the target scenario (such as a specific road segment, intersection, or other scenario where autonomous driving is operating), trajectory data formed by multiple drivers over a past period is collected. This data reflects the drivers' historical driving behavior in that scenario. Simultaneously, the driver's preferred driving trajectory data at the current moment is obtained. This preferred trajectory can be understood as information such as the driving route the driver is more likely to choose in the current scenario and state. This data serves as the foundational input for subsequent trajectory prediction.

[0028] S102. Input the historical trajectory data into the fully trained LSTM model to obtain the candidate trajectory set; After training with a large amount of data, the LSTM model has reached a relatively complete state. Inputting previously acquired historical trajectory data into this improved LSTM model, the model predicts the possible future trajectories of the target vehicle based on its internal parameters and structure, outputting a candidate trajectory set—a series of possible future trajectories. The key improvement lies in introducing the influence coefficients of surrounding vehicles on the target vehicle into the model's hidden states. This means that the model considers the presence and movement of other vehicles around the target vehicle when making predictions, thus making the prediction results more consistent with the interactions in real traffic scenarios.

[0029] S103. Obtain the displacement error based on the candidate trajectory set and the corresponding real trajectory data; The candidate trajectory set output by the model is compared with actual trajectory data (obtainable through actual vehicle driving records, etc.). For each trajectory in the candidate trajectory set and its corresponding real trajectory, the displacement error between them is calculated. This can be achieved by measuring the degree of difference between the model-predicted trajectory and the real trajectory, such as by calculating the distance deviation between corresponding points on the trajectory. This displacement error reflects the accuracy of the model's prediction.

[0030] S104. Based on displacement error and a preset driving habit scoring function, the candidate trajectory set is cleaned to obtain non-preferred trajectory data; The driving habit scoring function uses established rules of reasonable driving habits and traffic regulations, such as the smoothness of the trajectory and whether the driving path complies with traffic regulations, as evaluation criteria to score each trajectory in the candidate trajectory set. Trajectories with low scores are considered to be inconsistent with most driving habits or deviate significantly from reality. These trajectories are then removed from the candidate trajectory set to obtain non-preferred trajectory data, which are unlikely to be the driving trajectories that the target vehicle would actually choose.

[0031] S105. Combine historical trajectory data, preferred trajectory data, and non-preferred trajectory data to obtain a driving preference dataset; The driving preference dataset contains not only past driving behavior patterns, but also current preference information and trajectory information that does not conform to preferences, providing more comprehensive data support for further optimization of the model.

[0032] S106. Using the LSTM model as the policy model, the direct preference optimization algorithm is used to optimize the policy model based on the driving preference dataset to obtain the optimized policy model. The LSTM model is considered a policy model, specifically used to determine the future trajectory of a target vehicle. Using the previously constructed driving preference dataset, the Direct Preference Optimization (DPO) algorithm is employed to optimize this policy model. DPO adjusts the model's parameters based on preference information in the dataset, allowing the model to better align with driver preferences and actual driving conditions when predicting trajectories. This results in an optimized policy model that should exhibit higher accuracy and reasonableness in trajectory prediction.

[0033] S107. Input the trajectory data of the target vehicle in the target scene at the previous moment and the trajectory data of the vehicles interacting around the target vehicle at the previous moment into the optimized strategy model to predict the trajectory of the target vehicle at the next moment. In actual trajectory prediction, the target vehicle's trajectory data from the previous moment, along with the trajectory data of vehicles interacting with it at the same moment (such as nearby vehicles traveling in the same direction, oncoming vehicles, and vehicles about to merge into the lane), are input into the optimized strategy model. Based on this input information, combined with its optimized parameters and structure, the model predicts the target vehicle's trajectory for the next moment and outputs a specific prediction result. This result is an estimate of the target vehicle's future driving path, which can be used for advance planning and decision-making in autonomous driving systems.

[0034] Compared with existing technologies, this embodiment provides a method for predicting the trajectory of autonomous vehicles. It acquires historical trajectory data and current preferred trajectory data from multiple drivers in a target scene. The historical trajectory data is input into a fully trained LSTM model to obtain a candidate trajectory set, taking into account the influence of surrounding vehicles on the target vehicle. Displacement error is obtained based on the candidate trajectory set and the corresponding real trajectory data. The candidate trajectory set is cleaned based on the displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data, removing trajectory data that does not conform to human driving habits. The historical trajectory data, preferred trajectory data, and non-preferred trajectory data are combined to obtain a driving preference dataset, thereby incorporating human driving preferences. Using the LSTM model as a policy model, the direct preference optimization algorithm is applied to optimize the policy model based on the driving preference dataset, resulting in an optimized policy model that captures deeper levels of human driving preferences. The trajectory data of the target vehicle at the previous moment and the trajectory data of the vehicles interacting with the target vehicle at the previous moment are input into the optimized policy model to predict the trajectory of the target vehicle at the next moment, achieving a joint prediction trajectory that considers human driving preferences based on the policy model. This invention incorporates the influence of other vehicles into the LSTM model, introduces human driving preferences into the traditional trajectory prediction method, and achieves a better capture of deeper human driving preferences through human-like optimization of trajectory prediction using the direct preference optimization algorithm. This aligns with human driving habits, improves driving comfort, and thereby enhances trajectory prediction accuracy.

[0035] It should be noted that the application scenario of this invention is urban roads or highways.

[0036] In a specific embodiment of the present invention, in step S102, it should first be noted that the traditional LSTM model is widely used for vehicle trajectory prediction. It can learn from historical vehicle motion states to obtain reliable future vehicle trajectories. The observed trajectories of N vehicles in a given scenario are represented as follows:

[0037] In the formula: n∈{1,…, N}, representing that there are N cars in the scene; tr o n The observed trajectory of the nth vehicle; z t n Represents the observation trajectory point of the nth vehicle at time t; x t n , y t n These are the vertical and horizontal coordinates of the observation trajectory point, respectively; T o This represents the length of the observed time series.

[0038] The goal of motion state prediction is to observe each vehicle in t∈{1,…, T o Movement trajectory within the time period tr p n Predict the next time period t∈{1,…, T p Vehicle movement trajectory within} tr p n The predicted trajectory is represented as:

[0039] In the formula: z t np Let represent the predicted trajectory point of the nth vehicle at time t; x t np , y t np These represent the vertical and horizontal coordinates of the predicted trajectory points, respectively. T p This indicates the length of the predicted time series.

[0040] The LSTM neural network model is used to analyze the vehicle motion state sequence, and the prediction process is as follows: At time t, the vehicle motion state sequence in the scene is... The vehicle's coordinates are ( x t i , p t i LSTM hidden state H t i Capture its potential motion state by constructing a hidden state tensor H t i Its hidden state has a dimension of D, and the coordinates are embedded intoe t i In this process, these embedded values ​​are concatenated and used as the input to the corresponding trajectory LSTM at time t:

[0041]

[0042] In the formula: (.) is a ReLU embedding function with nonlinearity. W e For embedding layer weights, W l This represents the weights of the LSTM.

[0043] The hidden state at time t is used to predict the trajectory position at the next time step t+1. LSTM can be used to establish temporal relationships between vehicle motion states, using the previous time-series state to predict the trajectory of the next moment. However, the vehicle trajectory is not only related to the motion state of the previous time-series, but also to the real-time interaction between the current vehicle and surrounding vehicles, and its motion state is mutually constrained by both. Vehicles moving in the same direction interact in terms of speed, while lateral vehicles only interact when the vehicle is turning. Therefore, an elliptical driving interaction boundary is designed based on the maximum headway (THW) and the width of the lateral lane, incorporating the hidden state tensors of surrounding vehicles into the input of the current vehicle at the next moment, and dynamically adjusting the weights of lateral vehicles based on vehicle motion intention recognition to achieve joint prediction of multi-vehicle trajectories. In some embodiments of the present invention, the expression of the hidden state vector of the improved LSTM model is as follows:

[0044] In the formula, This represents the hidden state of vehicle j, which is adjacent to target vehicle i in the target scene. This represents the interaction depth coefficient between target vehicle j and vehicle i. This represents the hidden state of vehicle i in the target scene. , This represents the weights of the LSTM model.

[0045] like Figure 2 The diagram illustrates a human-like trajectory prediction method. The loss function predicts an "average" trajectory that is geometrically closest to the actual trajectory, but this also makes the result overly conservative. To preserve more of the prediction potential, a fully connected layer is used to represent the predicted trajectory using a two-dimensional Gaussian mixture distribution. t Hidden state h t i’Used to predict the next time step t+ The position of trajectory 1 The distribution of . Define a complex probability distribution consisting of a mixture of k bivariate Gaussian distributions. The mean of each Gaussian distribution is The standard deviation is The correlation coefficient is ρ t+1 ik These parameters are each determined by a given... 5 × D weight matrix W p For linear layer prediction, the predicted coordinates at time t are... for:

[0046] The relevant parameters in the improved LSTM model are learned by minimizing the negative log-likelihood loss, as defined below:

[0047] In the formula: W It includes all trainable parameters in the model, among which μ t n It is the mean of the distribution. σ t n It is the variance of the distribution. ρ t n It is their correlation coefficient.

[0048] Based on the driving dataset, the improved model in this paper is used to extract driving features, generate future trajectories point by point through autoregression, calculate the loss by minimizing the negative log-likelihood loss, and use the backpropagation algorithm to continuously learn the model parameters, thereby realizing the training of a multi-vehicle joint trajectory prediction model.

[0049] In some embodiments of the present invention, such as Figure 3 As shown, in step S104, the process of cleaning the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data includes: S301. Select the trajectories whose critical values ​​for the direction of motion are not less than the corresponding displacement error from the candidate trajectory set to obtain the transition trajectory set; S302. Filter out the trajectories whose driving habit scoring function values ​​are greater than a preset value from the transition trajectory set to obtain non-preferred trajectory data.

[0050] In specific embodiments of the present invention, such as Figure 4The diagram shows the flowchart for generating a human driving preference dataset. The preference dataset consists of triplets of historical trajectory data, preferred trajectory data, and non-preferred trajectory data. ,in Let n be the preferred trajectory of vehicle n. Let n be the non-preferred trajectory of vehicle n. For the historical trajectory.

[0051] Based on the acquired original trajectory dataset containing multiple road segments and multiple drivers, data points containing a historical trajectory segment are obtained. This historical trajectory is used as input to a pre-trained LSTM model, and the resulting probability distribution is randomly sampled. This repeated sampling process generates a set of M differential candidate trajectories with the same prediction length, denoted as... The trajectory data in this dataset is then used as the non-preferred data. The candidate trajectory set contains various potentially reasonable trajectory data, and their quality greatly affects the quality of the non-preferred trajectory. Excessive similarity between candidate and preferred trajectories leads to a weak learning signal, making it impossible to correctly learn human driving habit features. First, the final displacement error (FDE) is used to measure the distance between the predicted trajectory endpoint and the actual trajectory endpoint. A high FDE usually indicates a misunderstanding of the driver's intentions and is outside the scope of human-like trajectory learning. This type of data is rejected from candidate trajectories, among which The critical value used to distinguish the direction of trajectory motion.

[0052] To improve DPO learning efficiency, a driving habit scoring function is designed as the rejection criterion for candidate trajectories. This function measures the difference between candidate trajectories and preferred trajectories, creating clearer and stronger preference signals for guidance. Trajectory prediction models that minimize negative log-likelihood loss tend to generate trajectories with the closest absolute distance to the training dataset, lacking consideration for the overall trajectory rationality within a time series. Therefore, it is necessary to learn human driver characteristics from trajectory dynamic smoothness. In some embodiments of this invention, the expression of the preset driving habit scoring function is:

[0053] In the formula: Let represent the score of the m candidate trajectories. Indicates the degree of impact. Indicates lateral acceleration. This represents the total distance traveled on the trajectory. The weighting coefficients representing the impact level. The weighting coefficient represents the lateral acceleration. The weighting coefficient represents the total distance traveled on the trajectory.

[0054] Candidate trajectories for each vehicle are filtered according to a scoring function to obtain the trajectory data that differs most significantly from the preference data. The trajectory data is then stored in triples, and different road segments and different vehicles are iterated to form a human driving preference dataset.

[0055] In some embodiments of the present invention, such as Figure 5 As shown, in step S106, the strategy model is optimized using the direct preference optimization algorithm based on the driving preference dataset to obtain the optimized strategy model, including: S501. Take a weighted average of the probability density of a predetermined number of mixed Gaussian distributions for each trajectory of the non-preferred trajectory data to obtain a weighted average. S502. Obtain the first conditional probability of the preferred trajectory data and the second conditional probability of the non-preferred trajectory data based on the weighted average value using the chain method; S503. Based on the first conditional probability and the second conditional probability, obtain the response difference value between the policy model and the reference policy model for preferred trajectory data and non-preferred trajectory data; S504. Determine the human preference response of the strategy model based on the response difference value; S505. Obtain the loss function of the policy model based on the probability of the policy model responding to human preferences; S506. Determine the optimized strategy model based on the loss function of the strategy model.

[0056] In a specific embodiment of the present invention, the trajectory optimization method based on LSTM can obtain a relatively accurate predicted trajectory, but it does not care about the "style" of the trajectory, such as the smoothness of acceleration, the curvature of turning, etc. It cannot capture deeper driving behavior patterns, which often do not conform to human driving habits. These are precisely the key to making the predicted trajectory more human-like.

[0057] Direct Preference Optimization (DPO) is a novel method for optimizing language models based on human preferences, primarily applied in the field of Large Language Models (LLMs). It relies on the Bradley-Terry preference model and directly measures the alignment between the reward function and empirical preference data, without explicitly training a reward model or relying on Reinforcement Learning (RL). Traditional methods use preference models to train reward models and then maximize rewards to train the policy. DPO, however, implicitly transforms the model itself into a reward function, directly defining the preference loss based on the policy, and precisely optimizing the reward maximization problem through single-stage policy training. Through variable transformation, the mapping relationship between the reward function and the optimal policy can be obtained:

[0058] In the formula: It is a reward function. It is the optimal strategy. It is a reference strategy. It is used for control and The parameter for the degree of deviation, It is a partition function.

[0059] The trajectory prediction model based on LSTM lacks optimization for human preferences. This paper utilizes the method of directly fine-tuning the reference model from the Large Language Models (LLMs) domain, applying the DPO optimization method to improve the anthropomorphic nature of the predicted trajectory. The LSTM prediction model serves as the baseline reference policy, using the real driving trajectory (the preferred trajectory data at the current moment) as the preferred response and the generated trajectory (non-preferred trajectory data) as the non-preferred response. A baseline trajectory prediction model can be trained based on the driving dataset, and this model is used as the reference policy. and initial strategy Binary preference data is constructed using real trajectories and generated trajectories. The Bradley-Terry model is used to represent the difference in rewards between two different responses:

[0060] In the formula: It is a preference response. Yes / no preference response.

[0061] For predicted trajectory (non-preferred trajectory data points) The probability density of the non-preferred trajectory data under the k-th Gaussian distribution is given by: In some embodiments of the present invention, the expression for the probability density of each trajectory of the non-preferred trajectory data is:

[0062] In the formula, , These are the coordinates of the point on the trajectory. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the correlation coefficient of the Gaussian distribution. This represents the nth vehicle. Represents probability density, Let represent the k-th Gaussian distribution.

[0063] definition Let be the mixture weights of the k-th Gaussian distribution, then Predicting trajectory points The total probability density is the weighted average of these k Gaussian distributions, that is:

[0064] A complete vehicle trajectory consists of a series of time-series data, which can be decomposed into a product of conditional probabilities using the chain rule. The conditional probabilities of the actual trajectory and the generated trajectory are as follows:

[0065]

[0066] Based on the above formula, the vehicle trajectory is obtained in the reference strategy. and optimization strategy model The probability of the generated trajectory is used to construct the loss function of the trajectory prediction model. This function directly maximizes the likelihood probability of the preference response pair, that is, it increases the probability of the real trajectory data and reduces the probability of the generated trajectory data. Since both the real trajectory and the generated trajectory are driving data generated by making judgments based on historical data, the probability of the vehicle trajectory can be considered equal to its conditional probability. Therefore, in some embodiments of the present invention, the expression of the loss function of the optimized strategy model is:

[0067] In the formula, Represents the loss function. This indicates a deviation from the reference strategy. This represents the LSTM model to be trained. Indicates hyperparameters, Indicates historical trajectory, Let represent the preferred trajectory of vehicle n. Let represent the non-preference trajectory of vehicle n. ， The coordinate axes of vehicle n at time t are represented. , This represents the coordinates of the point on the trajectory. Hyperparameters are used to characterize the control deviation from the reference strategy. The degree; This represents the Sigmoid function.

[0068] The loss function quantifies the difference between the predicted trajectory and the true trajectory under the current parameters. The parameter gradient is calculated using the differential chain rule to find the direction that fastest reduces the loss in the multidimensional parameter space. Gradient descent is used to update the parameters during the iterative process until the model converges to a local or global minimum of the loss function. This cycle is repeated continuously on the entire preference dataset to optimize the policy model. It will gradually learn to internalize human driving preferences, and under the constraints of the reference strategy, it will not only be able to predict trajectories, but also learn to select the more human-like vehicle trajectory from multiple possible trajectories, ultimately arriving at the optimal strategy. This makes the predicted trajectory more consistent with human driving habits.

[0069] In summary, this method incorporates the hidden states of surrounding vehicles into the input of the next time step based on vehicle interaction relationships, enabling multi-vehicle joint trajectory prediction. It also introduces human driving preferences and innovatively uses a fine-tuning supervised learning method from the large model domain to optimize vehicle trajectory prediction. The direct preference optimization method is directly used to fine-tune the prediction results based on the improved LSTM trajectory prediction method, thus addressing the shortcomings of traditional trajectory prediction methods in terms of trajectory smoothness and human-likeness, and achieving vehicle trajectory prediction that better conforms to human driving habits.

[0070] To better implement the method for predicting the trajectory of an autonomous vehicle in this embodiment of the invention, based on the method for predicting the trajectory of an autonomous vehicle, correspondingly, as follows: Figure 6 As shown, this embodiment of the invention also provides an apparatus for predicting the trajectory of an autonomous vehicle. An apparatus 600 for predicting the trajectory of an autonomous vehicle includes: The trajectory data acquisition module 601 is used to acquire historical trajectory data of multiple drivers in the target scenario and preferred trajectory data at the current moment; The candidate trajectory set acquisition module 602 is used to input historical trajectory data into a fully trained LSTM model to obtain a candidate trajectory set; The displacement error determination module 603 is used to obtain the displacement error based on the candidate trajectory set and the real trajectory data corresponding to the candidate trajectory set. The non-preferred trajectory data determination module 604 is used to clean the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data. The driving preference dataset determination module 605 is used to combine historical trajectory data, preferred trajectory data and non-preferred trajectory data to obtain a driving preference dataset. The optimized strategy model determination module 606 is used to use the improved LSTM model as the strategy model, and optimize the strategy model based on the driving preference dataset using the direct preference optimization algorithm to obtain the optimized strategy model. The vehicle trajectory prediction module 607 is used to input the trajectory data of the target vehicle in the target scene at the previous moment and the trajectory data of the vehicles interacting around the target vehicle at the previous moment into the optimized strategy model to predict the trajectory of the target vehicle at the next moment. The device 600 for predicting the trajectory of an autonomous vehicle provided in the above embodiments can realize the technical solution described in the above embodiments of the method for predicting the trajectory of an autonomous vehicle. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the method for predicting the trajectory of an autonomous vehicle, and will not be repeated here.

[0071] like Figure 7 As shown, the present invention also provides an electronic device 700. The electronic device 700 includes a processor 701, a memory 702, and a display 703. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0072] In some embodiments, processor 701 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 702 or process data, such as a method for predicting the trajectory of an autonomous vehicle in this invention.

[0073] In some embodiments, processor 701 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 701 may be local or remote. In some embodiments, processor 701 may be implemented on a cloud platform. In some embodiments, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, or any combination thereof.

[0074] In some embodiments, memory 702 may be an internal storage unit of electronic device 700, such as a hard disk or memory of electronic device 700. In other embodiments, memory 702 may also be an external storage device of electronic device 700, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 700.

[0075] Furthermore, the memory 702 may include both internal storage units of the electronic device 700 and external storage devices. The memory 702 is used to store application software and various types of data installed on the electronic device 700.

[0076] In some embodiments, display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 703 is used to display information from electronic device 700 and to display a visual user interface. Components 701-703 of electronic device 700 communicate with each other via a system bus.

[0077] In one embodiment, when processor 701 executes a program for predicting the trajectory of an autonomous vehicle stored in memory 702, the following steps can be implemented: Acquire historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario; Historical trajectory data is input into a fully trained LSTM model to obtain a set of candidate trajectories; The displacement error is obtained based on the candidate trajectory set and the corresponding real trajectory data. The candidate trajectory set is cleaned based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data; Historical trajectory data, preferred trajectory data, and non-preferred trajectory data are combined to obtain a driving preference dataset; The LSTM model is used as the policy model, and the direct preference optimization algorithm is used to optimize the policy model based on the driving preference dataset to obtain the optimized policy model. The trajectory data of the target vehicle in the target scene at the previous moment, as well as the trajectory data of the vehicles interacting around the target vehicle at the previous moment, are input into the optimized strategy model to predict the trajectory of the target vehicle at the next moment. It should be understood that when the processor 701 executes a program for predicting the trajectory of an autonomous vehicle in the memory 702, in addition to the functions described above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0078] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 700 mentioned. Electronic device 700 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, electronic device 700 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0079] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0080] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting the trajectory of an autonomous vehicle, characterized in that, include: Acquire historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario; Historical trajectory data is input into a fully trained LSTM model to obtain a set of candidate trajectories; The displacement error is obtained based on the candidate trajectory set and the corresponding real trajectory data. The candidate trajectory set is cleaned based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data; Historical trajectory data, preferred trajectory data, and non-preferred trajectory data are combined to obtain a driving preference dataset; The LSTM model is used as the policy model, and the direct preference optimization algorithm is used to optimize the policy model based on the driving preference dataset to obtain the optimized policy model. The trajectory data of the target vehicle in the target scene at the previous moment, as well as the trajectory data of the vehicles interacting around the target vehicle at the previous moment, are input into the optimized strategy model to predict the trajectory of the target vehicle at the next moment.

2. The method for predicting the trajectory of an autonomous vehicle according to claim 1, characterized in that, The expression for the hidden state vector of the LSTM model is: In the formula, This represents the hidden state of vehicle j, which is adjacent to target vehicle i in the target scene. This represents the interaction depth coefficient between target vehicle j and vehicle i. This represents the hidden state of vehicle i in the target scene. , This represents the weights of the LSTM model.

3. The method for predicting the trajectory of an autonomous vehicle according to claim 1, characterized in that, The process of cleaning the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data includes: Trajectories whose critical values ​​for the direction of motion in the candidate trajectory set are not less than the corresponding displacement error are selected to obtain a transition trajectory set. Trajectories whose driving habit scoring function values ​​are greater than a preset value in the transition trajectory set are filtered out to obtain non-preferred trajectory data.

4. The method for predicting the trajectory of an autonomous vehicle according to claim 3, characterized in that, The expression for the preset driving habit scoring function is: In the formula: Let represent the score of the m candidate trajectories. Indicates the degree of impact. Indicates lateral acceleration. This represents the total distance traveled on the trajectory. The weighting coefficients representing the impact level. The weighting coefficient represents the lateral acceleration. The weighting coefficient represents the total distance traveled on the trajectory.

5. The method for predicting the trajectory of an autonomous vehicle according to claim 1, characterized in that, The strategy model is optimized using the direct preference optimization algorithm based on the driving preference dataset to obtain the optimized strategy model, including: The weighted average is obtained by taking a weighted average of the probability densities of a predetermined number of mixture Gaussian distributions for each trajectory in the unpreferred trajectory data; The first conditional probability of the preferred trajectory data and the second conditional probability of the non-preferred trajectory data are obtained by using the chain method based on a weighted average. Based on the first conditional probability and the second conditional probability, the difference in response values ​​between the policy model and the reference policy model to preferred trajectory data and non-preferred trajectory data is obtained. The strategy model's response to human preferences is determined based on the response difference value; The loss function of the policy model is obtained based on the probability of the policy model responding to human preferences. The optimized strategy model is determined based on the loss function of the strategy model.

6. The method for predicting the trajectory of an autonomous vehicle according to claim 1, characterized in that, The expression for the probability density of each trajectory in the non-preferred trajectory data is: In the formula, , These are the coordinates of the point on the trajectory. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the eigenvalue of the two-dimensional Gaussian distribution at time t. This represents the correlation coefficient of the Gaussian distribution. This represents the nth vehicle. Represents probability density, Let represent the k-th Gaussian distribution.

7. The method for predicting the trajectory of an autonomous vehicle according to claim 1, characterized in that, The expression for the loss function of the optimized policy model is as follows: In the formula, Represents the loss function. This indicates a deviation from the reference strategy. This represents the LSTM model to be trained. Indicates hyperparameters, Indicates historical trajectory, Let represent the preferred trajectory of vehicle n. Let represent the non-preference trajectory of vehicle n. ， The coordinate axes of vehicle n at time t are represented. , This represents the coordinates of the point on the trajectory. Hyperparameters are used to characterize the control deviation from the reference strategy. To what extent, This represents the Sigmoid function.

8. A device for predicting the trajectory of an autonomous vehicle, characterized in that, include: The trajectory data acquisition module is used to acquire historical trajectory data and current preferred trajectory data of multiple drivers in the target scenario. The candidate trajectory set acquisition module is used to input historical trajectory data into a fully trained LSTM model to obtain a candidate trajectory set; The displacement error determination module is used to obtain the displacement error based on the candidate trajectory set and the corresponding real trajectory data. The non-preferred trajectory data determination module is used to clean the candidate trajectory set based on displacement error and a preset driving habit scoring function to obtain non-preferred trajectory data; The driving preference dataset determination module is used to combine historical trajectory data, preferred trajectory data, and non-preferred trajectory data to obtain a driving preference dataset. The optimized strategy model determination module is used to use the LSTM model as the strategy model and optimize the strategy model based on the driving preference dataset using the direct preference optimization algorithm to obtain the optimized strategy model. The vehicle trajectory prediction module is used to input the trajectory data of the target vehicle in the target scene at the previous moment and the trajectory data of the vehicles interacting around the target vehicle at the previous moment into the optimized strategy model to predict the trajectory of the target vehicle at the next moment.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the method for predicting the trajectory of an autonomous vehicle as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the method for predicting the trajectory of an autonomous vehicle as described in any one of claims 1 to 7.

Images