Vehicle trajectory prediction method, device, medium and equipment

Abstract

The present invention discloses a vehicle trajectory prediction method, device, medium and equipment, and relates to the field of unmanned driving technology. When considering the influence of driving style on the vehicle's driving trajectory, the present invention does not directly specify the division of driving style, but clusters based on the historical trajectory of each vehicle, and performs subsequent driving behavior prediction under the corresponding driving style based on the obtained multi-class driving style clustering data, as well as the future trajectory prediction of each vehicle. Finally, the total loss is constructed according to the dissimilarity between the driving style clustering data of each vehicle, the deviation between the real driving behavior and the predicted driving behavior, and the deviation between the real future trajectory and the predicted future trajectory, and joint optimization training is performed. In this way, the driving style clustering is continuously adjusted, the driving style clustering process is optimized, and the subsequent vehicle behavior recognition model and trajectory prediction model are optimized at the same time, which reduces the transmission of erroneous information in the entire vehicle trajectory prediction process and improves the vehicle trajectory prediction accuracy.

Description

Technical Field

[0001] The present application relates to the field of unmanned driving technology, and in particular to a vehicle trajectory prediction method, device, medium and equipment. Background Art

[0002] At present, driverless vehicles have become an inevitable trend in the development of future automobiles. The technologies of driverless vehicles include perception, communication, decision-making and control. Specifically, perception technology can help driverless vehicles obtain information about the surrounding environment and predict the future movement trajectory of surrounding vehicles. Its advanced nature can reduce the probability and severity of traffic accidents, alleviate traffic congestion and improve resource utilization. With the development of driverless technology, vehicles will operate in mixed traffic conditions, with both manual and unmanned driving. This means that driverless vehicles need to perceive the different driving styles of human drivers, accurately predict their future trajectories, and make safe and efficient plans and decisions based on this information.

[0003] In the prior art, when an unmanned vehicle predicts the trajectory of other vehicles on the road, it can predict the trajectory of the vehicle based on the different driving styles of the driver. For example, on the one hand, the driver's driving style is divided into normal, aggressive and careless. Further, the driving style is combined with the driving behavior, and the driving task is divided into active lane change, passive lane change, active straight driving, passive straight driving, etc. After the driver's driving style is identified by the driving style recognition model, the trajectory prediction of each driving task is finally achieved.

[0004] However, current driverless vehicles adopt a sequential structure when predicting vehicle trajectories, that is, first dividing the driver's style, then building a driving style recognition model, and finally building a vehicle trajectory prediction model structure. If the driver's style division is inaccurate, it will lead to the transmission of erroneous information, which will reduce the accuracy of subsequent vehicle trajectory predictions. Summary of the invention

[0005] Based on this, it is necessary to provide a vehicle trajectory prediction method, device, medium and equipment to address the above technical problems.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a vehicle trajectory prediction method, including:

[0008] Obtain the historical trajectories of multiple vehicles in the same scenario before the target time, as well as the actual driving behavior and actual future trajectory of each vehicle after the target time;

[0009] The historical trajectories of each vehicle are clustered through a clustering algorithm model to obtain multi-class driving style clustering data;

[0010] The historical trajectory and multi-class driving style clustering data of each vehicle are input into the support vector machine as the vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style;

[0011] The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input into the neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period;

[0012] The total loss is constructed according to the dissimilarity between the clustering data of each vehicle's driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory. The clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model are jointly optimized and trained with the minimum total loss as the optimization goal. The vehicle trajectory is predicted by using the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model after the joint optimization training.

[0013] Optionally, clustering the historical trajectories of each vehicle using a clustering algorithm model to obtain multiple types of driving style clustering data specifically includes:

[0014] The speed variation coefficient of each vehicle is determined by the following formula:

[0015]

[0016] The speed variation coefficient, average speed and average acceleration corresponding to the historical trajectory of each vehicle are clustered through the K-means++ model to obtain multi-class driving style clustering data;

[0017] Where Vc is the speed variation coefficient of the vehicle, N is the number of historical trajectory data of the vehicle, is the average speed of the vehicle, V i is the instantaneous speed of the vehicle.

[0018] Optionally, determining the predicted driving behavior of each vehicle under the corresponding driving style specifically includes:

[0019] The optimal hyperplane is determined by the following formula, where the optimal hyperplane maximizes the distance from the closest vehicle historical trajectory in different driving behaviors to the optimal hyperplane:

[0020]

[0021] The support vector is obtained by solving the problem, and the decision function is calculated based on the support vector. The decision function is used to determine the predicted driving behavior under the corresponding driving style according to the historical trajectory of each vehicle and the multi-class driving style clustering data;

[0022] Among them, w and b are hyperplane parameters, C is the penalty coefficient, ξi is a slack variable, x is the vehicle historical trajectory data, y is the driving behavior corresponding to the vehicle historical trajectory data, and n is the number of samples.

[0023] Optionally, the input of the vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle into a neural network model of a trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period specifically includes:

[0024] The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input as the Autoformer model of the trajectory prediction model, and the autocorrelation information between the sequence data of the input data is obtained through the autocorrelation mechanism in the encoder of the trajectory prediction model; the autocorrelation information is combined with the input data, and the long-term characteristics and volatility characteristics in the input data are extracted based on the combined data according to the following formula through the internal sequence decomposition block in the encoder, and the long-term characteristics contained in the input data are eliminated;

[0025]

[0026] χ t =AvgPool(Padding(χ)),

[0027] χ s = x-x t ;

[0028] The volatility characteristics of the input data are transmitted to the decoder of the trajectory prediction model, the input data is combined with the volatility characteristics, and the volatility information is predicted through the autocorrelation module, the internal sequence decomposition block and the feedforward neural network to obtain the predicted volatility information; the input data is aggregated multiple times with the long-term information of the input data output by the internal sequence decomposition block in the decoder to obtain the predicted long-term information; the predicted future trajectory of each vehicle within a preset time period is determined by combining the predicted volatility information and the predicted long-term information;

[0029] Among them, χ is the input data, is the historical trajectory of the i-th target vehicle in the i-th scenario, N is the number of target vehicles, Δt is the time window, his is the historical time window, is the vehicle position of the i-th target vehicle at time t, is the historical trajectory of the surrounding vehicles of the i-th target vehicle, is the historical trajectory of the vehicle in front of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, is the historical trajectory of the vehicle in front of the left side of the i-th target vehicle, is the historical trajectory of the vehicle behind the left side of the i-th target vehicle, is the historical trajectory of the vehicle in front of the right side of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, VB i is the actual driving behavior of the i-th target vehicle, DS i is the driving style of the driver of the i-th target vehicle, χ t Indicates that the sliding average of the input data is used as the long-term feature, Padding means that when the input data size is inconsistent, the input data is padded, Avgpool means the sliding average process, χ s It represents the volatility characteristic of the input data minus its sliding mean.

[0030] Optionally, the loss function corresponding to the total loss is:

[0031]

[0032] Among them, R ρ is the loss function corresponding to the total loss, ρ is a non-negative parameter used to control the overlapping area between different driving style clustering data, δ ik is the indicator function. When the i-th sample belongs to the k-th class, δ ik The value is 1, otherwise the value is 0. k is the dissimilarity function between the clustering data of each vehicle’s driving style, a is the weight parameter of the driving behavior prediction task, and B k is the cross entropy loss function between the actual driving behavior and the predicted driving behavior, b is the weight parameter of the driving trajectory prediction task, T k is the deviation loss between the true future trajectory and the predicted future trajectory, is the historical trajectory of the i-th target vehicle in the i-th scenario, m k Indicates the location of the cluster center, VB ij is the jth type of real driving behavior of the i-th target vehicle, p j For VB ij The corresponding j-th predicted driving behavior, C is the number of driving behavior categories, pre is the prediction time window, t is the target time, is the true future trajectory of the i-th target vehicle at time h, for The corresponding predicted future trajectory of the i-th target vehicle at time h.

[0033] Optionally, the loss function corresponding to the total loss also includes a clustering gap penalty term, and the objective function is:

[0034]

[0035] Among them, ε>0 is the regularization parameter, δ is the current driving style clustering solution, This is the previous driving style clustering solution.

[0036] This specification provides a vehicle trajectory prediction device, comprising:

[0037] The acquisition module is used to obtain the historical trajectories of multiple vehicles before the target time in the same scene, as well as the actual driving behavior and actual future trajectory of each vehicle after the target time;

[0038] The clustering module is used to cluster the historical trajectories of each vehicle through a clustering algorithm model to obtain multi-class driving style clustering data;

[0039] A driving behavior prediction module is used to input the historical trajectory and multi-class driving style clustering data of each vehicle into a support vector machine as a vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style;

[0040] The vehicle trajectory prediction module is used to input the vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle into the neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period;

[0041] The joint optimization module is used to construct the total loss according to the dissimilarity between the clustering data of each vehicle's driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, and to perform joint optimization training on the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model with the minimum total loss as the optimization goal; and to predict the vehicle trajectory through the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model after the joint optimization training.

[0042] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle trajectory prediction method is implemented.

[0043] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above-mentioned vehicle trajectory prediction method is implemented when the processor executes the program.

[0044] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0045] When considering the impact of driving style on vehicle driving trajectory, the present invention does not directly specify the division of driving style, but clusters based on the historical trajectory of each vehicle, and performs subsequent driving behavior prediction under the corresponding driving style based on the obtained multi-class driving style clustering data, as well as the future trajectory prediction of each vehicle. Finally, the total loss is constructed according to the dissimilarity between the driving style clustering data of each vehicle, the deviation between the real driving behavior and the predicted driving behavior, and the deviation between the real future trajectory and the predicted future trajectory, and the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model are jointly optimized and trained with the minimum total loss as the optimization goal. In this way, the driving style clustering is continuously adjusted, the driving style clustering process is optimized, and the subsequent vehicle behavior recognition model and trajectory prediction model are optimized at the same time, which reduces the transmission of erroneous information in the entire vehicle trajectory prediction process and improves the vehicle trajectory prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A flow chart of a vehicle trajectory prediction method provided in this specification;

[0048] Figure 2 A trajectory prediction demonstration diagram provided in this manual;

[0049] Figure 3 A schematic diagram of vehicle trajectory prediction and model training provided in this manual;

[0050] Figure 4 A schematic diagram of an Autoformer structure and vehicle trajectory prediction process provided in this manual;

[0051] Figure 5 A schematic diagram of a vehicle trajectory prediction device provided in this specification;

[0052] Figure 6 A schematic diagram of a computer device for implementing a vehicle trajectory prediction method provided in this specification. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] At present, vehicle trajectory prediction has attracted extensive academic research attention due to its broad application prospects and significant benefits. Initially, some studies used some assumptions such as constant speed or constant acceleration to construct dynamic and kinematic models to predict the future movement of vehicles. The models were limited by the assumptions and mostly performed short-term vehicle trajectory prediction. With the development of deep learning models, vehicle trajectory prediction models are divided into vehicle trajectory prediction models based on convolutional neural networks (CNN), recursive neural networks (RNN), attention mechanisms (AM), graph neural networks (GNN), generative adversarial networks (GAN), etc. according to different usage methods. Deep learning models can learn the spatiotemporal dependencies between vehicles, show good performance in the field of vehicle trajectory prediction, and are suitable for long-term vehicle trajectory prediction. However, most deep learning-based models do not consider the heterogeneity of vehicle trajectory datasets. Vehicle trajectory data itself is diverse. Different styles of human drivers, different driving behaviors or different types of vehicles, different roads, and weather will lead to heterogeneity of datasets, which in turn affects the effectiveness of model use.

[0055] Recently, some studies have begun to focus on vehicle trajectory prediction for drivers with different styles. The results show that considering different driver styles is conducive to reducing the heterogeneity of the data set and improving the accuracy of vehicle trajectory prediction. However, most current studies adopt a sequential structure, that is, first divide the driver style, then build a driving style recognition model, and finally build a vehicle trajectory prediction model. This sequential structure may lead to the transmission of erroneous information, that is, the accuracy of style division and recognition directly affects the accuracy of vehicle trajectory prediction.

[0056] The purpose of the present invention is to improve the accuracy of vehicle trajectory prediction while finding the optimal driver style division and vehicle behavior recognition results. In order to reduce the heterogeneity of vehicle trajectory data and solve the problem that the sequential structure may cause incorrect information transmission, the present invention refers to Cluster-Aware Supervised Learning (CluSL) and proposes a joint optimization structure called Clu-SVM-Autoformer. This structure takes into account different driver styles and combines driving style with vehicle behavior to reduce the heterogeneity of the data set, improve the accuracy of vehicle trajectory prediction, and find the optimal driver style division and recognition results. This structure uses K-means++ to realize driver style clustering, uses support vector machine (SVM) to realize vehicle behavior recognition, and combines Autoformer to realize vehicle trajectory prediction. Finally, the driver style clustering, vehicle behavior recognition and vehicle trajectory results are optimized simultaneously through a joint optimization method.

[0057] The main purposes of the present invention are as follows:

[0058] (1) Not only does it focus on driver style research, but it also focuses on the heterogeneity of vehicle trajectory datasets caused by factors such as different vehicle behaviors, and reduces the heterogeneity of vehicle trajectory datasets through clustering, preprocessing and other methods.

[0059] (2) A joint optimization structure is proposed to improve the accuracy of vehicle trajectory prediction while avoiding the transmission of erroneous information caused by the use of a sequential structure.

[0060] (3) Autoformer is used to extract long-term and volatility information from the vehicle’s historical driving data and surrounding environment data for vehicle trajectory prediction.

[0061] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0062] Figure 1 This is a flow chart of a vehicle trajectory prediction method in this specification, which specifically includes the following steps:

[0063] S101: Obtain historical trajectories of multiple vehicles before a target time in the same scene, as well as the actual driving behavior and actual future trajectory of each vehicle after the target time.

[0064] Generally, before predicting vehicle trajectories, the server of the business platform can first train the model to be used, and then perform practical applications through the trained model.

[0065] Based on this, the server can first obtain sample data for training. Specifically, the server can obtain the pre-collected historical trajectories of multiple vehicles in the same scene before the target moment as sample data, as well as the actual driving behavior and actual future trajectory of each vehicle after the target moment as annotations.

[0066] The same scenario mentioned here may refer to the same road section based on a preset road section length, and the road section may be a straight road section, an intersection road section, etc. In this road section scenario, there is usually a direct or indirect impact between the vehicles traveling at the same time, and the driving behaviors and driving trajectories of the vehicles usually have a certain correlation.

[0067] Of course, for multiple vehicles in the scene, the server can also use one of them as the target vehicle, and at least some of the remaining vehicles as other vehicles around the target vehicle, so as to match the actual application. The target vehicle here can refer to the vehicle whose future driving trajectory the unmanned vehicle needs to predict. When it is actually applied, the unmanned vehicle needs to predict the future driving trajectory of each surrounding vehicle in order to perform lateral and longitudinal planning, then each vehicle to be predicted around it can be used as a target vehicle. When the unmanned vehicle predicts the future driving trajectory of the target vehicle, it can be understood that the driving trajectory of the vehicles around the target vehicle will have an impact on the driving trajectory of the target vehicle. Therefore, the trajectory prediction model can predict the future trajectory of the target vehicle based on the historical trajectory of the target vehicle and the historical trajectory of the vehicles around the target vehicle. Figure 2 shown.

[0068] Figure 2 This is a schematic diagram of a trajectory prediction demonstration in this manual. Figure 2 It can be seen that in this road scene, 7 vehicles are given as an example, among which red is the target vehicle, blue is the surrounding vehicles of the target vehicle, the upper figure shows the vehicle historical trajectory, the middle figure shows the driving style clustering and driving behavior recognition results, and the lower figure shows the vehicle trajectory prediction of the target vehicle.

[0069] The server mentioned in this specification can be a server set up on a business platform, or a device such as a desktop computer, a laptop computer, etc. that can execute the solution of this specification. For the convenience of description, the following description is based on the server as the execution subject.

[0070] S102: Clustering the historical trajectories of each vehicle using a clustering algorithm model to obtain multiple types of driving style clustering data.

[0071] S103: Inputting the historical trajectory of each vehicle and the multi-class driving style clustering data into a support vector machine as a vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style.

[0072] S104: The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input into a neural network model of a trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period.

[0073] After obtaining the sample data as described above, the server can implement the method in this manual to predict vehicle trajectories and train the model through the model. Figure 3 , Figure 3 This is a schematic diagram of vehicle trajectory prediction and model training in this specification.

[0074] First, the server can perform driver style clustering, input vehicle trajectory data, and use the K-means++ model to perform initial driver style clustering because it can efficiently initialize cluster centers and is applicable to large data sets, so as to better distinguish drivers' driving styles and output driver style clustering data.

[0075] The vehicle trajectory data is as follows: Figure 2 As shown in Figure 2, the driving trajectories of surrounding vehicles have an important impact on the driving decision and trajectory prediction of the target vehicle. Assume that (x i ,y i ) t represents the vehicle position of the i-th vehicle at time t, then the historical driving data of the i-th target vehicle at time t can be defined as:

[0076]

[0077] In the formula, represents the historical driving data of the i-th target vehicle, N represents the number of target vehicles, Δt represents the time window, his represents the historical time window, is the vehicle position of the i-th target vehicle at time t, which can be 2s in the present invention.

[0078] The historical trajectory data of the surrounding vehicles from the i-th target vehicle is as follows:

[0079]

[0080] in, is the historical trajectory of the surrounding vehicles of the i-th target vehicle, is the historical trajectory of the vehicle in front of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, is the historical trajectory of the vehicle in front of the left side of the i-th target vehicle, is the historical trajectory of the vehicle behind the left side of the i-th target vehicle, is the historical trajectory of the vehicle in front of the right side of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, and N represents the number of target vehicles.

[0081] In the clustering of driving styles, specifically, studies have shown that the driver's aggressiveness level is positively correlated with the vehicle's speed variation coefficient, deceleration variation, and average deceleration. The classification of driver styles involves three driving parameters: longitudinal acceleration, lateral acceleration, and speed. In the present invention, based on the clustering study of driver styles of speed variation coefficient, average vehicle speed, and average acceleration, the driver styles are divided into K categories.

[0082] The speed variation coefficient of each vehicle is determined by the following formula:

[0083]

[0084] Among them, V c is the speed variation coefficient of the vehicle, N is the number of historical trajectory data of the vehicle, is the average speed of the vehicle, V i is the instantaneous speed of the vehicle.

[0085] The speed variation coefficient, average speed and average acceleration corresponding to the historical trajectory of each vehicle are clustered using the K-means++ model to obtain multi-class driving style clustering data.

[0086] Secondly, the server can perform driving behavior recognition, input vehicle historical trajectory and driving style clustering data, and can use the model to discover the impact of different driving styles on driving behavior, so as to better understand the vehicle behavior rules. Specifically, in one or more embodiments of this specification, a support vector machine (SVM) can be selected for driving behavior recognition. SVM has good performance in processing high-dimensional data and nonlinear classification problems, and can effectively identify complex driving behavior patterns. Output vehicle behavior recognition results. The threshold method can identify different driving behaviors by establishing specific thresholds, such as Figure 3 , such as Category 1 lane change or Category 1 straight ahead, without the need for manual marking.

[0087] Specifically, the optimal hyperplane can be determined by the following formula, which maximizes the distance from the closest vehicle historical trajectory in different driving behaviors to the optimal hyperplane:

[0088]

[0089] Where w and b are hyperplane parameters, C is the penalty coefficient, ξ i is a slack variable, x is the vehicle historical trajectory data, y is the driving behavior corresponding to the vehicle historical trajectory data, n is the number of samples,

[0090] That is, this hyperplane is selected to maximize the distance between the closest vehicle historical trajectory data in two different driving behaviors (e.g., lane changing, lane keeping) and the hyperplane.

[0091] Then, the SMO (Sequential Minimal Optimization) algorithm can be used to solve the support vector. The basic idea of ​​the SMO algorithm is to continuously select a pair of Lagrange multipliers for optimization until the optimal solution is reached, including:

[0092]

[0093] Where a is the coefficient of the solution parameter, which is the Lagrange multiplier introduced to solve the SVM problem, σ1 is the standard deviation of x, and σ2 is the standard deviation of w.

[0094] After the support vector is obtained, the decision function can be calculated based on the support vector. The decision function is used to determine the predicted driving behavior under the corresponding driving style according to the historical trajectory of each vehicle and the multi-category driving style clustering data.

[0095] Next, the server can predict the vehicle trajectory, input the vehicle's historical trajectory, driving style clustering data, and driving behavior recognition results, and complete the vehicle trajectory prediction for a long period of time through the Autoformer model. The Autoformer model can extract long-term and volatility information from the data and can effectively predict the vehicle trajectory. Output the future driving trajectory of the target vehicle. The future vehicle trajectory is the future driving path obtained using the prediction model. The predicted trajectory of the i-th target vehicle after time t can be defined as:

[0096]

[0097] Where P i TV represents the predicted future trajectory of the i-th target vehicle, N represents the number of target vehicles, Δt represents the time window, and pre represents the prediction time window, which can be 5s in the present invention.

[0098] Specifically, the purpose of vehicle trajectory prediction research is to make the prediction results closer to the ground-truth trajectory data. Improving the accuracy of vehicle trajectory prediction can help vehicles make early warnings of danger and make planning decisions. Given the vehicle's historical driving data and surrounding environment data, Autoformer can complete vehicle trajectory prediction for a long period of time. Autoformer consists of an internal sequence decomposition block, an autocorrelation mechanism, a feedforward neural network, and corresponding encoders and decoders. Figure 4 As shown, Figure 4 This is a schematic diagram of an Autoformer structure and vehicle trajectory prediction process in this specification.

[0099] The server can use the historical trajectory, driving style, and predicted driving behavior of each vehicle in the scene as inputs to the Autoformer model of the trajectory prediction model, and extract the long-term and volatility characteristics of the vehicle surrounding environment data and the historical trajectory of the vehicle based on the following formula through the internal sequence decomposition block of the Autoformer model:

[0100]

[0101] χ t =AvgPool(Padding(χ))

[0102] χ s = x-x t

[0103] Among them, χ is the input data, is the surrounding environment data of the i-th target vehicle, VB i is the actual driving behavior of the i-th target vehicle, DS i is the driving style of the i-th target vehicle, χ t Indicates that the sliding average of the input data is used as the long-term feature, Padding means that when the input data size is inconsistent, the input data is padded, Avgpool means the sliding average process, χ s It represents the volatility characteristic of the input data minus its sliding mean.

[0104] The encoder is specifically used to model fluctuation information. The fluctuation information contained in the output data is used as additional data to help the decoder improve the prediction results. The encoder process is as follows: First, the input data such as historical driving data of the original vehicle, driver style and vehicle behavior are processed by autocorrelation to obtain relevant information. Secondly, this relevant information is combined with the input data, and an internal sequence decomposition block is used to eliminate the long-term information contained in the input data. Then, the long-term information contained in the input data is eliminated again through the feedforward network. Finally, the fluctuation information contained in the input data is passed to the decoder.

[0105] The running trajectory of the vehicle is determined by the decoder, which is mainly divided into two components: one for processing the fluctuation information contained in the input data, and the other for processing the long-term information contained in the input data. In the fluctuation information processing part, the input data is combined with the fluctuation information transmitted by the encoder, and processed by modules such as the autocorrelation module, the internal sequence decomposition block, and the feedforward neural network to obtain the prediction results of the fluctuation information. In the long-term information processing part, the input data is aggregated with the long-term information output by the internal sequence decomposition block multiple times to obtain the prediction results of the long-term information. The running trajectory of the vehicle is determined by combining the prediction results of the fluctuation and long-term information.

[0106] The autocorrelation mechanism calculates the autocorrelation between positions in the sequence data, allowing the Autoformer to better understand the relationship between different positions. This helps to more accurately model long-term dependencies in sequence data. The input data is processed by the projection layer, which maps it into query (Q), key (K), and value (V) vectors to calculate the correlation and perform a weighted sum. The autocorrelation mechanism uses a multi-head attention mechanism to produce results. Assume that there are h heads in the multi-head attention mechanism:

[0107] head i =Auto-Correlation(Q i ,K i ,V i )

[0108] MultiHead(Q,K,V)=W output *Concat(head1,…,head h )

[0109] Auto-Correlation refers to the autocorrelation mechanism of the i-th head, which can obtain different aspects and features of the input sequence, as well as relevant information between different positions. Concat refers to merging the attention results from multi-head attention calculations to achieve subsequent linear transformations. The weight matrix W output Used to linearly transform the results of multi-head attention to generate the final output. MultiHead(Q,K,V) represents the output of the autocorrelation mechanism, which captures the relevant information between different positions in the input data and effectively represents the important information in the input data.

[0110] The nonlinear transformation is applied to the features of each position through the Feedforward Neural Network (FNN), aiming to more effectively capture the semantic information between different positions. Setting the FNN layer enhances the model's ability to represent input data.

[0111] S105: constructing a total loss according to the dissimilarity between the clustering data of each vehicle driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, and jointly optimizing and training the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model with the minimum total loss as the optimization goal; and predicting the vehicle trajectory by using the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model after the joint optimization training.

[0112] After obtaining the predicted future trajectory of each vehicle through the above, the server can perform model optimization training based on this. Specifically, the joint optimization structure can continuously update the driving style clustering, driving style and driving behavior recognition, and vehicle trajectory prediction results until the target error is minimized.

[0113] In the joint optimization task of the present invention, CluSL is used to find the best clustering of driver styles and minimize the sum of the loss functions within each cluster, where the loss function includes the loss of vehicle behavior recognition and the loss of vehicle trajectory prediction. By solving the objective function of the joint optimization, the optimal driving style division result can be found while improving the accuracy of vehicle trajectory prediction and driving behavior recognition. Through CLUSL, the joint optimization of driving style clustering, driving behavior recognition and vehicle trajectory prediction is achieved.

[0114] The present invention includes training samples of N vehicles. Assuming that the driver style has K categories, the driving style of the i-th vehicle can be defined as DS i = k∈[K]:={1,2,…,K}, the joint optimization framework of the present invention is as follows:

[0115]

[0116] In the formula, R ρ is the loss function corresponding to the total loss, ρ is a non-negative parameter used to control the overlapping area between different driving style clustering data, δ ik is the indicator function. When the i-th sample belongs to the k-th class, δ ik The value is 1, otherwise the value is 0. k is the dissimilarity function between the clustering data of each vehicle’s driving style, a is the weight parameter of the driving behavior prediction task, and B k is the cross entropy loss function between the actual driving behavior and the predicted driving behavior, b is the weight parameter of the driving trajectory prediction task, T k is the deviation loss between the true future trajectory and the predicted future trajectory.

[0117] The dissimilarity function selected by the present invention is shown in the following formula:

[0118]

[0119] In the formula, m k represents the location of the cluster center, S k It is the Euclidean distance. In cluster analysis, by calculating the Euclidean distance from each data point to the cluster center, it helps to determine the degree of similarity between the data point and the cluster center, so as to perform effective clustering.

[0120]

[0121] In the formula, VBij is the jth type of real driving behavior of the i-th target vehicle, p j For VB ij The corresponding j-th predicted driving behavior, C is the number of driving behavior categories. In the driving behavior recognition task, B k It is a cross entropy loss function, which can effectively measure the difference between the actual category distribution and the model predicted category distribution. The smaller the index is, the better the recognition ability of the model is. Clu-SVM combined with SVM can achieve better recognition effect. The present invention selects to use SVM as the basic recognition model of driving behavior and uses CluSL to improve the recognition accuracy of driving behavior.

[0122]

[0123] In the formula, t is the target time, is the true future trajectory of the i-th target vehicle at time h, for The predicted future trajectory of the corresponding target vehicle i at time h. In the vehicle trajectory prediction task, T k The mean absolute error represents the average absolute difference between the predicted value and the true value. The smaller the index is, the better the prediction ability of the model is. The present invention selects to use Autoformer as the basic recognition model of driving behavior and uses CluSL to improve the accuracy of vehicle trajectory.

[0124] The Clu-SVM-Autoformer is solved by the Regularized Alternating Minimization (RAM) algorithm. A penalty term can also be added to the objective function to ensure that the difference between the current clustering solution and the previous clustering solution is not too large, as shown in the following formula:

[0125]

[0126] Where ε>0 is the regularization parameter, δ is the current driving style clustering solution, This is the previous driving style clustering solution.

[0127] The CluSL solution process is shown in Algorithm Table 1. In the first step, the historical trajectory of each vehicle is input, and the initial clustering result of driving style is obtained through the K-means++ model. In the second step, the driving style clustering data and the historical trajectory of the vehicle are passed on. In the third step, the loss function corresponding to the total loss is solved to obtain the optimal solution m,B of this step. k ,T k, which correspond to the results of driving style clustering, driving behavior recognition and vehicle trajectory prediction respectively; the fourth step is to solve through the RAM algorithm and update the driving style clustering result; the fifth step is to pass the new driving style clustering result back to the second step; when the new driving style clustering result is the same as the previous driving style clustering result, it means that the result tends to be stable, and the final result is output.

[0128] Table 1 Schematic diagram of solution process

[0129]

[0130] based on Figure 1 The vehicle trajectory prediction method shown in the present invention does not directly specify the division of driving styles when considering the influence of driving styles on vehicle driving trajectories, but clusters the vehicles based on their historical trajectories, and performs subsequent driving behavior prediction under the corresponding driving styles based on the obtained multi-class driving style clustering data, as well as the future trajectory prediction of each vehicle. Finally, the total loss is constructed according to the dissimilarity between the driving style clustering data of each vehicle, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, and the clustering algorithm model, the vehicle behavior recognition model, and the trajectory prediction model are jointly optimized and trained with the minimum total loss as the optimization goal. Thus, the driving style clustering is continuously adjusted, the driving style clustering process is optimized, and the subsequent vehicle behavior recognition model and trajectory prediction model are optimized at the same time, which reduces the transmission of erroneous information in the entire vehicle trajectory prediction process and improves the vehicle trajectory prediction accuracy.

[0131] The beneficial effects of the present invention are as follows:

[0132] (1) The present invention not only focuses on the study of driver style, but also on different vehicle behaviors, which can provide a reference for energy conservation, emission reduction and traffic safety; and can reduce the heterogeneity of vehicle trajectory datasets, providing more accurate data support for vehicle trajectory prediction.

[0133] (2) A joint optimization structure is adopted to avoid the transmission of erroneous information caused by the sequential structure, thereby improving the accuracy of vehicle trajectory prediction.

[0134] (3) Autoformer is used to extract long-term and volatility information from the vehicle's historical driving data and surrounding environment data for vehicle trajectory prediction. The present invention can predict the vehicle's driving trajectory for the next 5 seconds based on the target vehicle's 2 seconds of historical driving data and surrounding environment data, and can help autonomous driving vehicles to make early warnings of dangers and make planning decisions.

[0135] (4) No manual labeling is required. The vehicle’s historical trajectory data and surrounding environment data are input, and the final output includes data such as driving style, driving behavior, and vehicle future trajectory. This means that autonomous vehicles can perceive the different driving styles of human drivers, judge their behavior, predict their future trajectory, and make safe and efficient plans and decisions based on this information.

[0136] When applying the vehicle trajectory prediction method provided in this manual, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and this manual does not limit this.

[0137] The above is a vehicle trajectory prediction method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding vehicle trajectory prediction device, such as Figure 5 shown.

[0138] Figure 5 A schematic diagram of a vehicle trajectory prediction device provided in this specification includes:

[0139] The acquisition module 201 is used to acquire the historical trajectories of multiple vehicles before the target time in the same scene, as well as the actual driving behavior and the actual future trajectory of each vehicle after the target time;

[0140] A clustering module 202 is used to cluster the historical trajectories of each vehicle through a clustering algorithm model to obtain multiple types of driving style clustering data;

[0141] A driving behavior prediction module 203 is used to input the historical trajectory and multi-class driving style clustering data of each vehicle into a support vector machine as a vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style;

[0142] The vehicle trajectory prediction module 204 is used to input the vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle into a neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period;

[0143] The joint optimization module 205 is used to construct a total loss according to the dissimilarity between the clustering data of each vehicle driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, and to perform joint optimization training on the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model with the minimum total loss as the optimization goal; and to predict the vehicle trajectory through the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model after the joint optimization training.

[0144] For the specific definition of the vehicle trajectory prediction device, please refer to the definition of the vehicle trajectory prediction method above, which will not be repeated here. Each module in the above-mentioned vehicle trajectory prediction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0145] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 The vehicle trajectory prediction method is provided.

[0146] This manual also provides Figure 6 The structural diagram of the computer device shown in FIG. Figure 6 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The vehicle trajectory prediction method is provided.

[0147] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0148] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A vehicle trajectory prediction method, characterized in that: include: Obtain the historical trajectories of multiple vehicles in the same scenario before the target time, as well as the actual driving behavior and actual future trajectory of each vehicle after the target time; The historical trajectories of each vehicle are clustered through a clustering algorithm model to obtain multi-class driving style clustering data; The historical trajectory and multi-class driving style clustering data of each vehicle are input into the support vector machine as the vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style; The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input into the neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period; According to the dissimilarity between the clustering data of each vehicle's driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, the total loss is constructed, and the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model are jointly optimized and trained with the minimum total loss as the optimization goal; and the vehicle trajectory is predicted by the clustering algorithm model, the vehicle behavior recognition model and the trajectory prediction model after the joint optimization training; The loss function corresponding to the total loss is: Among them, R ρ is the loss function corresponding to the total loss, ρ is a non-negative parameter used to control the overlapping area between different driving style clustering data, δ ik is the indicator function. When the i-th sample belongs to the k-th class, δ ik The value is 1, otherwise the value is 0. k is the dissimilarity function between the clustering data of each vehicle’s driving style, a is the weight parameter of the driving behavior prediction task, and B k is the cross entropy loss function between the actual driving behavior and the predicted driving behavior, b is the weight parameter of the driving trajectory prediction task, T k is the deviation loss between the true future trajectory and the predicted future trajectory, is the historical trajectory of the i-th target vehicle in the i-th scenario, m k Indicates the location of the cluster center, VB ij is the jth type of real driving behavior of the i-th target vehicle, p j For VB ij The corresponding j-th predicted driving behavior, C is the number of driving behavior categories, pre is the prediction time window, t is the target time, is the true future trajectory of the i-th target vehicle at time h, for The corresponding predicted future trajectory of the i-th target vehicle at time h.

2. The vehicle trajectory prediction method according to claim 1, characterized in that: The historical trajectories of each vehicle are clustered by the clustering algorithm model to obtain multiple types of driving style clustering data, specifically including: The speed variation coefficient of each vehicle is determined by the following formula: The speed variation coefficient, average speed and average acceleration corresponding to the historical trajectory of each vehicle are clustered through the K-means++ model to obtain multi-class driving style clustering data; Among them, V c is the speed variation coefficient of the vehicle, N is the number of historical trajectory data of the vehicle, is the average speed of the vehicle, V i is the instantaneous speed of the vehicle.

3. The vehicle trajectory prediction method according to claim 1, characterized in that: The determining of the predicted driving behavior of each vehicle under the corresponding driving style specifically includes: The optimal hyperplane is determined by the following formula, where the optimal hyperplane maximizes the distance from the closest vehicle historical trajectory in different driving behaviors to the optimal hyperplane: The support vector is obtained by solving the problem, and the decision function is calculated based on the support vector. The decision function is used to determine the predicted driving behavior under the corresponding driving style according to the historical trajectory of each vehicle and the multi-class driving style clustering data; Among them, w and b are hyperplane parameters, C is the penalty coefficient, ξ i is a slack variable, x is the vehicle historical trajectory data, y is the driving behavior corresponding to the vehicle historical trajectory data, and n is the number of samples.

4. The vehicle trajectory prediction method according to claim 1, characterized in that: The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input into the neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period, specifically including: The vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle are input as the Autoformer model of the trajectory prediction model, and the autocorrelation information between the sequence data of the input data is obtained through the autocorrelation mechanism in the encoder of the trajectory prediction model; the autocorrelation information is combined with the input data, and the long-term characteristics and volatility characteristics in the input data are extracted based on the combined data according to the following formula through the internal sequence decomposition block in the encoder, and the long-term characteristics contained in the input data are eliminated; χ t = AvgPool(Padding(χ)), x s =x-x t ; The volatility characteristics of the input data are transmitted to the decoder of the trajectory prediction model, the input data is combined with the volatility characteristics, and the volatility information is predicted through the autocorrelation module, the internal sequence decomposition block and the feedforward neural network to obtain the predicted volatility information; the input data is aggregated multiple times with the long-term information of the input data output by the internal sequence decomposition block in the decoder to obtain the predicted long-term information; the predicted future trajectory of each vehicle within a preset time period is determined by combining the predicted volatility information and the predicted long-term information; Among them, χ is the input data, is the historical trajectory of the i-th target vehicle in the i-th scenario, N is the number of target vehicles, Δt is the time window, his is the historical time window, is the vehicle position of the i-th target vehicle at time t, is the historical trajectory of the surrounding vehicles of the i-th target vehicle, is the historical trajectory of the vehicle in front of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, is the historical trajectory of the vehicle in front of the left side of the i-th target vehicle, is the historical trajectory of the vehicle behind the left side of the i-th target vehicle, is the historical trajectory of the vehicle in front of the right side of the i-th target vehicle, is the historical trajectory of the vehicle behind the i-th target vehicle, VB i is the actual driving behavior of the i-th target vehicle, DS i is the driving style of the i-th target vehicle, χ t Indicates that the sliding average of the input data is used as the long-term feature, Padding means that when the input data size is inconsistent, the input data is padded, Avgpool means the sliding average process, χ s It means that the volatility feature is the input data minus its sliding average. is the surrounding environment data of the i-th target vehicle.

5. The vehicle trajectory prediction method according to claim 1, characterized in that: The loss function corresponding to the total loss also includes a clustering gap penalty term, and the objective function is: Among them, ε>0 is the regularization parameter, δ is the current driving style clustering solution, is the previous driving style clustering solution, and m is the driving style clustering center.

6. A vehicle trajectory prediction device, characterized in that: include: The acquisition module is used to obtain the historical trajectories of multiple vehicles before the target time in the same scene, as well as the actual driving behavior and actual future trajectory of each vehicle after the target time; The clustering module is used to cluster the historical trajectories of each vehicle through a clustering algorithm model to obtain multi-class driving style clustering data; A driving behavior prediction module is used to input the historical trajectory and multi-class driving style clustering data of each vehicle into a support vector machine as a vehicle behavior recognition model to determine the predicted driving behavior of each vehicle under the corresponding driving style; The vehicle trajectory prediction module is used to input the vehicle surrounding environment data in the scene, the historical trajectory of each vehicle, the driving style of each vehicle, and the predicted driving behavior of each vehicle into the neural network model of the trajectory prediction model to determine the predicted future trajectory of each vehicle within a preset time period; A joint optimization module is used to construct a total loss based on the dissimilarity between the clustering data of each vehicle's driving style, the deviation between the actual driving behavior and the predicted driving behavior, and the deviation between the actual future trajectory and the predicted future trajectory, and to perform joint optimization training on the clustering algorithm model, the vehicle behavior recognition model, and the trajectory prediction model with the minimum total loss as the optimization goal; and to predict the vehicle trajectory through the clustering algorithm model, the vehicle behavior recognition model, and the trajectory prediction model after the joint optimization training; The loss function corresponding to the total loss is: Among them, R ρ is the loss function corresponding to the total loss, ρ is a non-negative parameter used to control the overlapping area between different driving style clustering data, δ ik is the indicator function. When the i-th sample belongs to the k-th class, δ ik The value is 1, otherwise the value is 0. k is the dissimilarity function between the clustering data of each vehicle’s driving style, a is the weight parameter of the driving behavior prediction task, and B k is the cross entropy loss function between the actual driving behavior and the predicted driving behavior, b is the weight parameter of the driving trajectory prediction task, T k is the deviation loss between the true future trajectory and the predicted future trajectory, is the historical trajectory of the i-th target vehicle in the i-th scenario, m k Indicates the location of the cluster center, VB ij is the jth type of real driving behavior of the i-th target vehicle, p j For VB ij The corresponding j-th predicted driving behavior, C is the number of driving behavior categories, pre is the prediction time window, t is the target time, is the true future trajectory of the i-th target vehicle at time h, for The corresponding predicted future trajectory of the i-th target vehicle at time h.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images