Autonomous driving vehicle trajectory prediction method and system based on dual cross Transformer
Through dual-cross Transformer technology, learning between vehicles will generate more accurate trajectory prediction and uncertainty evaluation, solving the problems of trajectory prediction accuracy and robustness in complex traffic environments in existing technology, and improving the multi-vehicle interaction scenarios predictive performance.
Patent Information
- Application Number
- CN202410529409.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-29
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-04-29
AI Technical Summary
The existing methods of autonomous driving vehicle trajectory prediction are reduced in complex traffic environments, and it is difficult to effectively predict vehicle trajectory in long-term prediction and multi-vehicle interaction scenarios. Traditional methods have low computational efficiency and narrow application range in complex scenarios.
Using a dual-cross Transformer-based method, by learning the historical trajectory data and state data of the target vehicle and surrounding vehicles, capturing the interaction between vehicles, and using technologies such as bidirectional crossing attention mechanism and multi-scale graph convolution to generate more accurate trajectories. Forecast and uncertainty evaluation, optimize the loss function to improve prediction accuracy.
Improve the accuracy and robustness of vehicle trajectory prediction in complex traffic environments, and can more accurately evaluate individual and collaboration uncertainties among vehicles, improving the performance of multi-vehicle trajectory prediction systems.
Smart Images

Figure CN118419065B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and system for predicting the trajectory of an autonomous driving vehicle, and in particular provides a method and system for predicting the trajectory of an autonomous driving vehicle based on a double cross Transformer, belonging to the technical field of autonomous driving trajectory prediction. Background Art
[0002] Autonomous vehicle trajectory prediction is a key area in autonomous driving technology, which involves predicting the movement trajectory of an autonomous vehicle over a period of time in the future. This prediction is crucial for autonomous driving systems because it helps the vehicle make more accurate decisions and avoid potential dangers or conflicts. In complex traffic environments, accurately predicting the future trajectory of a vehicle is not only affected by the complex road structure, but also by the interaction between vehicles. These interactions based on complex road environments significantly increase the difficulty of prediction. Therefore, in this environment, it is an important challenge for autonomous vehicles to achieve accurate, efficient, and reliable trajectory prediction.
[0003] At present, there are many methods for trajectory prediction in the field of autonomous driving. These methods are mainly traditional and deep learning-based methods. Traditional methods mainly rely on perfect mathematical and statistical techniques, and make predictions based on historical data and predefined models, such as physics-based methods and maneuver-based methods.
[0004] Among them, physics-based methods mainly model low-level properties, physical laws, and dynamics of vehicle motion. Generally, this method can achieve good performance in short-term (less than one second) predictions due to its use of physical laws. However, it is insufficient in long-term predictions due to the lack of consideration of high-level vehicle information.
[0005] The maneuver-based method is based on the vehicle's driving characteristics and behavior patterns. It predicts the trajectory by identifying and modeling different maneuvers that the vehicle may perform. It assumes that the future vehicle trajectory is consistent with the action intentions that the identified vehicle tends to perform. Figure 1 In general, although the maneuver-based method considers different behavior patterns and possible actions of the vehicle and can perform high-level reasoning on the vehicle, in the face of complex road environments and diverse vehicle behaviors, there are many unknown changes or difficult-to-classify maneuvers, which will reduce the accuracy of trajectory prediction; in addition, the maneuver-based method is difficult to apply to roads with other topological structures.
[0006] Therefore, traditional trajectory prediction methods are computationally efficient and easy to implement in relatively simple scenarios, but they will be limited when faced with complex and uncertain scenarios, the prediction accuracy will be reduced and the scope of application will be relatively narrow. Summary of the invention
[0007] Purpose of the invention: In view of the deficiencies in the prior art, the present invention provides a method and system for autonomous driving vehicle trajectory prediction based on a dual cross Transformer. The present invention achieves more accurate trajectory prediction and uncertainty estimation by learning the mapping relationship from input data to preliminary predicted trajectory, individual uncertainty and collaborative uncertainty, and optimizing the loss value through a loss function.
[0008] Technical solution: The autonomous driving vehicle trajectory prediction method based on dual cross Transformer includes the following steps:
[0009] S1: Collect historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles as input sequence X;
[0010] S2: Capture the interaction between the target vehicle and surrounding vehicles, output the preliminary predicted trajectories of the target vehicle and surrounding vehicles, and evaluate the individual uncertainty and collaborative uncertainty;
[0011] S3: Measure the prediction effect through the loss function and update the optimization parameters.
[0012] Preferably, S2 comprises the following steps:
[0013] S201: Position encoding the input sequence X in S1;
[0014] S202: Capture the interaction between the target vehicle and surrounding vehicles through a bidirectional cross attention mechanism to obtain the interaction feature Atten bidir (f s ,f g ), and train repeatedly;
[0015] S203: Outputting preliminary predicted trajectories of the target vehicle and its surrounding vehicles;
[0016] S204: Atten the interactive features bidir (f s ,f g ) to conduct individual uncertainty and collaborative uncertainty assessment.
[0017] Preferably, the S1 is specifically:
[0018] Historical trajectory data acquisition:
[0019] The historical trajectory data of the vehicle includes the historical trajectory points in the previous three seconds, which are collected once every ten frames per second, and the trajectory points are represented as (x, y);
[0020] Vehicle status data acquisition:
[0021] Including vehicle speed v and heading angle θ, by analyzing the trajectory, vehicle speed v and heading angle θ, the vehicle's driving behavior is judged and the trajectory of the target vehicle and surrounding vehicles is predicted;
[0022] The historical trajectory data and vehicle status data are taken as the input sequence X.
[0023] Preferably, the S201 specifically comprises: performing position encoding on the input sequence X in S1;
[0024] Use sine and cosine functions of different frequencies for position encoding, which can be expressed as:
[0025]
[0026]
[0027] Among them, pos represents the position in the input sequence X; 2k, 2k+1 represents the dimension index; d model =512, indicating the dimension of the embedding layer.
[0028] Preferably, S202 is specifically: capturing the interaction between the target vehicle and surrounding vehicles through a bidirectional cross attention mechanism, obtaining interaction features, and repeatedly training:
[0029] Convert the input sequence X into input features:
[0030] Take the input features f of the two target vehicles and surrounding vehicles from the input sequence X s ,、f g , where f s ,f g ∈X;
[0031] f s As the original input sequence, f g As the target input sequence, it is input into the bidirectional cross attention mechanism to learn the interaction relationship between the target vehicle and the surrounding vehicles:
[0032] Calculate the input feature f s Q s , K s 、V s and f g Q g , K g 、V g , the formula is as follows:
[0033] Q s =W Q f s ,K s =W Kf s ,V s =W V f s ;
[0034] Q g =W Q f g ,K g =W K f g ,V g =W V f g
[0035] Among them, W Q , W K , W V They are used to generate Q s , K s 、V s and Q g , K g 、V g The conversion parameters, d is the input feature f s And the input feature f g The dimension, d q ,d k ,d v It's Q s , K s 、V s and Q g , K g 、V g Dimension; Q s , K s 、V s The input features f are s The query (query), key (key), value (value) three vectors; Q g , K g 、V g The input features f are g The query, key, and value vectors are:
[0036] The input feature f s 、f g The interaction between the target vehicle and the surrounding vehicles is learned through the bidirectional cross-attention Transformer, and the target vehicle sequence and the surrounding vehicle sequence, as well as the preliminary interaction features Atten bidir (f s ,f g ), the specific formula is as follows:
[0037]
[0038]
[0039]
[0040]
[0041] Atten bidir (f s ,f g )=MHA(f s )+MHA(f g )
[0042] Among them, n h is the number of heads; CA s is the input feature f s The output calculated after cross attention, CA g is the input feature f g The output calculated after cross attention; MHA(f s ) for multiple CAs s The result of splicing together; MHA (f g ) for multiple CAs g The result of stitching together.
[0043] Preferably, the S203 is specifically: outputting the preliminary predicted trajectory of the target vehicle and its surrounding vehicles:
[0044] Encode the actual observed future trajectory feature Y:
[0045] E fin =E y +PE(E y )
[0046] Among them, E y is the generation feature of the actual observed future trajectory sequence y, y∈Y; E fin Represents the actual observed future trajectory characteristics after position encoding.
[0047] Perform fast Fourier transform on the mixed feature Z to get F Z :
[0048] F Z =FFT(Z)
[0049] Among them, FFT() represents the calculation of fast Fourier transform;
[0050] For the processed feature F Z Multi-scale and adaptive graph convolutions are performed to capture specific and comprehensive cross-series dependencies:
[0051]
[0052] in, Represents the mixed features of node g, where nodes s and g are abstract representations of the target vehicle and surrounding vehicles; θ k The coefficient used to scale the contribution of the k-th neighbor; α sg is the adaptive weight, which determines the contribution of node g to node s; M k is the weight matrix of the k-th neighbor; RELU is a nonlinear activation function;
[0053] Features of nodes s, g Reshape to adapt to the multi-head self-attention mechanism and use the multi-head attention mechanism to capture features The temporal dependencies, spatial relationships, contextual information, and correlations among data;
[0054] By reshaping the return, we get the input tensor adapted to the softmax function, input it into the softmax function, and optimize the network weights through the Softmax function, thus helping to stabilize the training process:
[0055]
[0056]
[0057]
[0058]
[0059]
[0060]
[0061] Among them, Reshape() represents the calculation of the reshape module; ReshapeBack() represents the reshape return calculation;
[0062] Regularization layer processing features It is then input into the two-layer perceptron MLP, which extracts and combines data from the input features to obtain the predicted future position or trajectory of the target vehicle and surrounding vehicles:
[0063]
[0064]
[0065] Among them, W1 and b1 are the weight and bias of the first layer respectively; W2 and b2 are the weight and bias of the second layer, the output y is the predicted value of the vehicle's future trajectory, and h1 is the predicted feature output by the first layer perceptron.
[0066] Preferably, the S204 is specifically: performing individual uncertainty and collaborative uncertainty evaluation on the interactive features:
[0067] Let Z = [z1,z2,...,z k ] T ,z k ∈Z, Z represents the features after interaction; f p (·) is a neural network with equal transformation; the mixed feature Z is passed into the neural network with equal transformation f p (·) to learn and generate feature f p (Z)=[z′1,z′2,...,z′ k ], f p (Z) is the feature space, where <z′ i ,z′ i >, i∈[1,k] represents the individual uncertainty σ of the target vehicle and surrounding vehicles i i ; <z′ i ,z′ j >,i,j∈[1,k],i≠j represents the cooperation uncertainty σ between the target vehicle and surrounding vehicles i and j i,j , σ i,j ∈Σ, Σ is the covariance matrix, as follows:
[0068]
[0069] Z′=f p (Z)=[z′1,z′2,...,z′ k ]
[0070] σ i =Var(z′ i )
[0071] σ i,j = Cov(z′ i ,z′ j )
[0072] Among them, MLP represents the calculation of multi-layer perceptron; MLP(z i ,z j ) is responsible for processing the input element (z i ,z j); Var() represents the calculation of variance, which is used to measure the uncertainty of a single vehicle’s behavior; Cov() represents the calculation of covariance, which is used to measure the degree of interdependence and correlation between the behaviors of two vehicles.
[0073] Preferably, the specific steps of S3 are:
[0074] S301: Define the loss function for training trajectory prediction:
[0075] l tatal = l o +l u
[0076] Among them, l o is the trajectory prediction loss; l u Assess losses for uncertainty.
[0077] S302: Input the initial predicted trajectory and the actual trajectory of the target vehicle and surrounding vehicles in S2 into the loss function to calculate the error:
[0078]
[0079] Where N is the number of samples; They represent the components of the predicted speed of the i-th vehicle in the x and y directions respectively, and the predicted speed is the derivative of the predicted position; They represent the components of the true velocity of the i-th vehicle in the x and y directions respectively. The true velocity is the derivative of the true position;
[0080] S303: Substitute the individual uncertainty σ in S2 i and collaborative uncertainty σ i,j Enter the loss function for evaluation:
[0081]
[0082] Among them, l u is the loss of the uncertainty estimator; μ i is the mean vector of the predicted i-th sample, which here refers to the initial predicted trajectory; y i is the observation vector of the true trajectory of the i-th sample; Σ is the predicted covariance matrix, which describes the correlation between the output variables, and is composed of the covariance σ i,j composition; k is the dimension of the true trajectory sequence y; N is the number of samples.
[0083] Optimal options, to alleviate the gradient vanishing or gradient exploding problem that occurs during multi-layer network training in S202, and to prevent overfitting of specific input data:
[0084] Let f′ s =MHA(fs ), f′ g =MHA(f g ), MHA(·) represents the multi-head attention mechanism; f′ s and f′ g By inputting to residual connection, layer normalization, linear layer, and Dropout, the learning ability, stability, and efficiency of the neural network are enhanced; then the interaction feature f is extracted s and f g And the summed mixed feature Z is used as the input of the subsequent network, and the intermediate feature is expressed as:
[0085] f″ s =LN(Norm(f′ s )+f s )
[0086] f″ g =LN(Norm(f′ g )+f g )
[0087] f″′ s = Dropout(f″ s ,p)
[0088] f″′ g = Dropout(f″ g ,p)
[0089] f″″ s =Norm(LN(f″′) s ))+f′ s
[0090] f″″ g =Norm(LN(f″′) g ))+f′ g
[0091]
[0092]
[0093]
[0094] Among them, LN(·) represents the linear layer, Norm(·) represents the normalization layer, Dropout(·) is the regularization layer, and p represents the probability of each neuron being retained during training.
[0095] A system for implementing a trajectory prediction method for an autonomous driving vehicle based on a double cross Transformer, comprising a collection input module, a trajectory prediction module, a loss function module, and a visualization output module connected in sequence;
[0096] The acquisition input module collects historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles;
[0097] The trajectory prediction module includes a trajectory Transformer encoder, a trajectory predictor and an uncertainty evaluator connected in sequence. The trajectory Transformer encoder learns the interaction between the target vehicle and the surrounding vehicles, and outputs the predicted trajectory through the trajectory prediction of the encoded interaction features, and is input into the uncertainty evaluator to evaluate individual uncertainty and collaborative uncertainty.
[0098] The trajectory predictor includes a decoding embedding module, a multi-scale graph module, a regularization layer, and a multi-layer perceptron module connected in sequence. The trajectory predictor receives the interactive features output by the trajectory Transformer encoder, and outputs the preliminary predicted trajectory of the target vehicle and its surrounding vehicles through the decoding embedding module, the multi-scale graph module, the regularization layer, and the multi-layer perceptron module;
[0099] The multi-scale graph module includes a fast Fourier transform module, an adaptive graph convolution module, and a multi-head attention layer connected in sequence;
[0100] The loss function module minimizes the loss generated by the trajectory predictor and the loss generated by the uncertainty estimator by defining a loss function;
[0101] The visualization output module converts the training results into predictions of future target vehicle and surrounding vehicle trajectories, and visualizes the prediction results.
[0102] The present invention constructs an input sequence X, and takes the historical trajectory data and state information of the target vehicle and surrounding vehicles as input; secondly, the trajectory prediction module learns the mapping relationship from the input data to the preliminary predicted trajectory, individual uncertainty and collaborative uncertainty, so as to achieve more accurate trajectory prediction and uncertainty estimation; then, the error and uncertainty estimation between the preliminary prediction result and the real trajectory are passed to the loss function module, and the loss function module adjusts the parameters in the neural network through the back propagation process to minimize the loss value, thereby improving the accuracy of the prediction and the performance of the model; finally, the final prediction result is output through the visualization output module.
[0103] Beneficial effects: The present invention performs simulation training based on multiple vehicles in a vehicle-road cooperative environment, fully considering the interaction between the target vehicle and the surrounding vehicles. At the same time, an uncertainty evaluator is introduced to evaluate the individual and collaborative uncertainties introduced by vehicle interactions. The uncertainty evaluator uses an equal transformation neural network, which can capture the interaction between each vehicle in the input data and maintain invariance when arranging the input data and generating features; in addition, the equal transformation neural network generates features to more accurately obtain individual and collaborative uncertainties in the feature space, thereby improving the estimation accuracy of these uncertainties. The uncertainty evaluator can accurately evaluate uncertainty in a multi-vehicle trajectory prediction system, especially in a system based on a collaborative model, which will greatly improve the performance of the multi-vehicle trajectory prediction system;
[0104] Establish a trajectory Transformer encoder that learns trajectory sequences based on a bidirectional cross-attention Transformer architecture. The bidirectional cross-attention mechanism captures the mutual influence between different vehicles by learning the spatiotemporal dependencies between the vehicle and the surrounding vehicles, and then learns the interaction between vehicles. In addition, the bidirectional cross-attention mechanism helps the model reduce the uncertainty of prediction by dynamically adjusting the focus of attention, thereby achieving more accurate and robust trajectory prediction of the target vehicle and its surrounding vehicles;
[0105] A trajectory predictor is established, which is mainly composed of a multi-scale graph module. The multi-scale module mainly solves the cross-sequence correlation problem in multi-scale trajectory sequence data through three key components. First, it uses the fast Fourier transform method to project the original trajectory sequence data into the space associated with the most significant time scale, so as to effectively capture and represent various cross-sequence correlations exhibited at different time scales. Secondly, it introduces multiple adaptive graph convolution modules, combined with a learnable adjacency matrix, to capture cross-sequence correlations at different time scales. Finally, the multi-scale graph module also integrates a multi-head self-attention mechanism, which can capture the temporal pattern within the data, and combines it with an adaptive graph convolution module to effectively capture and represent cross-sequence and internal sequence correlations at multiple scales. Therefore, the multi-scale graph module can improve model performance while reducing computational complexity;
[0106] A loss function module is established, covering the losses generated by the trajectory predictor and the uncertainty evaluator. The loss generated by the trajectory predictor is measured by calculating the Euclidean distance difference between the predicted speed and the true speed. Since speed is the derivative of position with respect to time, the difference between the speeds can be used to check the smoothness and continuity of the trajectory. Smaller speed changes correspond to smoother trajectories, while larger speed changes indicate discontinuities or sudden changes in the trajectory. In the uncertainty evaluator, a log-likelihood loss function is used to generate a distribution representing the confidence or credibility of the prediction. Such a log-likelihood loss function helps the model more accurately assess the uncertainty of vehicle interactions, thereby improving the accuracy and robustness of trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0107] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0108] Figure 1 is a flow chart of the method of the present invention;
[0109] Figure 2 It is the system flow chart;
[0110] Figure 3 This is the action diagram of the overall trajectory prediction module;
[0111] Figure 4 This is the structure diagram of the trajectory Transformer encoder;
[0112] Figure 5 This is the working diagram of the bidirectional cross-attention Transformer module;
[0113] Figure 6 It is the structure diagram of trajectory predictor;
[0114] Figure 7 It is the action diagram of the multi-scale graph module;
[0115] Figure 8 It is the system structure diagram;
[0116] Fig. 9 is the action diagram of the uncertainty estimator;
[0117] Fig.10 It is the interaction diagram between the target vehicle and surrounding vehicles;
[0118] Fig.11 Visualization of the prediction results for the target vehicle and surrounding vehicles;
[0119] Fig.12 This is a diagram showing the application of the present invention;
[0120] Fig.13 This is an example diagram of an application scenario of the present invention. DETAILED DESCRIPTION
[0121] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0122] In the description of the present invention, it is necessary to understand that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0123] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.
[0124] like Figure 1 As shown, the autonomous driving vehicle trajectory prediction method based on double cross Transformer includes the following steps:
[0125] S1: Collect historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles as input sequence X:
[0126] Historical trajectory data acquisition:
[0127] The historical trajectory data of the vehicle includes the historical trajectory points in the previous three seconds, which are collected once every ten frames per second, and the trajectory points are represented as (x, y);
[0128] Vehicle status data acquisition:
[0129] Including vehicle speed v and heading angle θ, by analyzing the trajectory, vehicle speed v and heading angle θ, the vehicle's driving behavior is judged and the trajectory of the target vehicle and surrounding vehicles is predicted;
[0130] The historical trajectory data and vehicle status data are taken as the input sequence X.
[0131] S2: capturing the interaction between the target vehicle and surrounding vehicles, outputting preliminary predicted trajectories of the target vehicle and surrounding vehicles, and evaluating individual uncertainty and collaborative uncertainty; S2 includes the following steps:
[0132] S201: Positionally encode the input sequence X in S1:
[0133] Use sine and cosine functions of different frequencies for position encoding, which can be expressed as:
[0134]
[0135]
[0136] Among them, pos represents the position in the input sequence X; 2k, 2k+1 represents the dimension index; d model =512, indicating the dimension of the embedding layer.
[0137] S202: In order to increase the interaction between the target vehicle and the surrounding vehicle trajectory sequences, the interaction between the target vehicle and the surrounding vehicles is captured through a bidirectional cross attention mechanism to obtain the interaction feature Atten bidir (f s ,f g ), and repeatedly trained; the bidirectional cross attention mechanism is a variant of the self-attention mechanism, which allows learning complex spatiotemporal dependencies between vehicles in the input sequence, capturing the interactions between vehicles, and thus increasing the information representation between vehicles. The bidirectional cross attention mechanism can extract the interaction features between the target vehicle and the surrounding vehicles through the combination and interaction of the input sequences between different target vehicles and surrounding vehicles, providing more efficient learning capabilities, thereby improving the accuracy and robustness of trajectory prediction;
[0138] Convert the input sequence X into input features:
[0139] Take the input features f of the two target vehicles and surrounding vehicles from the input sequence X s ,、f g , where f s ,f g ∈X;
[0140] f s As the original input sequence, f gAs the target input sequence, it is input into the bidirectional cross attention mechanism to learn the interaction relationship between the target vehicle and the surrounding vehicles:
[0141] Calculate the input feature f s Q s , K s 、V s and f g Q g , K g 、V g , the formula is as follows:
[0142] Q s =W Q f s ,K s =W K f s ,V s =W V f s ;
[0143] Q g =W Q f g ,K g =W K f g ,V g =W V f g
[0144] Among them, W Q , W K , W V They are used to generate Q s , K s 、V s and Q g , K g 、V g The conversion parameters, d is the input feature f s And the input feature f g The dimension, d q d k d v It's Q s , K s 、V s and Q g , K g 、V g Dimension; Q s , K s 、V s The input features f are sThe query (query), key (key), value (value) three vectors; Q g , K g 、V g The input features f are g The query, key, and value vectors are:
[0145] The input feature f s 、f g The interaction between the target vehicle and the surrounding vehicles is learned through the bidirectional cross-attention Transformer, and the target vehicle sequence and the surrounding vehicle sequence, as well as the preliminary interaction features Atten bidir (f s ,f g ), the specific formula is as follows:
[0146]
[0147]
[0148]
[0149]
[0150] Atten bidir (f s ,f g )=MHA(f s )+MHA(f g )
[0151] Among them, n h is the number of heads, where n h =8; The purpose of CA is to improve the numerical stability of the softmax function; s is the input feature f s The output calculated after cross attention, CA g is the input feature f g The output calculated after cross attention; MHA(f s ) for multiple CAs s The result of splicing together; MHA (f g ) for multiple CAs g The result of stitching together.
[0152] To alleviate the gradient vanishing or gradient exploding problem in multi-layer network training in S202 and prevent overfitting of specific input data:
[0153] Let f′ s=MHA(f s ), f′ g =MHA(f g ), MHA(·) represents the multi-head attention mechanism; f′ s and f′ g By inputting to residual connection, layer normalization, linear layer, and Dropout, the purpose is to solve the problem of multi-layer network training, for example, to alleviate the gradient vanishing or gradient exploding problems that occur when training deep networks, to prevent the model from overfitting to specific input data, to accelerate training and improve the model convergence speed. In addition, sequence representation can also be added to jointly enhance the learning ability, stability, and efficiency of the neural network; then the interactive feature f is extracted s and f g And the summed mixed feature Z is used as the input of the subsequent network, and the intermediate feature is expressed as:
[0154] f″ s =LN(Norm(f′ s )+f s )
[0155] f″ g =LN(Norm(f′ g )+f g )
[0156] f″′ s = Dropout(f″ s ,p)
[0157] f″′ g = Dropout(f″ g ,p)
[0158] f″″ s =Norm(LN(f″′) s ))+f′ s
[0159] f″″ g =Norm(LN(f″′) g ))+f′ g
[0160]
[0161]
[0162]
[0163] Among them, LN(·) represents the linear layer, Norm(·) represents the normalization layer, Dropout(·) is the regularization layer, and p represents the probability of each neuron being retained during training.
[0164] S203: Output the preliminary predicted trajectory of the target vehicle and its surrounding vehicles:
[0165] Encode the actual observed future trajectory feature Y:
[0166] E fin =E y +PE(E y )
[0167] Among them, E y is the generation feature of the actual observed future trajectory sequence y, y∈Y; E fin Represents the actual observed future trajectory characteristics after position encoding.
[0168] Perform fast Fourier transform on the mixed feature Z to get F z :
[0169] F Z =FFT(Z)
[0170] Among them, FFT() represents the calculation of fast Fourier transform;
[0171] For the processed feature F Z Multi-scale and adaptive graph convolutions are performed to capture specific and comprehensive cross-series dependencies:
[0172]
[0173] in, Represents the mixed features of node g, where nodes s and g are abstract representations of the target vehicle and surrounding vehicles; θ k The coefficient used to scale the contribution of the k-th neighbor; α sg is the adaptive weight, which determines the contribution of node g to node s; M k is the weight matrix of the k-th neighbor; RELU is a nonlinear activation function;
[0174] Features of nodes s, g Reshape to adapt to the multi-head self-attention mechanism and use the multi-head attention mechanism to capture features The temporal dependencies, spatial relationships, contextual information, and correlations among data;
[0175] By reshaping the return, we get the input tensor adapted to the softmax function, input it into the softmax function, and optimize the network weights through the Softmax function, thus helping to stabilize the training process:
[0176]
[0177]
[0178]
[0179]
[0180]
[0181]
[0182] Among them, Reshape() represents the calculation of the reshape module; ReshapeBack() represents the reshape return calculation;
[0183] Regularization layer processing features It is then input into the two-layer perceptron MLP, which extracts and combines data from the input features to obtain the predicted future position or trajectory of the target vehicle and surrounding vehicles:
[0184]
[0185]
[0186] Among them, W1 and b1 are the weight and bias of the first layer respectively; W2 and b2 are the weight and bias of the second layer, the output y is the predicted value of the vehicle's future trajectory, and h1 is the predicted feature output by the first layer perceptron.
[0187] The combined data is: MLP captures the spatiotemporal relationships in trajectory data, such as how to learn the motion patterns and behaviors of objects from time series; MLP captures nonlinear relationships in the data, and it can learn high-level expressions from complex input features that can map complex input-to-output relationships; MLP processes detailed information from multiple prediction features to make the final trajectory prediction more refined and specific.
[0188] S204: Atten the interactive features bidir (f s ,f g ) for individual uncertainty and collaborative uncertainty assessment:
[0189] Let Z = [z1,z2,...,z k ] T ,z k ∈Z, Z represents the features after interaction; f p (·) is a neural network with equal transformation; the mixed feature Z is passed into the neural network with equal transformation f p (·) to learn and generate feature f p (Z)=[z′1,z′2,...,z′ k ], f p(Z) is the feature space, where <z′ i ,z′ i >, i∈[1,k] represents the individual uncertainty σ of the target vehicle and surrounding vehicles i i ; <z′ i ,z′ j >,i,j∈[1,k],i≠j represents the cooperation uncertainty σ between the target vehicle and surrounding vehicles i and j i,j , σ i,j ∈Σ, Σ is the covariance matrix, as follows:
[0190]
[0191] Z'=f p (Z)=[z′1,z′2,...,z′ k ]
[0192] σ i =Var(z′ i )
[0193] σ i,j = Cov(z′ i ,z′ j )
[0194] Among them, MLP represents the calculation of multi-layer perceptron; MLP(z i ,z j ) is responsible for processing the input element (z i ,z j ); Var() represents the calculation of variance, which is used to measure the uncertainty of a single vehicle’s behavior; Cov() represents the calculation of covariance, which is used to measure the degree of interdependence and correlation between the behaviors of two vehicles.
[0195] S3: Measure the prediction effect through the loss function and update the optimization parameters; the specific steps of S3 are:
[0196] S301: Define the loss function for training trajectory prediction:
[0197] l tatal = l o +l u
[0198] Among them, l o is the trajectory prediction loss; l u Assess losses for uncertainty.
[0199] S302: Input the initial predicted trajectory and the actual trajectory of the target vehicle and surrounding vehicles in S2 into the loss function to calculate the error:
[0200]
[0201] Where N is the number of samples; They represent the components of the predicted speed of the i-th vehicle in the x and y directions respectively, and the predicted speed is the derivative of the predicted position; They represent the components of the true velocity of the i-th vehicle in the x and y directions respectively. The true velocity is the derivative of the true position;
[0202] S303: Substitute the individual uncertainty σ in S2 i and collaborative uncertainty σ i,j Enter the loss function for evaluation:
[0203]
[0204] Among them, l u is the loss of the uncertainty estimator; μ i is the mean vector of the predicted i-th sample, which here refers to the initial predicted trajectory; y i is the observation vector of the true trajectory of the i-th sample; Σ is the predicted covariance matrix, which describes the correlation between the output variables, and is composed of the covariance σ i,j composition; k is the dimension of the true trajectory sequence y; N is the number of samples.
[0205] like Figure 2 As shown, a system for implementing a trajectory prediction method for an autonomous driving vehicle based on a double cross Transformer includes a collection input module, a trajectory prediction module, a loss function module, and a visualization output module connected in sequence;
[0206] The acquisition input module collects historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles; in this embodiment, sensors or monitoring equipment, such as cameras, radars, LiDAR, GPS, can be used to collect vehicle motion data, and the collected vehicle motion data includes vehicle position and speed, heading angle, and timestamp;
[0207] Preprocessing of the collected raw data includes data cleaning, data conversion, feature engineering, and data set division. The purpose of this stage is to make the processed data more suitable for system training or analysis to improve the performance and accuracy of the system. The following are the specific steps of data preprocessing:
[0208] Data cleaning: Detect and correct errors, outliers, missing values, and inconsistencies in data.
[0209] Data transformation: Transform the data. For example, operations such as logarithmic transformation, normalization, standardization, feature scaling, and discretization can help improve the distribution and range of data and improve the stability and convergence speed of the system.
[0210] Feature engineering: creating new features or selecting important features, involving operations such as feature extraction, feature selection, and dimensionality reduction to reduce the complexity of the model or improve the performance of the system.
[0211] Dataset division: The data set is divided into training set, validation set and test set. The training set is used for model training, the validation set is used for model parameter tuning and selection, and the test set is used to evaluate system performance.
[0212] like Figure 3 As shown, the trajectory prediction module includes a trajectory Transformer encoder, a trajectory predictor and an uncertainty evaluator connected in sequence. The trajectory Transformer encoder learns the interaction between the target vehicle and the surrounding vehicles, and outputs the predicted trajectory through the trajectory prediction of the encoded interaction features, and is input into the uncertainty evaluator to evaluate individual uncertainty and collaborative uncertainty.
[0213] like Figure 4 As shown, the trajectory Transformer encoder mainly includes position encoding, bidirectional cross attention Transformer module, and the bidirectional cross attention Transformer module includes a bidirectional cross attention mechanism and a 6-layer Transformer encoder layer. First, the input sequence is position encoded, and the position encoding is added so that it can consider the order of elements in the sequence. In this embodiment, sine and cosine functions of different frequencies are used for position encoding.
[0214] like Figure 5 As shown in the figure, the bidirectional cross-attention Transformer module captures the interaction between vehicles, including a multi-head attention mechanism module, a layer normalization module, a linear layer module, and a Dropout module. The bidirectional cross-attention mechanism is a variant based on the self-attention mechanism, which allows learning complex spatiotemporal dependencies between vehicles in the input sequence, capturing the interaction between vehicles, thereby increasing the information representation between vehicles. In addition, the bidirectional cross-attention mechanism better adapts to the interaction uncertainty by dynamically adjusting the focus of attention, thereby improving the accuracy and robustness of the trajectory prediction system.
[0215] like Figure 6 As shown, the trajectory predictor includes a decoding embedding module, a multi-scale graph module, a regularization layer, and a multi-layer perceptron module connected in sequence. The trajectory predictor receives the interactive features output by the trajectory Transformer encoder, and outputs the preliminary predicted trajectory of the target vehicle and its surrounding vehicles through the decoding embedding module, the multi-scale graph module, the regularization layer, and the multi-layer perceptron module;
[0216] The trajectory predictor is responsible for receiving the interactive features output by the trajectory transformer encoder and converting them into target trajectory features. Secondly, the trajectory predictor can use the background information provided by the trajectory transformer encoder and combine its own multi-scale graph module to predict the future trajectory of the vehicle. In this system, the multi-scale graph module discovers and captures cross-sequence associations in multi-scale time series data, captures the associations within and between sequences at the same time, improves system performance while reducing computational complexity, and helps improve the system's performance in predicting multivariate time series. In addition, the decoding embedding module consists of 6 stacked multi-scale graph modules, each layer is responsible for extracting features of different levels and different degrees of abstraction from the output of the trajectory transformer encoder. The layered extraction of these features helps to understand the movement patterns and trends of vehicles or other vehicles at different times, and helps the prediction system understand and predict the trajectory more comprehensively. Finally, the features output by the decoding embedding module are input into a two-layer multi-layer perceptron to output the final target vehicle and the preliminary predicted trajectory of the surrounding vehicles.
[0217] like Figure 7 As shown, the multi-scale graph module includes a fast Fourier transform module, an adaptive graph convolution module, and a multi-head attention layer connected in sequence;
[0218] The multi-scale module considers the correlation and influence at different time scales or time scales, and represents them in the form of graph structures. By identifying the graph structures at different time scales, the changes and correlations of time series data can be effectively represented. The multi-scale module mainly solves the cross-sequence correlation problem in multi-scale time series data through the fast Fourier transform module, the adaptive graph convolution module, and the multi-head attention layer. First, the fast Fourier transform module is used to project the original time series data into the space related to the most significant time scale, so as to effectively capture and represent various cross-sequence correlations displayed at different time scales. Secondly, multiple adaptive graph convolution modules are introduced, combined with a learnable adjacency matrix to capture cross-sequence correlations at different time scales. Finally, the multi-scale graph module also integrates a multi-head self-attention layer to capture the temporal pattern within the data, and combines it with the adaptive graph convolution module to effectively capture and represent cross-sequence and internal sequence correlations at multiple scales.
[0219] like Figure 8As shown, the working principle of the trajectory prediction system structure of the present invention is: the historical position, speed, and heading angle information of the target vehicle and surrounding vehicles are input into the trajectory prediction module, so as to learn the preliminary predicted trajectory, individual uncertainty, and collaborative uncertainty of the target vehicle and surrounding vehicles. Among them, the trajectory predictor receives the interactive features output by the trajectory Transformer encoder, and finally outputs the preliminary predicted trajectory of the target vehicle and its surrounding vehicles through the decoding embedding module, multi-scale graph module, regularization layer, and multi-layer perceptron module. In addition, the uncertainty evaluator can simultaneously measure the individual uncertainty σ of vehicle i i , the cooperation uncertainty σ between vehicles i and i-1 i-1,i Finally, we automatically adjust the uncertainty loss by minimizing u ), thereby training the uncertainty estimator and making the estimator more reliable, thereby improving the performance of the system.
[0220] like Fig. 9 As shown in the figure, the uncertainty estimator is composed of an equal transformation neural network. By using the equal transformation neural network, individual uncertainty and collaborative uncertainty can be modeled, and the variance and covariance matrices of a given distribution can be estimated more accurately. In addition, the equal transformation neural network can also ensure that the model output remains consistent under different permutations, further verifying its effectiveness in the multi-vehicle trajectory prediction system.
[0221] Let Z = [z1,z2,...,z k ] T ,z k ∈Z, Z represents the features after the trajectory Transformer encoder interaction. p (·) is a neural network with equal transformation. Secondly, the mixed feature Z is passed into the neural network with equal transformation f p (·) to learn and generate feature f p (Z)=[z′1,z′2,...,z′ k ], where f p (Z) is the feature space, where <z′ i ,z′ i >, i∈[1,k] represents the individual uncertainty σ of vehicle i i , <z′ i-1 ,z′ i >, i∈[2,k] represents the cooperation uncertainty σ between vehicles i and i-1 i-1,i .
[0222] The loss function module minimizes the loss generated by the trajectory predictor and the loss generated by the uncertainty estimator by defining a loss function;
[0223] The visualization output module converts the training results into predictions of future target vehicle and surrounding vehicle trajectories, and visualizes the prediction results.
[0224] like Fig.10 As shown in the figure, the target vehicle and surrounding vehicle interaction diagram is constructed based on multiple vehicles in the vehicle-road cooperative environment. Vehicles in the vehicle-road cooperative environment are equipped with various sensors, communication equipment and intelligent control systems, which can perceive the surrounding environment, exchange information and make intelligent decisions. Road infrastructure is also equipped with sensors, communication equipment and intelligent control systems, which can collect traffic information, monitor road conditions and communicate with vehicles. In addition, vehicles and infrastructure exchange information through the Internet of Vehicles or other communication technologies. Finally, the vehicles within 50 meters around the target vehicle are regarded as surrounding vehicles. The surrounding vehicles will interact with the target vehicle and will affect the future trajectory of the target vehicle.
[0225] like Fig.11 As shown, the prediction results of the target vehicle and the surrounding vehicles are visualized; the collected data are passed through the trajectory prediction module, the loss function module, and the visualization output module, and finally the results of the prediction curves of the target vehicle and its surrounding vehicles are plotted in the form of a scatter plot to better understand the output.
[0226] like Fig.12 As shown, the interface consists of four parts. Frame ① displays the current operating status of the vehicle, including driving, stopping, braking, accelerating, starting, etc.; Frame ② displays the current time, the signal status of the vehicle and the remaining power at the current moment; Frame ③ displays the buttons of the two display interfaces of the trajectory prediction system: "Multi-vehicle interactive information" and "Vehicle trajectory prediction", and their specific display information can be viewed respectively; Frame ④ displays the vehicle-related information corresponding to the button in Frame ③.
[0227] The "Multi-vehicle Interaction Information" interface displays the local interaction diagram of the vehicles at the current moment, provides local interaction information between the target vehicle and surrounding vehicles, and shows the interaction and influence between vehicles in detail. The "Vehicle Trajectory Prediction" shows the predicted future trajectory of the autonomous driving target vehicle and its surrounding vehicles based on the input information at the current moment.
[0228] like Fig.13As shown in the figure, taking the traffic scene in the figure as an example, the future trajectory of the target vehicle and surrounding vehicles during driving is predicted. In this figure, the orange car is the ego vehicle (EV) to predict the future trajectory of the target vehicle (TV) and surrounding vehicles (SV). The ego vehicle will collect various information, including vehicle interaction information, vehicle motion information, etc. Secondly, the ego vehicle will process the above information and pass it into the trained prediction model to output the future trajectory of the target vehicle and surrounding vehicles. Finally, all the predicted display results are displayed on the central control screen of the car to show the driver the real-time driving behavior of the vehicle and provide relevant suggestions.
[0229] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0230] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An autonomous driving vehicle trajectory prediction method based on dual cross Transformer, characterized by: The following steps are involved: S1: Collect historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles as input sequence X; S1 is specifically: Historical trajectory data acquisition: The historical trajectory data of the vehicle includes the historical trajectory points in the previous three seconds, which are collected once every ten frames per second, and the trajectory points are represented as (x, y); Vehicle status data acquisition: Including vehicle speed v and heading angle θ, by analyzing the trajectory, vehicle speed v and heading angle θ, the vehicle's driving behavior is judged and the trajectory of the target vehicle and surrounding vehicles is predicted; Take historical trajectory data and vehicle status data as input sequence X; S2: Capture the interaction between the target vehicle and surrounding vehicles, output the preliminary predicted trajectories of the target vehicle and surrounding vehicles, and evaluate the individual uncertainty and collaborative uncertainty; S201: Positionally encode the input sequence X in S1: Positionally encode the input sequence X in S1; Use sine and cosine functions of different frequencies for position encoding, which can be expressed as: Among them, pos represents the position in the input sequence X; 2k, 2k+1 represents the dimension index; d model =512, indicating the dimension of the embedding layer; S202: Capture the interaction between the target vehicle and surrounding vehicles through a bidirectional cross attention mechanism to obtain the interaction feature Atten bidir (f s ,f g ), and train repeatedly: The interaction between the target vehicle and surrounding vehicles is captured through a bidirectional cross-attention mechanism, the interaction features are obtained, and training is repeated: Convert the input sequence X into input features: Take the input features f of the two target vehicles and surrounding vehicles from the input sequence X s ,f g , where f s ,f g ∈X; f s As the original input sequence, f g As the target input sequence, it is input into the bidirectional cross attention mechanism to learn the interaction relationship between the target vehicle and the surrounding vehicles: Calculate the input feature f s Q s , K s 、V s and f g Q g , K g 、V g , the formula is as follows: Q s =W Q f s ,K s =W K f s ,V s =W V f s ; Q g =W Q f g ,K g =W K f g ,V g =W V f g Among them, W Q , W K , W V They are used to generate Q s , K s 、V s and Q g , K g 、V g The conversion parameters, d is the input feature f s And the input feature f g The dimension, d q d k d v It's Q s , K s 、V s and Q g , K g 、V g Dimension; Q s , K s 、V s The input features f are s The query query, key key, value value three vectors; Q g , K g 、V g The input features f are g The query query, key key, value value three vectors; The input feature f s 、f g The interaction between the target vehicle and the surrounding vehicles is learned through the bidirectional cross-attention Transformer, and the target vehicle sequence and the surrounding vehicle sequence, as well as the preliminary interaction features Atten bidir (f s ,f g ), the specific formula is as follows: Atten bidir (f s ,f g )=MHA(f s )+MHA(f g ) Among them, n h is the number of heads; CA s is the input feature f s The output calculated after cross attention, CA g is the input feature f g The output calculated after cross attention; MHA(f s ) for multiple CAs s The result of splicing together; MHA (f g ) for multiple CAs g The result of stitching together; S203: Outputting preliminary predicted trajectories of the target vehicle and its surrounding vehicles; Output the preliminary predicted trajectories of the target vehicle and its surrounding vehicles: Encode the actual observed future trajectory feature Y: It is fin =E y +PE(E y ) Among them, E y is the generation feature of the actual observed future trajectory sequence y, y∈Y; E fin represents the actual observed future trajectory characteristics after position encoding, Perform fast Fourier transform on the mixed feature Z to get F Z : F Z =FFT(Z) Among them, FFT() represents the calculation of fast Fourier transform; For the processed feature F Z Multi-scale and adaptive graph convolutions are performed to capture specific and comprehensive cross-series dependencies: in, Represents the mixed features of node g, where nodes s and g are abstract representations of the target vehicle and surrounding vehicles; θ k The coefficient used to scale the contribution of the k-th neighbor; α sg is the adaptive weight, which determines the contribution of node g to node s; M k is the weight matrix of the k-th neighbor; RELU is a nonlinear activation function; Features of nodes s, g Reshape to adapt to the multi-head self-attention mechanism and use the multi-head attention mechanism to capture features The temporal dependencies, spatial relationships, contextual information, and correlations among data; By reshaping the return, we get the input tensor adapted to the softmax function, input it into the softmax function, and optimize the network weights through the Softmax function, thus helping to stabilize the training process: Among them, Reshape() represents the calculation of the reshape module; ReshapeBack() represents the reshape return calculation; Regularization layer processing features It is then input into the two-layer perceptron MLP, which extracts and combines data from the input features to obtain the predicted future position or trajectory of the target vehicle and surrounding vehicles: Among them, W1 and b1 are the weight and bias of the first layer respectively; W2 and b2 are the weight and bias of the second layer, and the output is the predicted value of the vehicle's future trajectory, and h1 is the predicted feature output by the first layer of perceptron; S204: Atten the interactive features bidir (f s ,f g ) Conduct individual uncertainty and collaborative uncertainty assessment; S3: Measure the prediction effect through the loss function and update the optimization parameters.
2. The method for predicting trajectory of an autonomous driving vehicle based on dual crossover Transformers according to claim 1, characterized in that: The S204 specifically includes: performing individual uncertainty and collaborative uncertainty evaluation on the interactive features: Let Z = [z1,z2,...,z k ] T ,z k ∈Z, Z represents the features after interaction; f p (·) is a neural network with equal transformation; the mixed feature Z is passed into the neural network with equal transformation f p (·) to learn and generate feature f p (Z)=[z1',z'2,...,z' k ], f p (Z) is the feature space, where <z′ i ,z′ i >, i∈[1,k] represents the individual uncertainty σ of the target vehicle and surrounding vehicles i i ; <z′ i-1 ,z′ i >, i∈[2,k] represents the cooperation uncertainty σ between the target vehicle and surrounding vehicles i and j i,j , σ i,j ∈Σ, Σ is the covariance matrix, as follows: Z'=f p (Z)=[z1',z'2,...,z' k ] σ i =There is(z i ') σ i,j =Cov(z i ',With' j ) Among them, MLP represents the calculation of multi-layer perceptron; MLP(z i ,z j ) is responsible for processing the input element (z i ,z j ); Var() represents the calculation of variance, which is used to measure the uncertainty of a single vehicle’s behavior; Cov() represents the calculation of covariance, which is used to measure the degree of interdependence and correlation between the behaviors of two vehicles.
3. The method for predicting trajectory of an autonomous driving vehicle based on dual crossover Transformers according to claim 2, characterized in that: The specific steps of S3 are: S301: Define the loss function for training trajectory prediction: l tatal =l o +l u Among them, l o is the trajectory prediction loss; l u Assessing losses for uncertainty, S302: Input the preliminary predicted trajectories and the actual trajectories of the target vehicle and surrounding vehicles in S2 into the loss function to calculate the error: Where N is the number of samples; They represent the components of the predicted speed of the i-th vehicle in the x and y directions respectively, and the predicted speed is the derivative of the predicted position; They represent the components of the true velocity of the i-th vehicle in the x and y directions respectively. The true velocity is the derivative of the true position; S303: Substitute the individual uncertainty σ in S2 i and collaborative uncertainty σ i,j Input into the loss function for evaluation: Among them, l u is the loss of the uncertainty estimator; μ i is the mean vector of the predicted i-th sample, which here refers to the initial predicted trajectory; y i is the observation vector of the true trajectory of the i-th sample; Σ is the predicted covariance matrix, which describes the correlation between the output variables, and is composed of the covariance σ i,j composition; k is the dimension of the true trajectory sequence y; N is the number of samples.
4. The method for predicting trajectory of an autonomous driving vehicle based on dual crossover Transformers according to claim 3, characterized in that: To alleviate the gradient vanishing or gradient exploding problem in multi-layer network training in S202 and prevent overfitting of specific input data: Let f′ s =MHA(f s ), f′ g =MHA(f g ), MHA(·) represents the multi-head attention mechanism; f′ s and f′ g By inputting to residual connection, layer normalization, linear layer, and Dropout, the learning ability, stability, and efficiency of the neural network are enhanced; then the interactive features are extracted and And the summed mixed feature Z is used as the input of the subsequent network, and the intermediate feature is expressed as: f s ”=LN(Norm(f s ')+f s ) f g ”=LN(Norm(f g ')+f g ) f s ”'=Dropout(f s ”,p) f g ”'=Dropout(f g ”,p) f s ””=Norm(LN(f s ”'))+f s ' f g ””=Norm(LN(f g ”'))+f g ' Among them, LN(·) represents the linear layer, Norm(·) represents the normalization layer, Dropout(·) is the regularization layer, and p represents the probability of each neuron being retained during training.
5. A system for implementing the dual crossover Transformer based automatic driving vehicle trajectory prediction method according to claim 4, characterized in that: It includes a collection input module, a trajectory prediction module, a loss function module, and a visualization output module connected in sequence; The acquisition input module collects historical trajectory data and vehicle status data of the target vehicle and surrounding vehicles; The trajectory prediction module includes a trajectory Transformer encoder, a trajectory predictor and an uncertainty evaluator connected in sequence. The trajectory Transformer encoder learns the interaction between the target vehicle and the surrounding vehicles, and outputs the predicted trajectory through the trajectory prediction of the encoded interaction features, and is input into the uncertainty evaluator to evaluate individual uncertainty and collaborative uncertainty. The trajectory predictor includes a decoding embedding module, a multi-scale graph module, a regularization layer, and a multi-layer perceptron module connected in sequence. The trajectory predictor receives the interactive features output by the trajectory Transformer encoder, and outputs the preliminary predicted trajectory of the target vehicle and its surrounding vehicles through the decoding embedding module, the multi-scale graph module, the regularization layer, and the multi-layer perceptron module; The loss function module minimizes the loss generated by the trajectory predictor and the loss generated by the uncertainty estimator by defining a loss function; The visualization output module converts the training results into predictions of future target vehicle and surrounding vehicle trajectories, and visualizes the prediction results.
Citation Information
Patent Citations
Vehicle trajectory prediction method based on uncertainty estimation
CN114005280A
Vehicle track prediction method based on dynamic interaction graph convolution
CN114802296A
Attention mechanism and target point information-based surrounding vehicle track prediction method and device
CN116923450A
Vehicle track prediction method based on intention perception space-time attention network
CN117141518A
Vehicle track prediction method independent of map information
CN117493424A
