A robust enhancement method for vehicle trajectory prediction based on joint prediction framework

Abstract

The application discloses a vehicle trajectory prediction robust enhancement method based on a joint prediction framework, first, a joint space-time attention network is constructed, including a teacher model and a student model, forming a joint prediction structure: the teacher model focuses on high-precision prediction of normal trajectories, and the student model learns from the teacher model through knowledge distillation and is simplified and enhanced to adapt to interference of adversarial noise; secondly, in order to optimize the structure and robustness of the student model, a robust compression strategy combining multi-stage pruning and self-distillation is designed, the strategy prunes the teacher model stage by stage, and then generates a student model suitable for an adversarial environment through self-distillation; then, based on the extreme value theory, a flow peak value over threshold method is proposed to analyze the prediction difference between the teacher model and the student model and detect abnormal input. The application significantly reduces the prediction error under adversarial noise and has a high detection rate, fully proving the prediction robustness in a complex noise environment.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of vehicle trajectory prediction and robustness research, and is a robust enhancement method for trajectory prediction models. Background Technology

[0002] With the rapid development of autonomous driving technology, vehicle trajectory prediction has become a crucial module for ensuring the safety and reliability of autonomous driving systems. Traditional vehicle trajectory prediction methods primarily rely on deep learning models to provide autonomous vehicles with information about the status of other vehicles in the surrounding environment by predicting future trajectories. However, existing methods exhibit vulnerability to savvy or adversarial trajectories, leading to a significant reduction in prediction accuracy, which poses a potential threat to the safety of autonomous vehicles. Savvy or adversarial trajectories are generated by adding specific noise to the vehicle trajectory, which can easily induce misjudgments in the prediction model, thereby affecting the stability of the autonomous driving system.

[0003] Existing robustness enhancement methods, such as data augmentation and adversarial training, while improving model performance in noisy environments to some extent, still have significant shortcomings in handling complex and dynamic adversarial noise. Specifically, data augmentation can only expand upon existing data and cannot cover all possible extreme noise scenarios, thus exhibiting poor adaptability to novel noise. Adversarial training, on the other hand, is mainly based on known noise patterns, making the model vulnerable to unseen noise types or complex adversarial examples. Furthermore, the repeated generation of adversarial examples and complex data processing incur high computational costs, making these methods too resource-intensive for large-scale autonomous driving applications and impacting the system's real-time response capabilities. Therefore, while improving model robustness, existing methods still struggle to comprehensively address varied noise environments, especially predictive accuracy in complex adversarial scenarios. Summary of the Invention

[0004] To overcome the shortcomings of existing trajectory prediction models in terms of robustness in the face of novel noise or complex adversarial environments, and the poor applicability of robustness enhancement methods, this invention proposes a robustness enhancement method for vehicle trajectory prediction based on a joint prediction framework. This method employs a joint prediction framework of teacher and student models to improve the robustness of the model itself.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A robust enhancement method for vehicle trajectory prediction based on a joint prediction framework includes the following steps:

[0007] Step 1: Construct a joint prediction framework (United Spatio-Temporal Attention Network, USAN). Robust vehicle trajectory prediction is achieved based on the USAN framework. By introducing a joint framework of a teacher model (STAN) and a student model (STAN-S), the impact of noise and adversarial trajectories is effectively addressed. The teacher model is used for trajectory prediction in everyday environments, ensuring accuracy and real-time performance. The student model, on the other hand, obtains features from the teacher model through knowledge distillation, maintaining stable performance in adversarial noise scenarios, thus achieving predictive adaptability to various environments.

[0008] Step 2: A robust compression strategy is adopted. To optimize the robustness of the student model, a compression strategy based on multi-stage pruning and self-distillation is proposed. First, the feedforward layer and recurrent layer of the teacher model are pruned, and low-weight connections are grouped and pruned in stages to reduce the computational load and enhance the model's noise resistance. Then, knowledge is extracted from the pruned teacher model through self-distillation to construct a student model that can adapt to complex noise environments, making the student model more robust when predicting adversarial trajectories.

[0009] Step 3: Adversarial trajectory detection based on extreme value theory. To ensure the system's ability to identify abnormal trajectories, the Streaming Peaks-Over-Threshold (SPOT) method in extreme value theory is used to detect the difference between the prediction results of the teacher model and the student model. This method captures extreme differences in the prediction output and regards the difference value as an anomaly indicator, thereby effectively identifying adversarial trajectories. This enables the system to respond quickly to abnormal trajectories in the input and take adaptive predictions.

[0010] Step 4: Trajectory prediction result selection. Based on the detection results, the system dynamically selects between the outputs of the teacher model and the student model: if an abnormal adversarial trajectory is detected, the system prioritizes the output of the student model's prediction result to reduce the interference of noise on the prediction; if a normal trajectory is detected, the system outputs the prediction result of the teacher model to ensure the accuracy and real-time performance of the prediction. This selection mechanism ensures the stability and adaptability of the system under various noise conditions.

[0011] The technical concept of this invention is to achieve robust vehicle trajectory prediction by constructing a joint prediction framework that combines a teacher model and a student model. The teacher model provides high-precision predictions, while the student model streamlines its structure through pruning and self-distillation techniques, enhancing its adaptability to adversarial noise. An extremum-based detection algorithm is used to determine the difference between the two model predictions to identify abnormal trajectories, thereby dynamically selecting an appropriate prediction output to ensure prediction stability and accuracy in complex noise environments. This design enables the system to maintain efficient and robust prediction performance in both normal and abnormal environments.

[0012] The beneficial effects of this invention are as follows: This invention significantly improves the robustness and stability of vehicle trajectory prediction models in complex noise and adversarial environments. By combining predictions from both the teacher and student models, the system can achieve high-precision predictions under normal conditions, while maintaining high anti-interference capability when facing abnormal or adversarial noise trajectories, effectively identifying abnormal inputs and selecting the optimal output result. This invention reduces the impact of abnormal trajectories on autonomous driving systems, thereby improving the safety, reliability, and responsiveness of autonomous driving. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the joint prediction framework USTANS.

[0014] Figure 2 This is the structural diagram of the STAN teacher model.

[0015] Figure 3 This is a schematic diagram of a robust model compression strategy.

[0016] Figure 4 This is a schematic diagram of the multi-stage robust pruning (SWPO) process.

[0017] Figure 5 It represents the percentage increase in error when the model uses adversarial trajectory prediction.

[0018] Figure 6 This is a graph showing the change in the RMSE (Reversible Error Sequence) of the model using adversarial trajectory prediction.

[0019] Figure 7 It is a visualization of trajectory prediction under adversarial trajectory input. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings.

[0021] Reference Figures 1 to 7 A robust enhancement method for vehicle trajectory prediction based on a joint prediction framework includes the following steps:

[0022] Step 1: Construct a joint prediction framework, the process is as follows:

[0023] Step 1.1: Construct a teacher-student joint prediction framework

[0024] The framework includes a teacher model (STAN) and a student model (STAN-S), as follows: Figure 1 The teacher model STAN and the student model STAN-S predict the same trajectory respectively and calculate the difference between their predicted trajectories. Adversarial trajectories are detected by the Streaming Peaks-Over-Threshold (SPOT) method, and the output value is 1 or 0; 1 represents that an adversarial trajectory is detected and 0 represents that it is not detected. When the output is 0, USAN outputs the predicted trajectory of STAN; when the output is 1, USAN outputs the predicted trajectory of STAN-S.

[0025] Step 1.2: Building the Teacher Model

[0026] The teacher model STAN first encodes the historical trajectories of the target vehicle and neighboring vehicles using a multilayer perceptron (MLP) and LSTM, as shown in equations (1) and (2). Then, a spatiotemporal attention module is used for feature extraction. The spatial attention module captures the spatial interaction relationships between vehicles, and its Query, Key, and Value matrices are obtained through linear transformations, as shown in equation (3). Each pair (S... tar ,S nbrs The attention weights are obtained by calculating the dot product between them, as shown in equation (4); based on the correlation coefficient The features of neighboring vehicles are aggregated onto the features of the target vehicle through calculation using formula (5); the Z aggregated features of multi-head attention constitute... The spatial attention module captures the interaction relationships between vehicles at each time step, but the correlation of features between time steps is not considered; the temporal attention module takes the output of the spatial attention module as input and extracts the features of the vehicle in the time dimension. Its Query, Key and Value matrices are obtained by linear transformation of G, as shown in Equation (6); the attention score is obtained by Equation (7), and feature aggregation is achieved through Equation (8); the output of multi-head attention constitutes B. head =[B t B t ,…,B t For temporal attention and spatial attention, the output A head and B headFeatures are further encoded through gated linear units (GLUs), as shown in equations (9) and (10); finally, gated recursive units (GRUs) are used to generate the predicted trajectory, as shown in equation (11); STAN adopts the mean square error (MSE) loss function during training, defined as in equation (12), and the principle of the STAN teacher model structure is as follows. Figure 2 As shown;

[0027] S tar =LSTM(MLP(V i ;ω1);ω2) (1)

[0028] S nbrs =LSTM(MLP(V nbrs ;ω3);ω4) (2)

[0029] In the formula, ω1, ω2, ω3, and ω4 are the learnable weights of the corresponding layers, and V nbrs V does not contain V i This section represents information about neighboring vehicles;

[0030]

[0031] Where q t K t and V t These are the Query, Key, and Value matrices at time t. and ω4, ω5 and ω6 are the feature vectors of the target vehicle and the neighboring vehicle at time t, respectively, and the learnable weights of their corresponding layers are the learnable weights of their respective layers.

[0032]

[0033] In the formula, <,> represents the dot product operation, and r K It is K t The dimension; This is the attention vector;

[0034]

[0035] B z =β·V (8) where ω9, ω 10 and ω 11 These are the learnable parameter matrices for the corresponding layers, where Q, K, and V are the Query, Key, and Value matrices for the time attention module, respectively; β is derived from β... ab The resulting score matrix measures the temporal correlation of features between times a and b. It is the dimension of matrix K;

[0036]

[0037] In the formula, LN is the layer regularization, ω8 is the parameter matrix of GLU, and ω 12 It is the parameter matrix of GLU. It is the output of the spatial attention module;

[0038]

[0039] In the formula, This indicates that the target vehicle output by STAN is at time T+T. p The predicted coordinates at time ω 13 and ω 14 These are the parameter matrices for MLP and GLU, respectively; and These are the predicted coordinates and the actual ground coordinates of the target vehicle in the nth sample at time T+t, where t∈{1,2,…,T} p};

[0040] Step 2: Employ a robust model compression strategy, as follows:

[0041] The robust model compression strategy first uses stage-wise pruning with optimization (SWPO) to effectively prune the teacher model, and then performs self-distillation on the pruned teacher model to obtain the student model, such as... Figure 3 As shown;

[0042] Step 2.1: Optimize staged pruning

[0043] By grouping the neural connections in the teacher model according to their role in the network structure and pruning them in stages, the grouping is based on the formula (13). After grouping STAN, the connections in each group have low dependencies. Therefore, SWPO performs individual amplitude pruning on each group of connections, that is, assigns different pruning thresholds ε to each group of connections. If the weight of a connection is less than ε, the connection will be pruned. Figure 4 The process of phased pruning is described. Each feedforward layer (phase three) connection is divided into a separate phase three group, stored as `ffl_group`, and different phase three groups are represented, for example, `ffl_group1`. A pruning threshold is assigned to each phase three group. All connections belonging to phase one in the RNN layers are divided into a phase group, stored as `mixed_group`, and assigned a pruning threshold ε. mixedAll connections belonging to stage 2 in the RNN layer are grouped into a stage 2 group, stored as a recurrent_group, and a pruning threshold ε is assigned. recurrent For each group, connections with weights less than the pruning threshold for that group are pruned. The pruned layer connections are as follows: Figure 4 As shown in the rightmost part; the pruning threshold is set by iteratively solving two subproblems. The first subproblem aims to fine-tune the pruning threshold for each group of connections, and the second subproblem optimizes the weights of the current connections in the RNN-based layer, as shown in equations (14) and (15).

[0044]

[0045] In the formula, ω represents the neuron connections in STAN, and Layer ff It is the set of all feedforward layers, each feedforward layer l∈Layer ff The connections are grouped separately. The regression connections in all RNN layers are grouped into a set ω recurrent The remaining connections in all RNN layers are grouped into a set ω. mixed ;

[0046]

[0047] In the formula, where ω pruned The connection relationships of the model after pruning are represented by ω. retrained This represents the connection relationships of the pruned model after retraining. For the set of all pruning thresholds, |Layer ff |+2 represents the number of connection groups, SP() represents amplitude pruning based on phased grouping, f train and f val These are functions for calculating the accuracy of the vehicle trajectory prediction model during training and validation, respectively, where δ is the predefined accuracy and |ω′| is the number of connections in the network after SP() processing.

[0048] Given a STAN model, its number of connections |ω| is determined. Formula (14) adjusts the pruning threshold to minimize the decrease in validation accuracy of the pruned model within the range δ, thereby obtaining the optimal ω′. pruned This indicates that when the pruning model ω is obtained... pruned Then, formula (15) requires the weights to be optimized relative to the training set, i.e., the model ω is retrained. pruned To achieve the highest possible accuracy and better adapt to the structure, ω is obtained. retrained Formulas (14) and (15) are interdependent and are processed iteratively. prunedThat is, after the above steps, ω retrained Then, the optimization is performed using formula (14), and so on. Finally, the iteration stops when certain requirements are met, such as reaching the predefined number of iterations or when retraining fails to meet the defined accuracy constraint. In the optimization of formula (14), Negatively Correlated Search (NCS) is used. The pruning threshold ε of each group of connections is treated as an individual in NCS, and its fitness estimation function is shown in formula (16). Initially, the lowest absolute connection weight in each group is used as the pruning threshold for each group, and the set of these is ε as the initial population. This setting can ensure that the accuracy constraint in the formula is met. In each iteration, NCS generates several child solutions (i.e., new candidates for ε) in a heuristic manner, calculates the fitness value fitness(ε), and selects the next generation solution based on these fitness values ​​and some heuristic rules. The negatively correlated design allows NCS to perform extensive searches in the solution space of ε. After a given number of iterations, NCS will return the solution ε with the lowest fitness value. * And generate the corresponding pruning model SP(ω,ε) * );

[0049]

[0050] Step 2.2, Self-distillation

[0051] Based on the pruned teacher model (STAN-P), in order to enable the student model to fully learn the knowledge of the teacher model, the output of each module of STAN-P is included as a loss along with the prediction error in the distillation loss function, as shown in equation (17).

[0052] L D =λ DE L DE +λ DS L DS +λ DT L DT +λ DD L DD +λ GT L GT (17)

[0053] In the formula, L DE L DS L DT L DD These are the loss function values ​​output by the corresponding modules of the teacher and student models, belonging to the corresponding input encoding module, spatial attention module, temporal attention module, and decoding module, respectively. GT λ is the error between the predicted trajectory output by the student model and the actual trajectory. DE , λDS , λ DT , λ DD , λ GT These are the constraint coefficients corresponding to the five losses, used to balance the degree of knowledge learning; the MSE function is used to calculate the loss in distillation, where the loss of each of the four modules is represented by formula (18), and in addition, the prediction loss L GT Represented as in equation (18);

[0054]

[0055] In the formula, L Dj Let P represent any one of the four module losses. j Dimensions representing loss and Let each represent a vector output by a module in the teacher model and a student model. At time T+t, in the nth sample, the trajectory coordinates of the target vehicle predicted by the student model, t∈{1,2,…,T} p};

[0056] Step 3: Adversarial Trajectory Detection Based on Extreme Value Theory

[0057] In the USAN joint prediction framework, the received samples are simultaneously input into STAN and STAN-S. After obtaining the predicted trajectories of STAN and STAN-S, the degree of difference between the two is measured. If the input sample is an adversarial trajectory sample, the output difference will be large. Therefore, only a simple fully connected layer is needed to distinguish the trajectory categories. However, using a detection boundary threshold to distinguish between normal and abnormal differences will be faster. Therefore, this invention uses the SPOT anomaly detection algorithm based on extreme value theory to search for the detection boundary. In order to measure the overall difference between the two trajectories, the displacement difference (DD) is first calculated for the predicted trajectories output by the teacher model and the student model using formula (20). In the initialization stage, the initial peak threshold t and detection threshold z are obtained based on the sample data. q In subsequent streaming data processing, whenever a new predicted variance value is generated, SPOT checks whether it exceeds a threshold z. q The following checks are performed: if the value exceeds the threshold, it is marked as an outlier and added to the outlier set; if the value is between the initial thresholds t and z... q In between, the "peak" set is updated and used for the next round of t and z. q The calculation; through this method, SPOT can dynamically adjust the detection threshold and accurately identify adversarial trajectories in real-time streaming data;

[0058]

[0059] In the formula, and These are the predicted trajectory coordinates of the teacher model and the student model at time T+t, respectively.

[0060] Step 4: Selection of trajectory prediction results

[0061] Based on the results of adversarial trajectory detection, the system dynamically selects between the outputs of the teacher and student models: if an abnormal adversarial trajectory is detected, the system prioritizes outputting the prediction result of the student model to reduce noise interference with the prediction; if a normal trajectory is detected, the system outputs the prediction result of the teacher model to ensure prediction accuracy and real-time performance. This selection mechanism ensures the stability and adaptability of the system under various noise conditions. The trajectory detection output value is 1 or 0, where 1 represents the detection of an adversarial trajectory and 0 represents no detection; when the output is 0, USTUAN outputs the predicted trajectory of STAN; when the output is 1, USTUAN outputs the predicted trajectory of STAN-S.

[0062] The simulation experiment in this embodiment proceeds as follows:

[0063] (1) Selection of dataset

[0064] This method is evaluated on the publicly available NGSIM dataset. The NGSIM dataset collects detailed vehicle trajectory information for the San Francisco Bay Area I-80 eastbound and US-101 southbound in Los Angeles; the study areas for I-80 and US-101 are 503 meters and 640 meters, respectively. Due to the high noise content in the NGSIM dataset, this method uses a Savitzky-Golay filter to smooth vehicle positions and update their corresponding velocities. The dataset is divided into three 15-minute time intervals; one-quarter of the data in each of the three subsets is selected as the test set, and the remaining data is used as the training and validation sets; all 10 original frames are resampled into 5 frames, and the vehicle trajectories for the next 5 seconds are predicted using the vehicle trajectory data from the first 3 seconds.

[0065] (2) Performance comparison experiment

[0066] During the experiment, the optimizer used in training was Adam, and the learning rate was set to 0.001. The model was trained using PyTorch. During training and validation set operations, the batch size was set to 32, and the number of training epochs was set to 100. In the two attention layers of spatiotemporal attention, the number of attention heads was 4, and the output dimension of the attention was set to 48. The input dimension of the GRU was 72-dimensional, corresponding to the dimension of the input features; the hidden states were activated by linear units of a nonlinear rectifier. Neighboring vehicles were defined as having a distance d = 7.62m.

[0067] The evaluation indexes of the experimental process are root mean square error (RMSE), average displacement error (ADE), final displacement error (FDE) and average lateral error (LADE), and their calculation formulas are shown in equations (21)-(24). Among them, AED is the average lateral distance between all predicted coordinates and the true coordinates, and is only used to evaluate the attack task on the vehicle trajectory prediction model.

[0068] Three trajectories were used for prediction, and the lowest prediction error value for each metric is shown in bold, as shown in Table 1. The data shows that the proposed model performs worse than AI-TP when using the normal trajectory, and only slightly better than AI-TP in prediction accuracy at 2 seconds when using the Gaussian trajectory. However, when using the adversarial trajectory, USTAAN not only has significantly higher accuracy than AI-TP, but also exhibits the smallest increase in prediction error. Figure 5 As shown, ADE only increased by 32.39%, which is much smaller than the increase in ADE of other models. Figure 6 The model shows the error variation when using adversarial trajectories and normal trajectories for prediction. The prediction error of the adversarial trajectory gradually increases over time, but the error increase of this method is significantly smaller than that of other models, which proves its robustness to adversarial noise.

[0069]

[0070] Table 1 shows the results of the robustness comparison experiment.

[0071]

[0072]

[0073] Continued from table:

[0074]

[0075] Table 1

[0076] (3) Visualization of adversarial trajectory robust prediction

[0077] Figure 7The visualization of the predictions from the experiment is shown, including the trajectory of the target vehicle and the historical trajectories of neighboring vehicles. The dots to the left of time T represent the target vehicle's true historical trajectory points, the star-shaped dots represent the adversarial trajectory points formed after adding adversarial noise, the dots to the right of time T represent its true future trajectory points, the triangles represent STAN's predicted trajectory points, and the squares represent STAN-S's predicted trajectory points. It can be seen that when STAN uses adversarial trajectories for prediction, it predicts a vehicle's straight-ahead behavior as a lane-changing behavior, rendering the prediction meaningless. While STAN-S is also affected by noise in the historical trajectories, resulting in insufficient prediction accuracy, its overall trajectory trend is roughly consistent with the true trajectory, and the prediction deviation is small, making it a robust prediction.

[0078] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for robust enhancement of vehicle trajectory prediction based on a joint prediction framework, the method comprising: The method comprises the following steps: Step 1, constructing a joint prediction framework USTAN, by introducing a joint framework of a teacher model STAN for trajectory prediction in daily environment and a student model STAN-S, the student model learns the features from the teacher model through knowledge distillation technology, and maintains stable performance in an adversarial noise scene, so as to realize prediction adaptability to various environments; Step 2, in order to realize the robustness optimization of the student model, a compression strategy based on multi-stage pruning and self-distillation is proposed, first, the feedforward layer and the recurrent layer of the teacher model are pruned, the low weight connections are grouped and pruned stage by stage; then the knowledge is extracted from the pruned teacher model through self-distillation, and the student model which can adapt to complex noise environment is constructed; Step 3, using the flow peak value over threshold method SPO in the extreme value theory to detect the difference between the prediction results of the teacher model and the student model, the extreme difference change of the prediction output is captured, and the difference value is regarded as an abnormal index, so as to effectively identify the adversarial trajectory, so that the system can quickly respond to the abnormal trajectory in the input and take adaptive prediction; Step 4, trajectory prediction result selection, according to the detection result, the system dynamically selects the output of the teacher model and the student model: if the abnormal adversarial trajectory is detected, the system preferentially outputs the prediction result of the student model; if the normal trajectory is detected, the prediction result of the teacher model is outputted; The process of step 1 is as follows: Step 1.1, constructing a teacher-student joint prediction framework The framework includes a teacher model STAN and a student model STAN-S, and the teacher model STAN and the student model STAN-S predict the same trajectory respectively; Step 1.2, construction of the teacher model The teacher model STAN first encodes the historical trajectories of the target vehicle and neighbor vehicles through multi-layer perceptron MLP and LSTM, and the encoding process is as shown in formula (1) and formula (2); then feature extraction is performed using a space-time attention module, the space attention module is used to capture the spatial interaction relationship between vehicles, and the Query, Key and Value matrices thereof are obtained through linear transformation, as shown in formula (3); the attention weight of each pair (S tar ,S nbrs ) is obtained by calculating the dot product between them, as shown in formula (4); according to the correlation coefficient The features of the neighbor vehicles are aggregated into the features of the target vehicle through formula (5); the Z aggregated features of the multi-head attention constitute The space attention module captures the interaction relationship between vehicles at each time step, but the correlation between features at different time steps is not considered; the time attention module takes the output of the space attention module as input, and extracts the features of the vehicles in the time dimension, and the Query, Key and Value matrices thereof are obtained through linear transformation on G, as shown in formula (6); the attention score is obtained by formula (7), and feature aggregation is realized by formula (8); the output of the multi-head attention constitutes B head =[B t ,B t ,…,B t ]; for the outputs A hesd and B head of the time attention and the space attention, the features are further encoded through the gated linear unit GLU, as shown in formula (9) and (10); finally, the predicted trajectory is generated using the gated recurrent unit GRU, as shown in formula (11); the STAN adopts the mean square error MSE loss function in training, and the definition is as shown in formula (13); S tar = LSTM(MLP(V i ; ω1); ω2) (1) S nbrs = LSTM(MLP(V nbrs ; ω3); ω4) (2) where ω1, ω2, ω3, and ω4are the learnable weights of their corresponding layers, V nbrs is the part of V i that does not contain V i , and represents information about the neighboring vehicles. where q t , K t and V t are the Query, Key and Value matrices at time t, respectively, and are the feature vectors of the target vehicle and the neighbor vehicles at time t, respectively, and ω4, ω5and ω6are the learnable weights of their corresponding layers. wherein <,> is a dot product operation, r K is a dimension of K t ; is an attention vector; B z = β · V (8) where ω9, ω 10 and ω 11 are the learnable parameter matrices of the corresponding layer, Q, K and V are the Query, Key and Value matrices of the temporal attention module respectively; β is a score matrix composed of β ab , which measures the temporal correlation of features between time a, b, is the dimension of matrix K; where LN is the layer normalization, ω8is the parameter matrix of the GLU, ω 12 is the parameter matrix of the GLU, is the output of the spatial attention module; where, denotes the predicted coordinates of the target vehicle at time T + T p at time T + t, t ∈ {1, 2, …, T 13 and ω 14 are the parameter matrices of the MLP and GLU, respectively; and are the predicted coordinates and the real ground coordinates of the target vehicle at time T + t in the nth sample, t ∈ {1, 2, …, T p , respectively.

2. The method of claim 1, wherein the method is based on a joint prediction framework for robust enhancement of vehicle trajectory prediction, and wherein the method further comprises: The process of step 2 is as follows: Step 2.1, optimization of stage pruning By grouping the neural connections in the teacher model according to their role in the network structure, and pruning them stage by stage, the grouping is according to formula (13); After grouping STAN, each group of connections has low dependency between them, so the optimized multi-stage pruning SWPO performs separate amplitude pruning for each group of connections, i.e., assigning different pruning thresholds ε to each group of connections, if the weight of the connection is less than ε, the connection will be pruned; the connections of each layer of the feedforward layer are separately divided into a phase three group, stored in ffl_group, and represented in the form of ffl_group1, and different phase three groups are assigned a pruning threshold for each phase three group; all connections belonging to phase one in the RNN-based layer are divided into a phase one group, stored in mixed_group, and assigned a pruning threshold ε mixed All connections belonging to phase two in the RNN-based layer are divided into a phase two group, stored in recurrent_group, and assigned a pruning threshold ε recurrent The connections in each group whose weights are less than the pruning threshold of the group are pruned, and the pruning threshold is set by iteratively solving two sub-problems, the first sub-problem aims to fine-tune the pruning threshold of each group of connections, and the second sub-problem optimizes the weight of the current connection in the RNN-based layer, as shown in equations (14) and (15); wherein ω is the connection relation of neurons in STAN, Layer ff is the set of all feedforward layers, each feedforward layer l e Layer ff is divided into a group separately Regression connections in all RNN layers are divided into a group ω recurrent The remaining connections in all RNN layers are divided into a group ω mixed ; wherein ω pruned represents the connection relationship of the pruned model, ω retrained represents the connection relationship of the pruned model after retraining, is a set of all pruning thresholds, |Layer ff is the number of connection groups, SP() is the amplitude pruning based on the phased grouping, f train and f val are functions for calculating the accuracy of the vehicle trajectory prediction model during training and verification, respectively, δ is a predefined precision, and |ω'| is the number of connections in the network after SP() processing. Given the STAN model, the number of connections |ω| is determined, and formula (14) changes the pruning threshold, so that the validation accuracy of the pruned model decreases within the range δ, and the number of connections in the model is reduced, and the optimal ω' is obtained from ω pruned ; when the pruned model ω pruned is obtained, formula (15) requires the weights to be optimized with respect to the training set, that is, the model ω pruned is retrained so that the accuracy is high to adapt to the structure, and ω retrained is obtained; Equation (14) and Equation (15) are interdependent and are processed in an iterative manner for ω pruned i.e. after the above steps, ω retrained is optimized by Equation (14) again, and so on, and finally the iteration stops when a predefined number of iterations is reached or the defined accuracy constraint is also not reached by retraining. In the optimization of formula (14), the negative correlation search NCS is adopted, and the pruning threshold ε of each group of connections is taken as an individual in NCS, and the fitness estimation function is shown in formula (16); initially, the lowest absolute connection weight in each group is taken as the pruning threshold of each group, and the set of ε is taken as the initial population; At each iteration, NCS generates offspring solutions, i.e. new candidates for ε, in a heuristic manner, computes fitness values for these solutions, and selects the next generation of solutions according to these fitness values and some heuristic rules; the negative correlation design enables NCS to perform a broad search over the solution space of ε; after having experienced a given number of iterations, NCS returns the solution ε with the lowest fitness value * and generates the corresponding pruned model SP(ω, ε * ). Step 2.2, self-distillation Based on the pruned teacher model STAN-P, in order to make the student model learn the knowledge of the teacher model fully, the output of each module of STAN-P is taken as a loss together with the prediction error to form a distillation loss function, as shown in formula (17); L D = θ DE L DE + θ DS L DS + θ DT L DT + λ DD L DD + θ GT L GT (17) In the formula, L DE , L DS , L DT , L DD are loss function values of the corresponding module outputs of the teacher model and the student model, respectively belonging to the corresponding input encoding module, the spatial attention module, the temporal attention module and the decoding module, L GT is the error between the predicted trajectory output by the student model and the real trajectory, λ DE , λ DS , λ DT , λ DD , λ GT are the limiting coefficients of the five losses respectively, used to balance the learning degree of knowledge; in the distillation, the MSE function is used to calculate the loss, wherein each loss of the four modules is represented by formula (18), and in addition, the prediction loss L GT is represented as formula (19); where L Dj represents any one of the four module losses, P j represents the dimension of the loss, and represents the vector of module outputs in the teacher model and the student model, respectively; is the predicted trajectory coordinate of the target vehicle by the student model in the nth sample at time T + t, t e {1, 2, …, T p}.

3. The method of claim 2, wherein, In step 3, the SPOT algorithm based on extreme value theory is used to search for the detection boundary. In order to measure the overall difference between the two trajectories, first calculate the displacement difference DD between the predicted trajectories output by the teacher model and the student model according to formula (20). In the initialization stage, the initial peak threshold t and the detection threshold z are obtained according to the sample data q In subsequent streaming data processing, whenever a new prediction difference value is generated, SPOT judges whether it exceeds the threshold z q according to whether it exceeds the threshold z or not: if it exceeds, it is marked as an outlier and added to the outlier set. If between initial threshold t and z q then update the "peak" set and use for next round of t and z q calculation; wherein, and are the predicted trajectory coordinates of the teacher model and the student model at time T+t, respectively.

4. The method of claim 1-3, wherein, In step 4, the output value of trajectory detection is 1 or 0, 1 represents detecting an adversarial trajectory, and 0 represents not detecting; When the output is 0, USTAN outputs the predicted trajectory of STAN; When the output is 1, USTAN outputs the predicted trajectory of STAN-S.

Images