Robust automatic driving track prediction method based on causal effect

Abstract

The invention relates to the technical field of automatic driving, in particular to a causal effect-based robust automatic driving trajectory prediction method, which comprises the following steps of: establishing a causal graph of a vehicle trajectory prediction model in an attack scene, and analyzing a causal relationship among nodes in the causal graph; building the fact prediction of the vehicle track in the attack scene according to the historical track, leading anti-fact intervention on the historical track, and building the anti-fact prediction of the vehicle track in the attack scene; and calculating a direct total effect by subtracting the anti-fact prediction from the fact prediction, and taking the direct total effect as a final prediction result. According to the method, the direct total effect in causal reasoning is used for defending the adversarial attack, and compared with an existing defending method, the adversarial robustness of the trajectory prediction model under the attack scene is effectively improved at the cost of sacrificing small performance on a clean data set.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and specifically to a robust autonomous driving trajectory prediction method based on causal effects. Background Technology

[0002] Due to the promising prospects of autonomous driving and its potential to solve existing traffic problems such as road accidents, traffic congestion, and energy waste, the field of autonomous driving research has attracted significant attention from universities, automakers, and research centers both domestically and internationally. An autonomous vehicle is a complex system composed of multiple modules, including environmental perception, target tracking, motion planning, and a control center. The environmental perception module detects surrounding pedestrians, vehicles, and obstacles; the target tracking module aims to acquire the historical trajectory of observed targets; and the trajectory prediction model in the motion planning module can predict the future trajectory of the target vehicle based on its historical trajectory. The predicted trajectory is crucial for the motion planning module and directly impacts the driving decisions of the control center. Therefore, achieving accurate trajectory prediction for autonomous vehicles is essential for ensuring the safety of autonomous driving.

[0003] Currently, many trajectory prediction models based on deep neural networks achieve superior trajectory prediction performance in attack-free scenarios. However, recent research has evaluated the performance of trajectory prediction models under adversarial attack scenarios. In attack-free scenarios, autonomous vehicles can correctly predict the future trajectory of attacking vehicles and make correct driving decisions. However, in attack scenarios, attacking vehicles mislead the trajectory prediction model of autonomous vehicles by generating adversarial trajectories. When the attacking vehicle travels along the adversarial trajectory, the autonomous vehicle predicts its future trajectory based on the attacking vehicle's trajectory, mistakenly believing that the attacking vehicle will not change its route and continuing to travel along its original route. However, the actual trajectory of the attacking vehicle differs from the future trajectory predicted by the autonomous vehicle, leading to a serious collision. Therefore, it is essential to research corresponding defense methods to address the driving safety problems caused by adversarial attacks in trajectory prediction.

[0004] To improve the adversarial robustness of trajectory prediction models, most existing defense methods focus on adversarial training. General adversarial training involves two steps: first, generating adversarial examples by maximizing the trajectory prediction error; second, optimizing the model using the generated adversarial examples to minimize the prediction error. However, adversarial training suffers from high training costs and poor generalization ability for different attack methods. Furthermore, while adversarial training improves the adversarial robustness of trajectory prediction models in attack scenarios, it sacrifices significant performance in attack-free scenarios (clean datasets). Summary of the Invention

[0005] The purpose of this invention is to provide a robust autonomous driving trajectory prediction method based on causal effects, which solves the problems of high training cost and poor generalization ability of adversarial training, and the fact that adversarial training improves the adversarial robustness of trajectory prediction models in attack scenarios, but sacrifices a large performance of prediction models in non-attack scenarios (clean datasets).

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: a robust autonomous driving trajectory prediction method based on causal effects, comprising the following steps:

[0007] Establish a causal graph for a vehicle trajectory prediction model under attack scenarios, and analyze the causal relationships between nodes in the causal graph;

[0008] Based on historical trajectories, factual predictions of vehicle trajectories under attack scenarios are constructed, and counterfactual intervention is led on historical trajectories to construct counterfactual predictions of vehicle trajectories under attack scenarios.

[0009] The direct total effect is calculated by subtracting the counterfactual prediction from the factual prediction, and this is used as the final prediction result.

[0010] Furthermore, the nodes in the causal graph include historical trajectories, historical trajectory features, adversarial perturbations, and predicted trajectories.

[0011] Furthermore, the predicted trajectory is obtained through a trajectory prediction model, the input of which includes the vehicle's historical trajectory.

[0012] The historical trajectory of a vehicle is defined as:

[0013] L obs =[X1,X2,...,X i ,...,X M ]

[0014] In the formula, Let represent the historical trajectory of the i-th vehicle (i∈[1,M]). Represents the two-dimensional coordinates of the vehicle at time t;

[0015] The actual trajectory of a vehicle is defined as:

[0016] L truth =[Y1,Y2,…,Y i ,…,Y M ]

[0017] in, This represents the actual trajectory of the i-th vehicle (i∈[1,M]). Represents the two-dimensional coordinates of the vehicle at time t;

[0018] The predicted future trajectory of a vehicle is defined as:

[0019]

[0020] in, Let represent the predicted trajectory of the i-th vehicle (i∈[1,M]). Let t represent the two-dimensional coordinates of the vehicle at time t.

[0021] Furthermore, the causal relationships between nodes in the causal graph are as follows: historical trajectories have a direct impact on predicted trajectories; historical trajectories have an indirect impact on predicted trajectories through their characteristics; historical trajectories and adversarial perturbations jointly determine the final predicted trajectory; and both historical trajectories and predicted trajectories are simultaneously affected by adversarial perturbations.

[0022] Furthermore, based on the causal graph of the vehicle trajectory prediction model under the attack scenario, x represents the historical trajectory, c represents the adversarial perturbation, and m represents the historical trajectory features. Indicates the predicted trajectory;

[0023] The factual prediction is expressed as:

[0024] Furthermore, it involves dominating counterfactual interventions along historical trajectories, including:

[0025] Utilizing counterfactual intervention principles and counterfactual trajectories Replace historical trajectory x with counterfactual trajectory Set it to the zero vector or the average vector of all historical trajectories;

[0026] Based on the counterfactual principle of counterfactual intervention, the value of the historical trajectory feature m is kept constant, and the intervention... The final prediction is expressed as follows:

[0027] In the formula, f(·) represents the trajectory predictor. This represents the trajectory features obtained based on the counterfactual trajectory encoding;

[0028] Counterfactual prediction is expressed as:

[0029] Furthermore, the direct total effect is the difference between factual and counterfactual predictions, defined as:

[0030]

[0031] And use it as a causal robust prediction:

[0032]

[0033] Furthermore, the training loss is represented as the L2 loss between the true trajectory and the causally robust predicted trajectory, and the training loss is expressed as:

[0034]

[0035] The above approach has the following beneficial effects:

[0036] This invention establishes a causal graph for a vehicle trajectory prediction model under attack scenarios and analyzes the causal relationships between nodes in the causal graph. The causal link indicates that the historical trajectory and adversarial perturbation jointly determine the final predicted trajectory. Counterfactual intervention is constructed by severing the link between adversarial perturbation and historical trajectory, and the historical trajectory is replaced with the counterfactual trajectory. Finally, the direct total effect is calculated by subtracting the counterfactual prediction from the factual prediction and is used as the final prediction result.

[0037] This invention utilizes the direct total effect in causal reasoning to defend against adversarial attacks. Compared with existing defense methods, it achieves a 13.4% performance improvement in attack scenarios at the cost of a 7.7% performance degradation on clean datasets. Furthermore, compared with adversarial training, this method improves performance by 20.6% on clean datasets and exhibits similar performance to adversarial training in attack scenarios. Experiments demonstrate that this scheme effectively improves the adversarial robustness of trajectory prediction models in attack scenarios at the cost of a small performance sacrifice on clean datasets.

[0038] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0039] Figure 1 This is a flowchart of an embodiment of the robust autonomous driving trajectory prediction method based on causal effects of the present invention;

[0040] Figure 2 This is a schematic diagram of the structural framework of an embodiment of the robust autonomous driving trajectory prediction method based on causal effects of the present invention;

[0041] Figure 3 This is a schematic diagram of the basic structure of the causal graph in an embodiment of the robust autonomous driving trajectory prediction method based on causal effects of the present invention;

[0042] Figure 4 This is a causal graph of vehicle trajectory prediction under an attack scenario, representing an embodiment of the robust autonomous driving trajectory prediction method based on causal effects of the present invention.

[0043] Figure 5 This is a trajectory prediction causal graph for different scenarios in an embodiment of the robust autonomous driving trajectory prediction method based on causal effects of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] The following detailed description illustrates the specific implementation method:

[0046] Implementation, for example, attached Figure 1 and attached Figure 2 As shown: A robust autonomous driving trajectory prediction method based on causal effects includes the following steps:

[0047] S1: Establish a causal graph for the vehicle trajectory prediction model under the attack scenario, and analyze the causal relationships between nodes in the causal graph.

[0048] Specifically, the basic structure of a cause-effect graph is a directed acyclic graph consisting of nodes and unidirectional arrows, such as... Figure 3 As shown in (a), the variables in the graph (e.g., A and B) are called nodes, and the one-way arrows (e.g., A→B) are called links. A cause-effect graph represents the causal relationships between different variables. Each node in the graph represents a variable, and the relationships between nodes are also different; for example, A points to B, and B points to C. For the link A→B, A is the parent node of B, and B is the child node of A. Cause-effect graphs include three basic structures: chain structure, fork structure, and anti-fork structure, such as... Figure 3 As shown. For a chain structure, the causal relationship is passed down layer by layer; for a fork structure, B is a confounding factor because B affects both A and C. For a reverse fork structure, there are two independent variables that cause the same result.

[0049] When the trajectory prediction model of an autonomous vehicle is subjected to adversarial attacks, the trajectory prediction causal graph in this case exhibits a forked structure because the adversarial perturbation simultaneously affects both the historical trajectory and the final prediction result. Therefore, this embodiment establishes a structure as follows: Figure 4 The example shown is a trajectory prediction causal graph for an attack scenario.

[0050] Cause-effect graphs represent nodes Through causal links The causal relationships between interactions can be represented as a directed acyclic graph. exist Figure 4 In a cause-effect graph, the nodes This includes historical trajectory X, adversarial perturbation C, historical trajectory features M, and predicted trajectory. Causal Link This represents the causal relationship between nodes. Figure 4 In the causal link Indicates X pairs It has a direct impact. Therefore, X is... The parent node, It is a child node of X. Simultaneously, a causal link... This indicates that X indirectly affects [something] through intermediate node M. Therefore, M is called X and Intermediate nodes between nodes. A causal relationship represents a causal link from a parent node to its child nodes. Causal link This means that the historical trajectory X and the adversarial perturbation C jointly determine the final predicted trajectory. At the same time, causal links This indicates that the counter-perturbation C influences the predicted trajectory by perturbing the historical trajectory X. If the historical trajectory X and the predicted trajectory Since both are affected by the adversarial perturbation C, the adversarial perturbation C is called the interference factor. Therefore, in order to defend against adversarial attacks, it is necessary to eliminate the influence of the interference factor on the final prediction result.

[0051] S2: Construct factual predictions of vehicle trajectories under attack scenarios based on historical trajectories, and take the lead in counterfactual intervention on historical trajectories to construct counterfactual predictions of vehicle trajectories under attack scenarios.

[0052] For vehicle trajectory prediction, vehicle trajectory prediction models can help autonomous vehicles plan reasonable paths and ensure the safety of autonomous driving. Trajectory prediction models in attack scenarios include trajectory encoders, attack models, counterfactual interventions, and trajectory decoders. Vehicle trajectory prediction belongs to a sequence prediction task, and the input to the trajectory prediction model includes the vehicle's historical trajectory. Based on this information, the future trajectory of the target vehicle can be predicted.

[0053] First, this embodiment defines the vehicle's historical trajectory as follows:

[0054] L obs =[X1,X2,...,X i ,...,X M ]

[0055] in, Let represent the historical trajectory of the i-th vehicle (i∈[1,M]). Let represent the two-dimensional coordinates of the vehicle at time t. Secondly, the vehicle's true trajectory can be defined as:

[0056] L truth =[Y1,Y2,...,Y i ,...,Y M ]

[0057] in, This represents the actual trajectory of the i-th vehicle (i∈[1,M]). Let represent the two-dimensional coordinates of the vehicle at time t. Finally, the predicted future trajectory of the vehicle can be defined as:

[0058]

[0059] in, Let represent the predicted trajectory of the i-th vehicle (i∈[1,M]). Let t represent the two-dimensional coordinates of the vehicle at time t.

[0060] In summary, the objective of vehicle trajectory prediction can be expressed as predicting the future trajectory of a target vehicle given a historical trajectory. Therefore, the vehicle trajectory prediction model can be optimized using the following L2 loss:

[0061]

[0062] Attack model targeting the trajectory prediction module: In the attack scenario, the attacking vehicle's goal is to maximize the prediction errors of other vehicles by generating adversarial trajectories. The attacking vehicle solves the adversarial objective... To obtain the optimal perturbation δ i Here, S = {δ|||δ|| p ≤ε} represents the adversarial set, ε represents the perturbation range, f(·) represents the trajectory predictor, and f(X) represents the trajectory predictor. i +δ i ) indicates based on the adversarial trajectory X i +δ i The obtained predicted trajectory. In generating adversarial trajectories, this embodiment selects a white-box attack based on Projected Gradient Descent (PGD) and a black-box attack based on Particle Swarm Optimization (PSO).

[0063] This embodiment uses six metrics to evaluate trajectory prediction results. First, this embodiment introduces two common evaluation metrics in the field of trajectory prediction: Average Displacement Error (ADE) and Final Displacement Error (FDE). ADE represents the average Euclidean distance between the actual trajectory and the predicted trajectory at all times, while FDE represents the Euclidean distance between the actual trajectory and the predicted trajectory at the final prediction time.

[0064] However, ADE and FDE cannot effectively evaluate the impact of directional attacks on prediction results. These two metrics only evaluate the Euclidean distance between the predicted trajectory and the true trajectory, without evaluating the displacement errors caused by directional attacks in various directions. Therefore, the literature Zhang Q, Hu S, Sun J, et al. On adversarial robustness of trajectory prediction for autonomous vehicles[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2022:15159-15168 proposes four additional evaluation metrics: average displacement error in the left direction, average displacement error in the right direction, average displacement error in the front direction, and average displacement error in the rear direction. These four evaluation metrics can be expressed as follows:

[0065]

[0066] here, and Let R(·) represent the actual trajectory vector and the predicted trajectory vector, and let R(·) represent the function that generates a unit vector in a specific direction. The longitudinal direction is approximated as... However, approximating the longitudinal direction as That's inaccurate because of the vertical direction. It changes continuously with the increase of time t. Meanwhile, the above calculation method is also quite complex. Therefore, this embodiment unifies the trajectory coordinates to the world coordinate system and proposes four indicators to evaluate directional attacks. These four indicators include the average displacement error in the left direction (left-ADE), the average displacement error in the right direction (right-ADE), the average displacement error in the front direction (front-ADE), and the average displacement error in the rear direction (rear-ADE). The average displacement error in the left and right directions can be defined as:

[0067]

[0068] in, and The x-coordinate represents the predicted trajectory and the actual trajectory in the world coordinate system.

[0069] Meanwhile, the average displacement error in the front and rear lateral directions can be defined as

[0070]

[0071] in, and The x-coordinate represents the predicted trajectory and the actual trajectory in the world coordinate system.

[0072] Counterfactual intervention in historical trajectories specifically includes: using causal reasoning theory to analyze trajectory prediction models under attack scenarios, mitigating the impact of adversarial perturbations on prediction results. From the perspective of causal graphs, inputting the entire graph into the model for prediction is unnecessary. Therefore, the values ​​of certain nodes can be directly modified and the changes in results observed. For example, cutting off... Figure 4 The causal link between C and X is changed to X = x. Then observe the resulting effects. In causal reasoning, the above treatment is called an intervention, which can be represented as a do(·) operation, for example, Intervention This indicates that the causal link c→X is severed, and the value of X is replaced with... For example, Figure 5 (b) Figure 5 The trajectory prediction causal graphs under different conditions were analyzed, including the original trajectory prediction causal graph, the trajectory prediction causal graph under intervention conditions, and the trajectory prediction causal graph under counterfactual conditions.

[0073] This embodiment utilizes counterfactual intervention to mitigate the impact of adversarial perturbations on prediction results. In counterfactual intervention, "intervention" means cutting off all links to a node and replacing the node's value; "counterfactual" means contrary to the fact. If a node is intervened, its child nodes will be unaffected and retain their original values. Figure 5 As shown in (c), intervention Change the value of X Node M retains its initial value m and does not change it.

[0074] This embodiment dominates counterfactual intervention along the historical trajectory. First, this embodiment establishes a causal graph of the autonomous vehicle under adversarial attack scenarios, such as... Figure 5 As shown in (a). Here, x represents the historical trajectory, c represents the counter-perturbation, and m represents the historical trajectory characteristics. This represents the final prediction; then, counterfactual trajectories are used based on the "intervention" principle of counterfactual intervention. Replace the historical trajectory x, such as Figure 5 As shown in (c). Counterfactual trajectory It can be set to the zero vector or the average vector of all historical trajectories; finally, according to the "counterfactual" principle of counterfactual intervention, the value of the trajectory feature m is kept unchanged, even if its parent node x is modified. like Figure 5 As shown in (c). In Figure 5 In (b), intervention The final prediction can be expressed as:

[0075]

[0076] In the formula, f(·) represents the trajectory predictor. This represents the trajectory features obtained based on the counterfactual trajectory encoding. In summary, Figure 5 The original predictions in (a) and Figure 5 The counterfactual predictions in (c) can be expressed as follows: and

[0077] S3: Calculate the direct total effect by subtracting the counterfactual prediction from the factual prediction, and use it as the final prediction result.

[0078] Specifically, causal effect refers to the difference between two predicted results when the same node takes two different values. For example, Figure 5 (a) Factual predictions and Figure 5 (c) Counterfactual prediction The difference between them. In causal reasoning, the above difference is called the direct total effect, which can be defined as:

[0079]

[0080] Besides the direct total effect, there is another type of causal effect called the total effect (TE). Figure 5 (b) The intervention status indicates that the value of the X child node follows the intervention. Operational changes, for example, when intervention Operation Order The value of the child node M of X becomes Therefore, TE can be represented as:

[0081]

[0082] To address the issue that trajectory prediction model results are affected by adversarial perturbations in attack scenarios, this embodiment calculates the factual prediction for the i-th vehicle. Counterfactual prediction The differences between them, and their use as causal robust predictions:

[0083]

[0084] Compared to traditional prediction methods in attack scenarios, the proposed method reduces the impact of adversarial attacks on the prediction results by calculating the difference between factual predictions and counterfactual predictions. Therefore, the training loss in this embodiment can be expressed as the L2 loss between the true trajectory and the causally robust predicted trajectory. The training loss can be expressed as:

[0085]

[0086] In this embodiment, based on the robust autonomous driving trajectory prediction method based on causal effects described above, the following experiment is conducted:

[0087] 1. Dataset

[0088] Experiments were conducted on three publicly available vehicle trajectory datasets: Apolloscape, NGSIM, and nuScenes. These three datasets are representative of data collected from different cities, reflecting various traffic scenarios. Therefore, they are widely used for vehicle trajectory prediction.

[0089] Apolloscape: The Apolloscape dataset was collected from urban areas in China during peak hours. Trajectories of all vehicles were obtained using object detection and tracking algorithms. The dataset includes 53 minutes of training sequences and 50 minutes of test sequences. During evaluation, the prediction model predicted the trajectory for the next 3 seconds (6 frames) based on the trajectory of the previous 3 seconds (6 frames).

[0090] NGSIM: The NGSIM dataset was collected from US-101 and I-80 highways in the United States, including 45 minutes of vehicle trajectories. This dataset encompasses various traffic scenarios, such as smooth traffic, light congestion, and severe congestion. It includes approximately 6,000 vehicles, with 5 frames of trajectories captured per second for each vehicle. During the evaluation phase, the prediction model predicts the trajectory for the next 5 seconds (25 frames) based on the trajectory of the previous 3 seconds (15 frames).

[0091] nuScenes: nuScenes is a large autonomous driving dataset comprising 1000 driving scenarios, collected in Boston and Singapore. Each driving scenario lasts 20 seconds, with vehicle trajectories sampled at 2 frames per second. nuScenes also provides semantic maps for 11 semantic categories. During the evaluation phase, the prediction model predicts the trajectory for the next 6 seconds (12 frames) based on the trajectory of the previous 2 seconds (4 frames).

[0092] 2. Model and Attack Setup

[0093] Two state-of-the-art trajectory prediction models, FQA and GRIP++, were evaluated. The FQA model models the inherent information, relative motion, intent, and interactions of agents in the scene, proposing a general prediction framework to predict the future trajectory of a target. The GRIP++ model utilizes a graph structure to model the interaction information between agents and employs a GRU-based encoder-decoder model to predict the target's future trajectory. This embodiment combines different models and datasets for training and fine-tunes hyperparameters to improve the performance of the prediction models. The CausalRobustTraj method in this embodiment can be seen as a plug-and-play defense method on existing trajectory prediction models. This embodiment validates the proposed method using both the FQA and GRIP++ models. Here, this embodiment analyzes the model learning complexity of the CausalRobustTraj method on the two prediction models. For CausalRobustTraj on the FQA model, the model parameter size is 0.028MB and the inference time for predicting a single trajectory is 0.0032s; for CausalRobustTraj on the GRIP++ model, the model parameter size is 0.496MB and the inference time for predicting a single trajectory is 0.0036s.

[0094] In terms of attack setup, this embodiment constructs six different attacks, implemented through a white-box attack based on Projected Gradient Descent (PGD) and a black-box attack based on Particle Swarm Optimization (PSO). Then, this embodiment evaluates the performance of the vehicle trajectory prediction model under these six different attack types: ADE attack, FDE attack, Left attack, Right attack, Front attack, and Rear attack. For any attack, the objective is to maximize the prediction error by calculating the aforementioned evaluation metrics. Simultaneously, this embodiment limits the range of each trajectory offset, setting the maximum offset for each trajectory to 1 meter. For the PGD-based white-box attack, this embodiment selects the Adam optimizer, sets the learning rate to 0.01, and the maximum number of iterations to 100; for the PSO-based black-box attack, this embodiment sets the particle number, inertia weight, acceleration coefficient, and maximum iterations to 10, 1.0, (0.5, 0.3), and 100, respectively.

[0095] 3. Analysis of Experimental Results

[0096] This embodiment compares the performance of the proposed method with existing defense methods. Existing methods include Adversarial Training (DA), Data Augmentation (DA), Training-time Smoothing (TrainSmooth), Test-time Smoothing (TestSmooth), DA+Train-time Smoothing (DA+TrainSmooth), and Detection+Test-time Smoothing (Detect+TestSmooth). Specifically, the DA method uses randomly perturbed trajectories that satisfy trajectory deviation constraints for adversarial training; the TrainingSmooth method uses a convolution-based linear smoother to smooth the trajectory during training; the TestSmooth method uses a convolution-based linear smoother to smooth the trajectory during testing; the DA+TrainSmooth method combines the DA and TrainingSmooth methods; and the Detect+TestSmooth method first detects whether the trajectory is adversarial during testing, and if so, performs smoothing. Here, this embodiment analyzes the time complexity of different defense algorithms, using T, N, M, and S. The numbers represent the training period, data sample size, attack steps, and detection threshold, respectively. The time complexity of the CausalRobustTraj method proposed in this embodiment is O(TN), the time complexity of the AT method is O(TMN), the time complexity of the DA method is O(TN), the time complexity of the TrainSmooth method is O(TN), the time complexity of the TestSmooth method is O(TN), the time complexity of the DA+TrainSmooth method is O(TN), and the time complexity of the Detect+TestSmooth method is O(N(logS+T)). Therefore, it can be seen that the method proposed in this embodiment has a lower time complexity compared to other methods.

[0097] In the experimental section, firstly, this embodiment implements six different types of attacks through white-box and black-box attacks, and evaluates the prediction errors of two prediction models (FQA and GRIP++) under white-box and black-box attack scenarios; secondly, this embodiment compares the prediction performance of the proposed method with other defense methods under different types of attacks; finally, this embodiment compares the performance degradation of the proposed method with other defense methods on clean datasets. Note that the experimental data in the table is in meters.

[0098] (1) Trajectory prediction before and after the attack

[0099] For different models and datasets, this embodiment implemented six different types of attacks using white-box attack methods and compared the prediction errors of the prediction model before and after the attacks, as shown in Table 1. Table 1 shows the prediction errors of the model before and after different attacks implemented by the white-box attack methods, including ADE attack, FDE attack, Left attack, Right attack, Front attack, and Rear attack. In the table, Normal represents the normal situation, and Attack represents the attack situation. The prediction errors in different directions (horizontal and vertical directions) can be calculated using the above evaluation metrics. The metrics proposed in this embodiment are more accurate than those described in Zhang Q, Hu S, Sun J, et al. On adversarial robustness of trajectory prediction for autonomous vehicles[C] / / Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition.2022:15159-15168. Note that when the trajectory prediction model is under different types of attacks, this embodiment only evaluates the evaluation metrics corresponding to that attack. For example, when the prediction model is under a Left attack, only the left-ADE metric is evaluated. Experimental results show that, compared to normal conditions, different types of attacks implemented using white-box attack methods significantly increase prediction errors. For ADE and FDE attacks, the average prediction error increases by 125% and 117% respectively compared to normal conditions. For Left and Right attacks, the average prediction deviation reaches 2.61 meters and 3.34 meters respectively compared to normal conditions, which may cause autonomous vehicles to mistakenly believe that vehicles in the horizontal direction are preparing to change lanes and brake suddenly. For Front and Rear attacks, the average prediction deviation reaches 3.25 meters and 3.33 meters respectively compared to normal conditions, which may cause autonomous vehicles to mistakenly believe that vehicles in the longitudinal direction are preparing to accelerate or decelerate and brake suddenly. Emergency braking caused by these prediction deviations can lead to serious traffic accidents, especially on highways.

[0100] In addition to white-box attack methods, this embodiment also evaluates the prediction performance of different models and datasets under black-box attacks, as shown in Table 2. Table 2 shows the prediction errors of the model before and after different types of attacks implemented by black-box attack methods. Similarly, only the corresponding evaluation metrics are evaluated for different types of attacks. For example, when the prediction model is under an ADE attack, only the ADE metric is evaluated. Experimental results show that, compared with the normal situation, black-box attacks significantly improve the prediction error of the model. For ADE and FDE attacks, the prediction error is improved by an average of 113% and 102% compared with the normal situation; for Left and Right attacks, the average prediction error reaches 2.87 meters and 2.93 meters compared with the normal situation; for Front and Rear attacks, the average prediction error reaches 2.97 meters and 3.01 meters compared with the normal situation. At the same time, as shown in Table 3, this embodiment also compares the prediction errors of the trajectory prediction model under white-box and black-box attacks, where White represents white-box attacks and Black represents black-box attacks. It can be observed that, compared to normal conditions, both white-box and black-box attack methods significantly increase prediction errors, which could potentially lead to serious traffic accidents in real life. Black-box and white-box attacks exhibit similar performance, with white-box attacks performing slightly better. For example, compared to black-box attack methods, white-box attack methods improve prediction errors by 5.7%, 7.4%, 10.2%, and 9.03% for ADE attacks, FDE attacks, attacks in all four directions, and all attacks, respectively. This is because white-box attack methods incorporate gradient guidance, thus generating smaller adversarial perturbations and reducing unnecessary disturbances.

[0101] Table 1. Prediction errors before and after different types of attacks implemented using white-box attack methods.

[0102]

[0103]

[0104] Table 2. Prediction errors before and after different types of attacks implemented using black-box attack methods.

[0105]

[0106] Table 3 Prediction errors under white-box and black-box attacks

[0107]

[0108] (2) Comparison of different defense methods in attack scenarios

[0109] This embodiment implements six different types of attacks on the Apolloscape dataset using white-box attack methods for two models, and compares the performance of the proposed CausalRobustTraj method with existing defense methods under different attack types. Table 4 shows the prediction errors of the GRIP++ model on the Apolloscape dataset under different attack types and different defense methods. Similarly, only the corresponding metrics are evaluated for each attack type. For example, when the prediction model is under an FDE attack, only the FDE metric is evaluated. Without a defense method, the trajectory prediction error is significantly higher than the prediction error under normal conditions. It can be seen that the method proposed in this embodiment significantly reduces the error caused by attacks. For ADE and FDE attacks, compared with the case without a defense method, the performance of the method in this embodiment is improved by an average of 9.9% and 9.2%, respectively, and the performance of the AT method is improved by an average of 10.7% and 11.3%. In this case, the best-performing method is TestSmooth. For ADE and FDE attacks, the performance of the TestSmooth method is improved by an average of 11.8% and 11.8% respectively compared to the case without a defense method. For attacks in four specific directions, compared to the case without a defense method, the performance of the method in this embodiment is improved by an average of 15.3%, the performance of the AT method is improved by an average of 14.6%, and the performance of the TestSmooth method is improved by an average of 14.3%. Regarding the average performance improvement for all six attacks, compared to the case without a defense method, the performance of the method in this embodiment is improved by 13.4%, the performance of the AT method is improved by 13.4%, and the performance of the TestSmooth method is improved by 13.5%. It can be seen that the method proposed in this embodiment has similar performance to the optimal method. This is because this embodiment starts from the perspective of causal reasoning, constructs a trajectory prediction causal graph, and analyzes the causal relationships between nodes. Since both factual prediction and counterfactual prediction include the impact of adversarial perturbations on the final prediction result, in order to eliminate interference factors, the factual prediction is subtracted from the counterfactual prediction to eliminate the impact of adversarial perturbations on the prediction result. This embodiment analyzes the generalization performance of the CausalRobustTraj method under different types of attacks. Compared to the case without defense, the CausalRobustTraj method improves performance by 9.9%, 9.2%, 16.1%, 15.1%, 13.4%, and 16.7% under ADE, FDE, Left, Right, Front, and Rear attacks, respectively. For the average performance improvement across the six different attacks, the CausalRobustTraj method achieves an average improvement of 13.4%. It can be observed that the method proposed in this embodiment has good generalization performance.

[0110] Table 4 shows the prediction errors of the GRIP++ model under different attack types and defense methods on the Apolloscape dataset.

[0111] Method ADE attack FDE attack Left attack Right attack Front attack Rear attack No defense 7.36 11.56 4.70 4.37 2.49 3.42 AT 6.57 10.25 3.98 3.87 2.11 2.85 DA 7.67 12.20 4.39 4.28 1.95 3.06 TrainSmooth 7.37 11.81 4.24 3.84 2.15 2.99 TestSmooth 6.49 10.19 3.81 3.86 2.07 3.08 DA+TrainSmooth 7.47 11.63 4.36 4.65 2.33 3.26 Detect+TestSmooth 6.72 10.48 4.05 4.01 2.27 3.10 CausalRobustTraj 6.63 10.49 3.94 3.71 2.16 2.85

[0112] This embodiment also implements six different types of attacks on the Apolloscape dataset using white-box attack methods for the FQA model, and compares the performance of the proposed CausalRobustTraj method with existing defense methods under different attack types. Table 5 shows the prediction error of the FQA model on the Apolloscape dataset under different attack types and different defense methods. Similarly, only the corresponding metrics are evaluated for different attack types. For example, when the prediction model is under a Right attack, only the right-ADE metric is evaluated. For ADE and FDE attacks, compared with no defense method, the method in this embodiment improves performance by an average of 28.8% and 28.9%, respectively, while the AT method improves performance by an average of 27.5% and 25.8%. The method with the best performance in this case is TestSmooth. For ADE and FDE attacks, the TestSmooth method improves performance by an average of 33.3% and 33.3%, respectively. For attacks in four specific directions, compared with no defense method, the performance of the method in this embodiment is improved by an average of 23.7%, the performance of the AT method is improved by an average of 28.4%, and the performance of the TestSmooth method is improved by an average of 30.7%. Regarding the average performance improvement for six types of attacks, compared with no defense method, the performance of the method in this embodiment is improved by 25.4%, the performance of the AT method is improved by 27.8%, and the performance of the TestSmooth method is improved by 31.6%. It can be seen that under ADE and FDE attacks, the method in this embodiment has similar performance to the optimal method, and the experimental results prove the effectiveness of the method proposed in this embodiment. Here, this embodiment analyzes the generalization performance of the CausalRobustTraj method under different types of attacks. Compared to the undefended scenario, the CausalRobustTraj method achieves performance improvements of 28.8%, 28.9%, 24.4%, 27.3%, 19.1%, and 23.7% under ADE, FDE, Left, Right, Front, and Rear attacks, respectively. For the average performance improvement across the six different attacks, the CausalRobustTraj method achieves an average improvement of 25.4%. It can be observed that the method proposed in this embodiment exhibits good generalization performance.

[0113] Table 5 shows the prediction errors of the FQA model under different types of attacks and different defense methods on the Apolloscape dataset.

[0114] Method ADE attack FDE attack Left attack Right attack Front attack Rear attack No defense 9.87 15.26 5.85 6.21 3.35 3.74 AT 7.15 11.31 4.32 4.57 2.49 2.41 DA 9.79 15.18 5.87 6.24 3.45 3.70 TrainSmooth 6.80 10.51 3.95 3.95 2.28 3.09 TestSmooth 6.58 10.17 3.78 4.01 2.39 2.86 DA+TrainSmooth 6.77 10.45 3.96 3.95 2.33 3.05 Detect+TestSmooth 7.21 11.10 4.46 4.48 2.74 2.95 CausalRobustTraj 7.02 10.85 4.42 4.51 2.71 2.85

[0115] (3) Comparison of different defense methods in the absence of attack

[0116] To evaluate the performance degradation of different defense methods under attack-free conditions (clean dataset), this embodiment conducted experiments on the Apolloscape dataset for two models. First, for the GRIP++ model on the Apolloscape dataset, this embodiment compared the performance degradation of the proposed CausalRobustTraj method with existing defense methods. Table 6 shows the prediction results of the GRIP++ model on the Apolloscape dataset under normal conditions using different defense methods. It can be found that the prediction performance of all defense methods degrades to varying degrees on clean data. For the ADE and FDE metrics, compared with the normal prediction results, the proposed method degrades by 8.1% and 7.3% respectively, while the AT method degrades by 28.9% and 32.2% respectively. In this scenario, the suboptimal method is Detect+TestSmooth, which degrades performance by 12% and 10.4%. For four different evaluation metrics, compared to normal prediction results, the performance of this embodiment's method decreased by an average of 7.7%, the AT method by an average of 36.9%, and the Detect+TestSmooth method by an average of 10.7%. For all evaluation metrics, compared to normal prediction results, the performance of this embodiment's method decreased by an average of 7.6%, the AT method by an average of 34.8%, and the Detect+TestSmooth method by an average of 10.9%. Experimental results show that, compared to other defense methods, this embodiment's method sacrifices the least performance on clean data. This is because this embodiment's method analyzes the causal relationships between nodes in the causal graph, and the model learns causal features as much as possible, rather than spurious features, through causal effects. Causal features reflect the true correlation between different variables and the final result. Therefore, this embodiment's method outperforms other defense methods on clean datasets.

[0117] Table 6 shows the prediction errors of the GRIP++ model on clean data using different defense methods on the Apolloscape dataset.

[0118] Method ADE FDE left-ADE right-ADE front-ADE rear-ADE No defense 2.21 3.72 0.11 -0.11 0.16 -0.16 AT 2.85 4.92 0.15 -0.15 0.22 -0.22 DA 2.77 4.74 0.38 -0.38 0.27 -0.27 TrainSmooth 2.56 4.29 0.30 -0.30 0.14 -0.14 TestSmooth 2.49 4.13 0.11 -0.11 0.18 -0.18 DA+TrainSmooth 2.63 4.37 0.14 -0.14 0.18 -0.18 Detect+TestSmooth 2.48 4.11 0.12 -0.12 0.18 -0.18 CausalRobustTraj 2.39 3.99 0.12 -0.12 0.17 -0.17

[0119] Meanwhile, this embodiment compares the performance of the proposed CausalRobustTraj method with existing defense methods on the Apolloscape dataset for the FQA model. Table 7 shows the prediction results of different defense methods for the FQA model on the Apolloscape dataset under normal conditions. For the ADE and FDE metrics, compared with the normal prediction results, the performance of the proposed method in this embodiment degrades by 11.4% and 13.7%, respectively; the performance of the AT method degrades by 47.6% and 30.9%, respectively; and the performance of the TestSmooth method degrades by 10.6% and 12.2%, respectively. For the four different evaluation metrics, compared with the normal prediction results, the performance of the proposed method decreases by an average of 9.1%, the performance of the AT method decreases by an average of 42.9%, and the performance of the TestSmooth method decreases by an average of 15.8%. For all evaluation metrics, compared with the normal prediction results, the performance of the proposed method decreases by an average of 10.3%, the performance of the AT method decreases by an average of 41.7%, and the performance of the TestSmooth method decreases by an average of 14.3%. Experimental results under this condition demonstrate that the method in this embodiment outperforms other defense methods, sacrificing minimal performance on clean datasets. In summary, compared to other defense methods, the proposed method in this embodiment significantly improves the robustness of the trajectory prediction model under attack scenarios at the cost of less performance degradation on clean datasets. This is because the proposed method analyzes the causal relationships in vehicle trajectory prediction, identifies interference factors in the trajectory prediction model under attack scenarios, and eliminates these interference factors by calculating the direct total effect, enabling the prediction model to learn as many causal features as possible rather than spurious features. Furthermore, the method in this embodiment is designed from a causal perspective, minimizing the impact of clean datasets on prediction performance.

[0120] Table 7 shows the prediction errors of the FQA model on clean data using different defense methods on the Apolloscape dataset.

[0121] Method ADE FDE left-ADE right-ADE front-ADE rear-ADE No defense 2.35 3.78 0.040 -0.040 0.12 -0.12 AT 3.47 4.95 0.061 -0.061 0.16 -0.16 DA 2.94 4.78 0.072 -0.072 0.18 -0.18 TrainSmooth 2.72 4.46 0.063 -0.063 0.14 -0.14 TestSmooth 2.60 4.24 0.046 -0.046 0.14 -0.14 DA+TrainSmooth 2.63 4.33 0.051 -0.051 0.11 -0.11 Detect+TestSmooth 2.66 4.31 0.052 -0.052 0.13 -0.13 CausalRobustTraj 2.62 4.30 0.044 -0.044 0.13 -0.13

[0122] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A robust autonomous driving trajectory prediction method based on causal effects, characterized in that, Includes the following steps: Establish a causal graph for a vehicle trajectory prediction model under attack scenarios, and analyze the causal relationships between nodes in the causal graph; Based on historical trajectories, factual predictions of vehicle trajectories under attack scenarios are constructed, and counterfactual intervention is led on historical trajectories to construct counterfactual predictions of vehicle trajectories under attack scenarios. The direct total effect is calculated by subtracting the counterfactual prediction from the factual prediction, and this is used as the final prediction result.

2. The robust autonomous driving trajectory prediction method based on causal effects according to claim 1, characterized in that, The nodes in the causal graph include historical trajectory, historical trajectory features, adversarial perturbations, and predicted trajectory.

3. The robust autonomous driving trajectory prediction method based on causal effects according to claim 2, characterized in that, The predicted trajectory is obtained through a trajectory prediction model, and the input of the trajectory prediction model includes the vehicle's historical trajectory. The historical trajectory of a vehicle is defined as: L obs =[X1,X2,...,X i ,…,X M ] In the formula, Let represent the historical trajectory of the i-th vehicle (i∈[1,M]). Represents the two-dimensional coordinates of the vehicle at time t; The actual trajectory of a vehicle is defined as: L truth =[Y1,Y2,…,Y i ,…,Y M ] in, This represents the actual trajectory of the i-th vehicle (i∈[1,M]). Represents the two-dimensional coordinates of the vehicle at time t; The predicted future trajectory of a vehicle is defined as: in, Let represent the predicted trajectory of the i-th vehicle (i∈[1,M]). Let t represent the two-dimensional coordinates of the vehicle at time t.

4. The robust autonomous driving trajectory prediction method based on causal effects according to claim 3, characterized in that, The causal relationships between nodes in the causal graph are as follows: historical trajectory has a direct impact on the predicted trajectory; historical trajectory has an indirect impact on the predicted trajectory through its characteristics; historical trajectory and adversarial perturbation jointly determine the final predicted trajectory; and historical trajectory and predicted trajectory are both affected by adversarial perturbation.

5. The robust autonomous driving trajectory prediction method based on causal effects according to claim 1, characterized in that, Based on the causal graph of the vehicle trajectory prediction model under the attack scenario, x represents the historical trajectory, c represents the adversarial perturbation, and m represents the historical trajectory features. Indicates the predicted trajectory; The factual prediction is expressed as:

6. The robust autonomous driving trajectory prediction method based on causal effects according to claim 5, characterized in that, Dominating counterfactual interventions throughout history, including: Utilizing counterfactual intervention principles and counterfactual trajectories Replace historical trajectory x with counterfactual trajectory Set it to the zero vector or the average vector of all historical trajectories; Based on the counterfactual principle of counterfactual intervention, the value of the historical trajectory feature m is kept constant, and the intervention... The final prediction is expressed as follows: In the formula, f(·) represents the trajectory predictor. This represents the trajectory features obtained based on the counterfactual trajectory encoding; Counterfactual prediction is expressed as:

7. The robust autonomous driving trajectory prediction method based on causal effects according to claim 6, characterized in that, The direct total effect is the difference between factual and counterfactual predictions, defined as: And use it as a causal robust prediction:

8. The robust autonomous driving trajectory prediction method based on causal effects according to claim 7, characterized in that... The training loss is represented as the L2 loss between the true trajectory and the causally robust predicted trajectory.

Images