Training method and system of trajectory prediction model

Abstract

The invention relates to the technical field of automatic driving, in particular to a trajectory prediction model training method and system, and the method comprises the steps: firstly obtaining a trajectory data set of a vehicle and a pedestrian in an automatic driving scene; secondly, training a trajectory prediction model by using the constructed trajectory data set; according to the data set, hyper-parameters for adding random disturbance in the historical track are determined; and then designing a path time sensitive saliency algorithm, adaptively selecting a trajectory with the highest model attack sensitivity in different scenes, and taking the trajectory as a sensitive path of the model. And then designing a method based on space-time gradient descent, and generating a reasonable antagonistic trajectory path under actual physical constraint descent. And finally, performing adversarial training by using a clean sample and a generated adversarial sample which is higher in concealment and more reliable, so as to obtain a trajectory prediction model with higher robustness. According to the method, the confrontation sample with higher concealment and flexibility in the trajectory prediction task is generated, and the robustness of the trajectory model can be better enhanced.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a training method and system for a trajectory prediction model. Background Art

[0002] An autonomous vehicle (AV) is a complex system that integrates modules such as road perception, behavior planning, and control decisions. Trajectory prediction within the perception module predicts the future trajectories of nearby moving objects. This function is crucial for the planning module and influences the AV's driving behavior. Therefore, accurate and reliable trajectory prediction is crucial for AV operation.

[0003] Many studies have proposed trajectory prediction models based on deep neural networks. They use the L2 norm distance between the predicted trajectory and the true trajectory as an evaluation metric to evaluate the model based on datasets collected from the real world. However, these trajectory prediction models based on deep neural networks are easily vulnerable to adversarial attacks, which can cause significant deviations in the model output. For example, Figure 1 As shown in the figure, for trajectory prediction, an attacker controls the target vehicle (OV). By driving the OV along a carefully designed trajectory close to the normally moving AV, the attacker can influence the AV's trajectory prediction and driving behavior, posing a safety threat to the AV's driving, such as sudden braking or lane changes, which can cause serious traffic accidents. Therefore, accurate and robust trajectory prediction is crucial for AV driving. To ensure the robustness of the trajectory prediction model, adversarial examples are generated and trained on the model.

[0004] There are two challenges in the adversarial attack task of vehicle trajectory prediction: 1) How to select the attack path to make the attack more covert and achieve the best attack effect? At the same observation moment, there are usually multiple occupants (agents) on the road, that is, there are multiple different historical trajectories within the observation period. How to select the historical attack trajectory to achieve the best attack effect? Existing methods select attack paths based on experience. The attack effect achieved by such methods may not be optimal and the attack lacks concealment and flexibility. 2) The attack method lacks explainability. Existing attack methods based on path perturbation usually select the trajectories of agents that move during the observation time for perturbation, and cannot explain the connection between the attack effect and the selected path.

[0005] There are two difficulties in attacking pedestrian trajectory prediction: 1) the feasibility of the attack method. Although perturbation attacks can be performed on all trajectories, this perturbation method is a global attack. In real-world scenarios, attackers with limited resources cannot control all agents, making it unsuitable for pedestrian trajectory prediction attacks in real-world scenarios. 2) the effectiveness of the attack method. The model's predicted future pedestrian trajectory is typically randomly sampled K times from the output bivariate Gaussian distribution. This randomness can cause gradient confusion, making it impossible to generate effective adversarial perturbations to achieve a stable and optimal attack effect. Summary of the Invention

[0006] The present invention aims to solve the problems of high budget and poor concealment in the existing technology for generating adversarial samples. A training method and system for a trajectory prediction model are proposed to generate more concealed and flexible adversarial samples in trajectory prediction tasks while ensuring the effectiveness of the attack. The adversarial training method is used to improve the robustness of the trajectory prediction model.

[0007] In order to achieve the above purpose, the technical solutions adopted are:

[0008] The present invention provides a method for training a trajectory prediction model, comprising the following steps:

[0009] Step 1: Obtain a trajectory dataset of vehicles and pedestrians in an autonomous driving scenario. Each trajectory in the trajectory dataset consists of a sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identifier.

[0010] Step 2: Use the constructed trajectory dataset to train the trajectory prediction model. According to the different object identifications, use the respective trajectory datasets to train the vehicle trajectory prediction model and the pedestrian trajectory prediction model respectively;

[0011] Step 3: Determine the hyperparameters of the added historical random perturbations based on the vehicle or pedestrian trajectories in the dataset.

[0012] Step 4: Design a path time-sensitive saliency algorithm that adaptively identifies the paths that are most sensitive to disturbances in different scenarios based on different models.

[0013] Step 5: Based on the identification of the most sensitive path of the model, a method based on spatiotemporal gradient descent is designed to iteratively generate the optimal longitude and latitude perturbation values of the trajectory points;

[0014] Step 6: Based on the hyperparameters of the historical random perturbation determined in step 3, use this as the actual physical constraint to optimize the perturbation value obtained in step 5, and add the final constrained perturbation value to the most sensitive path obtained in step 4 to generate a reasonable adversarial trajectory path;

[0015] Step 7: Use the clean samples and the adversarial samples generated in step 6 to retrain and obtain the final trajectory prediction model.

[0016] According to the training method of the trajectory prediction model of the present invention, further, in step 3, the boundary range ε, displacement mask threshold τ, scalar velocity and acceleration of the added historical random disturbance are determined based on the sampling duration of the vehicle or pedestrian trajectory in the data set, the scalar velocity of the normally moving object and the object displacement coordinates.

[0017] According to the training method of the trajectory prediction model of the present invention, further, the calculation process of the path time-sensitive saliency algorithm in step 4 is as follows:

[0018] Step 41: Take all the vehicle or pedestrian trajectories with random perturbations as input X′, use the vehicle trajectory prediction model and pedestrian trajectory prediction model constructed in step 2, the model is represented by ψ, and the model parameters obtained by training are represented by θ, and predict the corresponding output trajectory, thereby obtaining the model loss function L(ψ θ (X′), Y g ), Y g Represents the real future trajectory;

[0019] Step 42, L(ψ θ (X′), Y g ) is updated by the trajectory PGD attack method;

[0020] Step 43: Based on the loss function obtained in step 42, calculate its partial derivative with respect to the perturbation input X′ to obtain the path time-sensitive significance gradient;

[0021] Step 44: Use the ReLU activation function to perform a nonlinear transformation on the path time-sensitive saliency gradient calculated based on step 43 and take the L2 norm to obtain the non-negative saliency scores of N objects at time step t when the random perturbation R is added for the πth time.

[0022] Step 45, take the random perturbation calculation of π times The non-negative significance score corresponding to each object is calculated, and the object ID corresponding to the maximum value is selected as κ. The path corresponding to κ is the sensitive path of the model.

[0023] According to the training method of the trajectory prediction model of the present invention, further, the trajectory PGD attack method in step 42 is expressed by the formula:

[0024]

[0025]

[0026] Among them, mask is the displacement mask of the input data, marking whether the object moves in the observed time step, R is the random perturbation, ⊙ represents the multiplication of matrix elements, and i represents the i-th iterative attack. represents the adversarial trajectory of the i-th iteration, sign() is the Signum function, L() is the cross entropy loss, and α is the attack step size;

[0027]

[0028] in, Indicates whether n moves in time step t, n represents the object ID, t represents the time frame of observation, represents the moving speed of n in time step t, and τ is a hyperparameter.

[0029] According to the training method of the trajectory prediction model of the present invention, further, the calculation formula of the path time-sensitive saliency gradient obtained in step 43 is:

[0030]

[0031] in, tT represents the path time-sensitive saliency gradient of N objects at t time steps when the random perturbation R is added for the πth time. o +1:t represents the time length of the historical trajectory.

[0032] According to the training method of the trajectory prediction model of the present invention, further, when attacking the pedestrian trajectory prediction model, the loss function in the path time-sensitive saliency gradient calculation formula is modified, and the loss function is obtained by minimizing the negative log-likelihood loss.

[0033] According to the training method of the trajectory prediction model of the present invention, further, in step 5, the adversarial trajectory is generated based on the spatiotemporal gradient descent method, and the formula is:

[0034]

[0035]

[0036] Among them, S N is the indicator matrix S of the same size as X N ∈{0,1}, only when the object ID is n=κ, the corresponding feature element value is 1, and the rest are 0;

[0037] Then, the optimal trajectory point longitude and latitude disturbance value is generated and expressed as δ=X′-X.

[0038] According to the trajectory prediction model training method of the present invention, further, in step 6, the disturbance value obtained in step 5 is optimized using actual physical constraints, specifically including:

[0039] Use dynamic and kinematic constraints to ensure that adversarial trajectories conform to realistic driving scenarios;

[0040] Using the mean μ and standard deviation σ of the scalar velocity, acceleration, and acceleration derivative of all trajectories in the data set in longitude and latitude obtained in step 3, calculate the velocity, acceleration, and acceleration derivative values of the perturbed trajectory to be no more than μ±3σ;

[0041] The deviation of each trajectory position is limited to ε;

[0042] The perturbed data after physical constraints are added to the historical data to obtain the locally perturbed trajectory.

[0043] Furthermore, the present invention also provides a trajectory prediction model training system for implementing the above-mentioned trajectory prediction model training method, comprising:

[0044] The dataset construction module is used to obtain the trajectory dataset of vehicles and pedestrians in the autonomous driving scenario. Each trajectory in the trajectory dataset consists of the sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identifier;

[0045] The model pre-training module is used to train the trajectory prediction model using the constructed trajectory dataset. According to the different object identifications, the vehicle trajectory prediction model and the pedestrian trajectory prediction model are trained using their respective trajectory datasets respectively.

[0046] The hyperparameter acquisition module is used to determine the hyperparameters of the added historical random perturbations based on the vehicle or pedestrian trajectories in the dataset;

[0047] The sensitive path calculation module is used to design a path time-sensitive saliency algorithm that adaptively identifies the paths that are most sensitive to disturbances in different scenarios based on different models.

[0048] The disturbance value calculation module is used to design a method based on spatiotemporal gradient descent based on the identification of the most sensitive path of the model to iteratively generate the optimal longitude and latitude disturbance values of the trajectory points;

[0049] The adversarial trajectory generation module is used to optimize the obtained perturbation value based on the determined hyperparameters of the historical random perturbation as the actual physical constraints, and add the final constrained perturbation value to the obtained most sensitive path to generate a reasonable adversarial trajectory path;

[0050] The model retraining module is used to retrain the final trajectory prediction model using clean samples and generated adversarial samples.

[0051] The beneficial effects achieved by adopting the above technical solution are:

[0052] 1. More stealthy attacks: This paper proposes an automated attack method that adaptively selects model-sensitive paths for attack in different scenarios, achieving more stealthy attacks. This method avoids the shortcomings of empirically selected attack paths and increases the stealth and flexibility of the attack.

[0053] 2. Explainability of attack effects: This paper increases the explainability of attack effects through a sensitive path selection method. This method can explain why a certain path was chosen for attack and demonstrates the connection between the attack effect and the selected path, providing a more flexible attack framework for different trajectory prediction tasks.

[0054] 3. Reduced attack budget: This invention provides a more practical local attack method. Based on the path time-sensitive saliency algorithm, it only identifies one or a specific number of historical trajectories for perturbation attacks, rather than performing global adversarial perturbations on the trajectory data of all objects in the scene. This method reduces the budget for generating attacks and reduces the cost of attacks while achieving better attack results.

[0055] 4. Avoiding gradient confusion: The attack framework designed in this paper avoids gradient confusion caused by the randomness of the model's multimodal output through a path-time-sensitive saliency algorithm. It creates a deterministic gradient path in the model to generate adversarial trajectories, ensuring the stability of the attack effect.

[0056] 5. Improved model robustness: The adversarial examples generated by this method are more concealed and reliable, and can be used together with clean examples for adversarial training, thereby significantly improving the robustness of trajectory prediction models (vehicle trajectory prediction models and pedestrian trajectory prediction models). This method can effectively enhance the stability of the model in the face of adversarial attacks and improve the safety of autonomous driving systems.

[0057] 6. Adaptive Attack: This invention can adaptively select the most sensitive path to attack based on different scenarios and model characteristics, greatly reducing the detectability of the attack. This adaptability makes the attack method more flexible and practical.

[0058] 7. Physical constraints ensure trajectory authenticity: When generating adversarial trajectories, the present invention takes into account physical constraints (such as speed and acceleration) to ensure that the generated perturbed trajectories conform to real driving scenarios, avoid abnormal driving behaviors, and enhance the reproducibility and authenticity of the perturbed trajectories.

[0059] In summary, the present invention significantly improves the stealth, flexibility, and interpretability of attacks through adaptive selection of sensitive paths, localized attacks, and avoidance of gradient confusion. It also enhances the robustness of trajectory prediction models through adversarial training, which is of great significance for the practical application of autonomous driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention. The drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0061] Figure 1 This is an example of an attack scenario for a trajectory prediction task using existing technologies. By driving along a carefully crafted adversarial historical trajectory, the attacker misleads the AV system's predictions. This can lead to collisions in future trajectory planning, posing a safety threat to the AV.

[0062] Figure 2 is a framework diagram of a method for training a trajectory prediction model according to an embodiment of the present invention;

[0063] Figure 3 4 is a flow chart of a method for training a trajectory prediction model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the accompanying drawings of specific embodiments of the present invention to clearly and completely describe the exemplary embodiments of the present invention. Unless otherwise defined, technical or scientific terms used in the present invention should be given the common meanings understood by people with ordinary skills in the relevant field.

[0065] This embodiment discloses a method for training a trajectory prediction model. Figure 2 and Figure 3 As shown, the following steps are included:

[0066] Step S1: Obtain vehicle and pedestrian trajectory data in an actual autonomous driving scenario, preprocess the data, and construct a vehicle trajectory dataset and a pedestrian trajectory dataset. Each trajectory in the trajectory dataset consists of a sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identifier. The dataset is a series of time series data.

[0067] Step S2: Use the constructed trajectory dataset to train the trajectory prediction model. According to the different object identifiers, use the respective trajectory datasets to train and obtain the vehicle trajectory prediction model and the pedestrian trajectory prediction model respectively.

[0068] Step S3: Determine the boundary range (-ε, ε), displacement mask threshold τ, scalar velocity, and acceleration parameters of the added historical random perturbation based on the sampling duration of the vehicle or pedestrian trajectory in the dataset, the scalar velocity of the normally moving object, and the displacement coordinates of the object in the scene. The perturbation boundary ε for the vehicle trajectory model is 1 meter, and the perturbation boundary ε for the pedestrian trajectory model is 0.2 meters. The above parameters serve as hyperparameters for applying adversarial perturbations for different scenarios.

[0069] Step S4: Based on steps S2 and S3, a path time-sensitive saliency algorithm is designed. This algorithm can adaptively identify the path that is most sensitive to disturbances in different scenarios based on different models. This step includes sub-steps S401-S405.

[0070] Step S401: All vehicle or pedestrian trajectories with random disturbances are used as input X′, and the vehicle trajectory prediction model and pedestrian trajectory prediction model constructed in step S2 are used. The model is represented by ψ, and the model parameters obtained by training are represented by θ. The corresponding output trajectory is predicted, and the loss function L(ψ) of the model is obtained. θ (X′), Y g ), Y g Represents the true future trajectory.

[0071] Step S402, L(ψ θ (X′), Y g ) is updated via the trajectory PGD attack method.

[0072] The traditional PGD attack is as follows:

[0073]

[0074] Among them, sign() is the Signum function, L() is the cross entropy loss, α is the attack step size, R is the random perturbation, and Clip() ensures that ||x′-x|| p ≤ε. Note that adversarial attacks occur during the testing phase, and the attacker cannot manipulate the prediction model or its output. To apply it to attack prediction tasks, Formula (1) needs to be transformed to form a new attack method called trajectory PGD attack, as shown in Formula (2):

[0075]

[0076] Among them, mask is the displacement mask of the input data, marking whether the object (vehicle or pedestrian) moves in the observed time step. Like the random perturbation R, its size is the same as X. ⊙ represents the multiplication of matrix elements, i represents the i-th iterative attack, and X t ￠ (i) represents the adversarial trajectory of the i-th iteration.

[0077]

[0078] Where n represents the object ID, t represents the time frame of observation, represents the moving speed of n in time step t, τ is a hyperparameter, which is determined by factors such as the sampling frequency of the dataset and the moving speed of the object, and is determined by step S3. Indicates whether n moves in time step t, R⊙mask indicates that in the initial case, only the moving objects are randomly perturbed. Similarly, Path perturbations only occur on objects that move in time steps t. For obvious reasons, if the object itself is stationary, adding a perturbation to its path violates the original limitation of perturbations: they are imperceptible to the human eye.

[0079] Step S403: Based on the loss function obtained in step S402, calculate its partial derivative with respect to the disturbance input X′ to obtain the path time-sensitive significance gradient, as shown in formula (4):

[0080]

[0081] in, When the random disturbance R is added for the πth time, N objects are o )Path time-sensitive saliency gradient over time steps, tT o +1:t represents the time length of the historical trajectory, with the current time as 0 as the mark, and the historical length of the observation is (-T o ,0); the time period of the predicted future trajectory is represented by (0, T p ); reveals the impact of different path perturbations on the model loss under the same degree of perturbation.

[0082] Step S404: Add random perturbations to the historical paths of all objects (vehicles or pedestrians) Where T o is the historical duration of observation, U(-ε,ε) represents a random distribution with a boundary of ±ε, and the real future trajectory Y g , the path time-sensitive saliency of each object is derived as formula (5):

[0083]

[0084] Where, is the loss function, σ is the activation function, and the time step t∈(0, T o ).

[0085] Preferably, a ReLU activation function is used to calculate the non-negative significance score for each time step. As shown in formula (6):

[0086]

[0087] Take L2 norm to obtain represents the non-negative saliency scores of N objects at t time steps when the random perturbation R is added for the πth time

[0088] Step S405: Take the random perturbation calculation The non-negative saliency score corresponding to each object is calculated by taking the average value of

[0089]

[0090] in, is a list of time-sensitive saliency values of the paths corresponding to N objects. The object ID corresponding to the maximum value is κ, and the path corresponding to κ is the sensitive path of the model. That is, adding the same initial perturbation to this path causes the maximum loss of the model.

[0091] When attacking the pedestrian trajectory prediction model, the loss function in formula (4) is modified and the loss function is obtained by minimizing the negative log-likelihood loss, as shown in formula (8):

[0092]

[0093] in, represents the mean of the bivariate Gaussian distribution output by the trajectory prediction model, represents the standard deviation, In formulas (2) and (3), Still, n represents whether it has moved at time step t. Its value is either 0 or 1, controlled by the hyperparameter τ. Based on the sampling frequency of the pedestrian trajectory dataset and the normal pedestrian movement speed of 0.8m / s to 1.2m / s, τ is set to 0.2. The model's sensitive path is derived using a time-sensitive saliency algorithm. This algorithm avoids gradient confusion caused by the model's randomness and creates a deterministic gradient path in the model to generate adversarial perturbations.

[0094] Step S5: Based on the identification of the most sensitive path of the model, a method based on spatiotemporal gradient descent is designed to iteratively generate the optimal longitude and latitude disturbance values of the trajectory points.

[0095] On the basis of selecting the sensitive path of the model, the perturbed path is generated based on the gradient descent method. The projected gradient descent PGD attack is used as an example. The variant attack method combined with the trajectory prediction attack task is called: spatiotemporal trajectory PGD attack, and the following formula (9) is constructed:

[0096]

[0097] Among them, S N is the indicator matrix S of the same size as X N ∈{0,1}, only when the object ID is n=κ, the corresponding feature element value is 1, and the rest are 0. The above formula ensures that perturbations are only added to the sensitive path selected by the sensitive path identification algorithm.

[0098] This method generates local attack samples for historical trajectory data. Of course, it can also generate perturbed local adversarial samples by extending other gradient-based methods, such as FSGM and MIM. Here we only introduce PGD attacks.

[0099] The disturbance value δ = X′-X represents the difference between the perturbed characteristics of the sensitive path and the benign characteristics. The difference here is the difference in physical longitude and latitude coordinates. Only when n = κ, X′ κ ≠X κ .

[0100] Step S6: To ensure the authenticity and reproducibility of the generated perturbation trajectory, the hyperparameters of the historical random perturbation determined in step S3 are used as actual physical constraints to optimize the perturbation value obtained in step S5. The final restricted perturbation value is added to the most sensitive path obtained in step S4 to generate a reasonable adversarial trajectory path, thereby obtaining a more concealed adversarial sample.

[0101] The trajectories generated by adversarial attacks should ensure naturalness. Otherwise, abnormal paths are easily identified as abnormal trajectories by the autonomous driving system. Therefore, the adversarial trajectory must follow physical constraints and includes the following sub-steps:

[0102] Step S601: Use dynamic and kinematic constraints to ensure that the trajectory conforms to the real driving scenario; at the same time, the adversarial trajectory should exhibit smoothness, which helps to reduce abnormal driving behaviors such as sharp turns and maintain consistency in target movement and other physical limitations.

[0103] Step S602: Using the mean μ and standard deviation σ of the scalar velocity, acceleration, and acceleration derivative of all trajectories in the data set in longitude and latitude obtained in step S3, calculate the velocity, acceleration, and acceleration derivative of the trajectory after the disturbance to ensure that the values do not exceed μ±3σ.

[0104] Step S603: Limit the deviation of each trajectory position to ε. When physical constraints are violated, the constraints are strengthened by reducing the disturbance. Considering the disturbance δ, the target vehicle's real historical trajectory X, and the constraint function f, the maximum coefficient γ is obtained by reducing the disturbance γ·δ (where 0≤γ≤1) so that f(X+γ·δ) satisfies all constraints.

[0105] Step S604: Add the disturbance data δ after physical constraints to the historical data to obtain the trajectory after local disturbance, as shown in formula (10):

[0106] X′=X+γ·δ (10)

[0107] Only when n = κ κ ≠0, ensuring that perturbations are only added to identified sensitive paths, generating local perturbation attacks rather than global attacks. During the testing phase, this solution is used in different scenarios rather than empirical selection. Based on the characteristics of different models, the most sensitive paths of the model are selected for individual perturbations to generate adversarial inputs, ensuring the concealment of the perturbations and reducing their detectability.

[0108] Due to the inherent randomness and social nature of pedestrian motion trajectories, no physical constraints are added when generating perturbed trajectories. Instead, the perturbation boundary is controlled to ensure δ∈(-ε, ε). The pedestrian trajectory prediction task randomly samples the model output K times to obtain the optimal predicted trajectory. The attack method described in this solution adds a minimum error path selection process, randomly sampling from the bivariate Gaussian distribution of the perturbed predicted output and selecting the path with the smallest average displacement error from the true future trajectory as the perturbed predicted trajectory.

[0109] Step S7: To improve the robustness of the trajectory prediction model, the more concealed and reliable adversarial samples generated in step S6 are used together with clean samples for adversarial training, thereby obtaining a vehicle and pedestrian trajectory prediction model with higher stability and robustness.

[0110] Corresponding to the above method, this embodiment also discloses a trajectory prediction model training system, including:

[0111] The dataset construction module is used to obtain the trajectory dataset of vehicles and pedestrians in autonomous driving scenarios. Each trajectory in the trajectory dataset consists of the sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identification.

[0112] The model pre-training module is used to train the trajectory prediction model using the constructed trajectory dataset. According to the different object identifications, the vehicle trajectory prediction model and the pedestrian trajectory prediction model are trained separately using their respective trajectory datasets.

[0113] The hyperparameter acquisition module is used to determine the hyperparameters of the added historical random perturbations based on the vehicle or pedestrian trajectories in the dataset.

[0114] The sensitive path calculation module is used to design a path time-sensitive saliency algorithm, which adaptively identifies the paths that are most sensitive to disturbances in different scenarios based on different models.

[0115] The disturbance value calculation module is used to design a method based on spatiotemporal gradient descent based on the identification of the most sensitive path of the model, and iteratively generate the optimal longitude and latitude disturbance values of the trajectory points.

[0116] The adversarial trajectory generation module is used to optimize the obtained perturbation value based on the determined hyperparameters of historical random perturbations as actual physical constraints, add the final restricted perturbation value to the obtained most sensitive path, and generate a reasonable adversarial trajectory path.

[0117] The model retraining module is used to retrain the final trajectory prediction model using clean samples and generated adversarial samples.

[0118] This invention utilizes a path-time-sensitive saliency algorithm to adaptively select paths with high model attack sensitivity based on different scenarios. It then generates optimal perturbation values using a designed spatiotemporal gradient descent method. This approach, while taking physical constraints into account, can produce perturbation trajectories that are closer to reality. This adaptive attack ensures both stealth and flexibility. Because the attack path can vary depending on the model and scenario, the detectability of the attack is significantly reduced.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for training a trajectory prediction model, characterized in that: The following steps are involved: Step 1: Obtain a trajectory dataset of vehicles and pedestrians in an autonomous driving scenario. Each trajectory in the trajectory dataset consists of a sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identifier. Step 2: Use the constructed trajectory dataset to train the trajectory prediction model. According to the different object identifications, use the respective trajectory datasets to train the vehicle trajectory prediction model and the pedestrian trajectory prediction model respectively; Step 3: Determine the hyperparameters of the added historical random perturbations based on the vehicle or pedestrian trajectories in the dataset. Step 4: Design a path time-sensitive saliency algorithm that adaptively identifies the paths that are most sensitive to disturbances in different scenarios based on different models. Step 5: Based on the identification of the most sensitive path of the model, a method based on spatiotemporal gradient descent is designed to iteratively generate the optimal longitude and latitude perturbation values of the trajectory points; Step 6: Based on the hyperparameters of the historical random perturbation determined in step 3, use this as the actual physical constraint to optimize the perturbation value obtained in step 5, and add the final constrained perturbation value to the most sensitive path obtained in step 4 to generate a reasonable adversarial trajectory path; Step 7: Use the clean samples and the adversarial samples generated in step 6 to retrain and obtain the final trajectory prediction model.

2. The method for training a trajectory prediction model according to claim 1, wherein: In step 3, the boundary range ε, displacement mask threshold τ, scalar velocity and acceleration of the added historical random perturbation are determined according to the sampling duration of the vehicle or pedestrian trajectory in the dataset, the scalar velocity of the normally moving object and the object displacement coordinates.

3. The method for training a trajectory prediction model according to claim 1, wherein: The calculation process of the path time-sensitive saliency algorithm in step 4 is as follows: Step 41: Take all the vehicle or pedestrian trajectories with random perturbations as input X′, use the vehicle trajectory prediction model and pedestrian trajectory prediction model constructed in step 2, the model is represented by ψ, and the model parameters obtained by training are represented by θ, and predict the corresponding output trajectory, thereby obtaining the model loss function L(ψ θ (X′), Y g ), Y g Represents the real future trajectory; Step 42, L(ψ θ (X′), Y g ) is updated by the trajectory PGD attack method; Step 43: Based on the loss function obtained in step 42, calculate its partial derivative with respect to the perturbation input X′ to obtain the path time-sensitive significance gradient; Step 44: Use the ReLU activation function to perform a nonlinear transformation on the path time-sensitive saliency gradient calculated based on step 43 and take the L2 norm to obtain the non-negative saliency scores of N objects at time step t when the random perturbation R is added for the πth time. Step 45, take the random perturbation calculation of π times The non-negative significance score corresponding to each object is calculated, and the object ID corresponding to the maximum value is selected as κ. The path corresponding to κ is the sensitive path of the model.

4. The method for training a trajectory prediction model according to claim 3, wherein: The trajectory PGD attack method in step 42 is expressed as follows: X t ′ (0) =X+R⊙mask, Among them, mask is the displacement mask of the input data, marking whether the object moves in the observed time step, R is the random perturbation, ⊙ represents the multiplication of matrix elements, and i represents the i-th iterative attack. represents the adversarial trajectory of the i-th iteration, sign() is the Signum function, L() is the cross entropy loss, and α is the attack step size; in, Indicates whether n moves in time step t, n represents the object ID, t represents the time frame of observation, represents the moving speed of n in time step t, and τ is a hyperparameter.

5. The method for training a trajectory prediction model according to claim 4, wherein: The calculation formula for the path time-sensitive significance gradient obtained in step 43 is: in, tT represents the path time-sensitive saliency gradient of N objects at t time steps when the random perturbation R is added for the πth time. o +1:t represents the time length of the historical trajectory.

6. The method for training a trajectory prediction model according to claim 5, wherein: When attacking the pedestrian trajectory prediction model, the loss function in the path time-sensitive saliency gradient calculation formula is modified and the loss function is obtained by minimizing the negative log-likelihood loss.

7. The method for training a trajectory prediction model according to claim 4, wherein: In step 5, the adversarial trajectory is generated based on the spatiotemporal gradient descent method, and the formula is: X t ′ (0) =X+R⊙mask⊙S N , Among them, S N is the indicator matrix S of the same size as X N ∈{0,1}, only when the object ID is n=κ, the corresponding feature element value is 1, and the rest are 0; Then, the optimal trajectory point longitude and latitude disturbance value is generated and expressed as δ=X′-X.

8. The method for training a trajectory prediction model according to claim 2, wherein: In step 6, the perturbation value obtained in step 5 is optimized using actual physical constraints, specifically including: Use dynamic and kinematic constraints to ensure that adversarial trajectories conform to realistic driving scenarios; Using the mean μ and standard deviation σ of the scalar velocity, acceleration, and acceleration derivative of all trajectories in the data set in longitude and latitude obtained in step 3, calculate the velocity, acceleration, and acceleration derivative values of the perturbed trajectory to be no more than μ±3σ; The deviation of each trajectory position is limited to ε; The perturbed data after physical constraints are added to the historical data to obtain the locally perturbed trajectory.

9. A trajectory prediction model training system, characterized in that: A method for implementing a trajectory prediction model according to any one of claims 1 to 8, comprising: The dataset construction module is used to obtain the trajectory dataset of vehicles and pedestrians in the autonomous driving scenario. Each trajectory in the trajectory dataset consists of the sampling duration, the physical latitude and longitude of the corresponding sampling point, and the corresponding object identifier; The model pre-training module is used to train the trajectory prediction model using the constructed trajectory dataset. According to the different object identifications, the vehicle trajectory prediction model and the pedestrian trajectory prediction model are trained using their respective trajectory datasets respectively. The hyperparameter acquisition module is used to determine the hyperparameters of the added historical random perturbations based on the vehicle or pedestrian trajectories in the dataset; The sensitive path calculation module is used to design a path time-sensitive saliency algorithm that adaptively identifies the paths that are most sensitive to disturbances in different scenarios based on different models. The disturbance value calculation module is used to design a method based on spatiotemporal gradient descent based on the identification of the most sensitive path of the model to iteratively generate the optimal longitude and latitude disturbance values of the trajectory points; The adversarial trajectory generation module is used to optimize the obtained perturbation value based on the determined hyperparameters of the historical random perturbation as the actual physical constraints, and add the final constrained perturbation value to the obtained most sensitive path to generate a reasonable adversarial trajectory path; The model retraining module is used to retrain the final trajectory prediction model using clean samples and generated adversarial samples.

Images