A safety-critical traffic simulation method based on adversarial transfer of driving intent

Abstract

This invention relates to a safety-critical traffic simulation method based on adversarial transfer of driving intent. The method includes the following steps: S1, identifying the attacking vehicle, the autonomous vehicle, and the background vehicle, and extracting their motion states and predicting the future trajectory of the autonomous vehicle; S2, training the attack trajectory prediction model and the background trajectory prediction model; S3, establishing a motion adversarial optimization problem based on the predicted future trajectory of the autonomous vehicle, and solving it to obtain the position of the adversarial target and its corresponding preliminary trajectory; S4, retaining the position of the adversarial target as the endpoint of the attacking vehicle's trajectory and inputting it into the attack trajectory prediction model to obtain the complete trajectory of the attacking vehicle, and obtaining the trajectory of the background vehicle based on the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, the complete trajectory of the attacking vehicle, and the background trajectory prediction model. Compared with the prior art, this invention has advantages such as improving the realism and efficiency of generated traffic accident collision data.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a safety-critical traffic simulation method based on adversarial migration of driving intentions. Background Technology

[0002] In recent years, various deep learning-based autonomous driving algorithms have made significant progress. However, due to the long-tail distribution of real-world data, the development and evaluation of data-driven autonomous driving algorithms typically focus on routine scenarios, neglecting accident-prone scenarios. Furthermore, due to the severe losses (and even life-threatening consequences) caused by traffic accidents, data collection in such dangerous traffic scenarios is costly and difficult to achieve in large quantities.

[0003] A viable alternative is to synthesize realistic hazardous traffic scenarios. In these scenarios, surrounding traffic participants will engage in adversarial interactions with autonomous vehicles, exhibiting near-realistic driving behaviors. However, due to the scarcity of hazardous scenario data, it is difficult to directly apply the latest deep generation techniques to synthesize such scenarios. Recently, several studies have explored different strategies to alleviate this problem. Existing hazardous scenario generation methods can be mainly divided into the following three categories:

[0004] (1) Adversarial optimization-based methods. For example, AdvSim generates hazardous scenarios by combining random perturbations and using gradient descent for black-box search. STRIVE uses a conditional variational autoencoder to learn the data distribution from real data and generates hazardous scenarios by performing adversarial learning in the latent space.

[0005] (2) Safety-critical trajectory generation based on reinforcement learning. Feng suggests using deep Q-networks to discretely generate hazardous scenarios, while Kuutti uses A2C reinforcement learning to control surrounding vehicles, thereby generating safety-critical scenarios. Reinforcement learning methods do not rely on large amounts of data, but they have certain limitations in terms of realism and diversity.

[0006] (3) Knowledge-based safety-critical scenario generation methods. These methods guide the generative model to generate near-collision trajectories by using prior knowledge as conditions, such as safety-critical constraints and causal relationships. For example, DiffScene uses safety and traffic rule-based constraints as conditions to control the generation process of the diffusion model. Chang and Yin introduced input variables to control the degree of aggressiveness of driving behavior during training. Causalaf further introduced causal graphs to model and generate hazardous scenarios.

[0007] Furthermore, in the work related to the closed-loop training of the planner, CAT uses a multi-modal trajectory prediction model to sample multiple possible trajectories and selects the trajectory with the highest probability of collision with the autonomous vehicle. However, this prediction model is learned from normal scenarios and cannot capture aggressive driving intentions that could lead to collisions in extreme situations.

[0008] In summary, traffic simulation utilizes synthetic data to supplement real-world data, effectively evaluating and enhancing the overall capabilities of autonomous vehicles. However, due to the rarity and high cost (even life-threatening risks) of real-world hazardous scenarios, existing evaluations of autonomous driving methods rarely consider dangerous traffic scenarios. Therefore, the need for simulating hazardous traffic scenarios is urgent. However, existing methods cannot accurately simulate data leading to collisions in extreme situations, resulting in insufficient realism in the simulated data, and they are difficult to generalize to complex scenarios. While adversarial optimization-based methods improve realism through generative models, their reliance on iterative processes leads to high time complexity. Summary of the Invention

[0009] The purpose of this invention is to provide a safety-critical traffic simulation method based on adversarial migration of driving intentions in order to improve the realism and efficiency of traffic accident collision generation data.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] A traffic simulation method based on adversarial transfer of driving intent, the method comprising the following steps:

[0012] S1. Identify the attacking vehicle, the autonomous vehicle, and the background vehicle, and extract the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, as well as predict the future trajectory of the autonomous vehicle.

[0013] S2. Obtain a large-scale real-world traffic scene dataset, train a trajectory prediction network based on the large-scale real-world traffic scene dataset, and obtain the attack trajectory prediction model and the background trajectory prediction model.

[0014] S3. Based on the predicted future trajectory of autonomous vehicles, establish a motion adversarial optimization problem and solve it to obtain the position of the adversarial target and the corresponding preliminary trajectory.

[0015] S4. Retain the position of the adversarial target as the endpoint of the attack vehicle's trajectory, input the endpoint of the attack vehicle's trajectory into the attack trajectory prediction model to obtain the complete trajectory of the attack vehicle, and obtain the trajectory of the background vehicle based on the current and historical motion states of the attack vehicle, the autonomous vehicle, and the background vehicle, the complete trajectory of the attack vehicle, and the background trajectory prediction model.

[0016] Furthermore, the motion adversarial optimization problem is specifically as follows:

[0017]

[0018] Where p, v, a, and j represent the vehicle's position, velocity, acceleration, and jerk on the x and y axes, respectively; θ represents the orientation; and the superscript i and subscript t represent a certain attribute of vehicle i at timestamp t, where i is either ov or av, where av represents an autonomous vehicle and ov represents an attacking vehicle. d represents the location of the attacking vehicle at timestamp t. margin (·,·) represents a function that calculates the shortest distance between a vehicle and a road boundary based on the input location attribute p and map information M, where π a and φ a Let π represent the acceleration distributions of real-world data and generated data, respectively, where Δt represents the time interval between two consecutive timestamps, and π represents the acceleration distribution of generated data. j and φ j , and Let d represent the distribution and probability density function of the jerk, respectively. thres This represents the distance threshold at which the attacking vehicle collides with the autonomous vehicle. Let v represent the position of the autonomous vehicle at timestamp t, determined based on the vehicle's predetermined trajectory. Let E represent the mathematical expectation. max a represents the maximum speed. max This represents the maximum acceleration, (Δθ). max This indicates the maximum value of the change in orientation angle.

[0019] Furthermore, the specific steps for solving the problem are as follows: the motion adversarial optimization problem is transformed into a two-level nested optimization problem, in which a convex optimization problem is corresponding to the interior of the two-level nested optimization problem. For each time stamp, the internal convex optimization problem is solved by an optimization solver, and it is determined whether the internal solution satisfies the constraints of the external problem. The solution that satisfies all constraints is retained, and the optimal solution of the motion adversarial optimization problem is determined according to the objective function of the problem.

[0020] Furthermore, the specific steps for training the trajectory prediction network based on a large-scale real-world traffic scene dataset to obtain the attack trajectory prediction model are as follows:

[0021] A trajectory prediction network is constructed by inputting a large-scale real-world traffic scene dataset into the network. The network includes an encoder and a decoder, and processes all historical vehicle observation trajectories {s} from the large-scale real-world traffic scene dataset. iThe map information M is considered as different polylines, where each polyline contains a set of directed points, each directed point is represented by a descriptor (x, y, θ), where (x, y, θ) represent the two-dimensional position coordinates and orientation of the vehicle, respectively, and i represents the i-th vehicle. The trajectories of all vehicles {s} are considered as different polylines. i The broken line representing the trajectory of the k-th vehicle in the equation is as follows: The polyline corresponding to map information M is represented as M. in The trajectory of the k-th vehicle, corresponding to the polyline input encoder, is encoded as a feature in the multilayer perceptron. feature All features derived from the codes of each vehicle are denoted as Where K is the total number of vehicles, and the polyline M corresponding to the map information M is... in Input the first map into a multilayer perceptron, encoded as local structural features M local Max pooling is performed on the local structural features to output the global structural features M. global Local structural features M local and global structural features M global The concatenated data is then input into a second map multilayer perceptron and subjected to max pooling to obtain the encoded map features M. p All features obtained from the coding of each vehicle and the coded map features M p Inputting a multi-head attention mechanism yields contextual features E. p ;

[0022] context feature E p The predicted target location is obtained by inputting the target location decoding result into the target multilayer perceptron of the decoder. The predicted target location decoding result is then input into the target location encoding multilayer perceptron. The output of the target location encoding multilayer perceptron and the context features E are then input into the target location encoding multilayer perceptron. p The intermediate trajectory is spliced ​​and input into the intermediate trajectory multilayer perceptron, and the intermediate trajectory is output. The loss is calculated based on the intermediate trajectory, and the encoder and decoder are iteratively trained to obtain the attack trajectory prediction model.

[0023] Furthermore, the multi-head attention mechanism includes multiple attention modules, with the last attention module outputting contextual features E. p The output of the intermediate attention module is:

[0024] E j =MultiheadAttn(q=E j-1 ,k=v=[S p M p ]+CE),

[0025] Among them, E jrepresents the output of the j-th attention module and the input of the (j+1)-th attention module. CE represents the learnable category code, and q, k, and v represent the query vector, key vector, and value vector, respectively.

[0026] Furthermore, the specific steps for inputting the endpoint of the attack vehicle's trajectory into the attack trajectory prediction model to obtain the overall trajectory of the attack vehicle are as follows:

[0027] The endpoint of the attack vehicle's trajectory is input into the decoder of the attack trajectory prediction model. The encoder of the attack trajectory prediction model outputs the actual context features. The endpoint of the attack vehicle's trajectory is used as the target position. The target position and the actual context features are input together into the trained intermediate trajectory multilayer perceptron, which outputs the actual intermediate trajectory. The actual intermediate trajectory and the endpoint of the attack vehicle's trajectory are integrated to obtain the complete trajectory of the attack vehicle.

[0028] Furthermore, the specific steps of obtaining the background vehicle trajectory based on the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, and the complete trajectory prediction model of the attacking vehicle and the background trajectory, are as follows:

[0029] The encoded actual context features are obtained by inputting the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle into the encoder of the background trajectory prediction model. The actual context features include context features for each vehicle obtained based on its motion state.

[0030] A three-dimensional descriptor is constructed, which represents the relative position features. The three-dimensional descriptor is input into the relative position encoding multilayer perceptron of the decoder in the background trajectory prediction model to obtain the relative position encoding. The relative position encoding and the actual context features are input into the attention mechanism of the decoder in the background trajectory prediction model to output the updated context features. The updated context features are input into the multilayer perceptron of the decoder in the background trajectory prediction model to obtain the background vehicle trajectory.

[0031] Furthermore, the three-dimensional descriptor is represented as The three-dimensional descriptor represents the offset between the local coordinate systems of two different vehicles, wherein and Let x and y be the coordinates of the origin of the local coordinate system of the i-th vehicle, and x be the orientation of the x-axis, respectively. and Let x and y represent the origin, x and y coordinates, and x-axis orientation of the local coordinate system of vehicle j, respectively.

[0032] Furthermore, the attention mechanism of the decoder in the background trajectory prediction model is as follows:

[0033] Hi =MultiheadAttn(q=H i k=υ=H+LE i )

[0034] Among them LE i H represents the position code of all vehicles relative to the i-th vehicle. i H represents the updated context features of the i-th car, H = [H 1 H 2 ,…,H K ], where q, k, and v represent the query vector, key vector, and value vector, respectively; in the attention mechanism, the output of the previous attention module corresponds to the input of the next attention module.

[0035] Furthermore, the position code of vehicle j relative to vehicle i is obtained using a relative position coding multilayer perceptron:

[0036]

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) This invention formulates adversarial transfer of driving intent as a constrained optimization problem that is insensitive to the planner and can be solved efficiently. This characteristic ensures compatibility with different motion planning algorithms and promotes scalability to large-scale datasets.

[0039] (2) This invention effectively balances adversarialness, realism, and generation efficiency by explicitly decoupling the adversarial transfer of driving intentions of surrounding traffic participants from their motion planning, that is, by separately and independently calculating the endpoint and intermediate trajectories of the attacking vehicle's trajectory. This decoupling design allows the use of large-scale real-world data to capture subtle motion differences that are difficult to model explicitly, thereby enhancing the realism of the simulation. Attached Figure Description

[0040] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0042] Traffic simulation supplements real-world data with synthetic data, effectively evaluating and enhancing the overall capabilities of autonomous vehicles. However, due to the rarity and high cost (even life-threatening risks) of real-world hazardous scenarios in real-world environments, existing evaluations of autonomous driving methods rarely consider hazardous traffic scenarios. Therefore, the need for simulating hazardous traffic scenarios is urgent. Considering the complex dynamic adversarial interactions between the future movement of autonomous vehicles and surrounding traffic participants, simulating such hazardous scenarios from typical conventional scenario data is not easy. To achieve realistic and efficient hazardous scenario simulation, we propose this invention: a safety-critical traffic simulation method based on adversarial transfer of driving intentions, used to efficiently generate high-quality hazardous traffic scenarios and further improve the performance of autonomous driving algorithms.

[0043] The flowchart of this invention is as follows Figure 1 As shown.

[0044] The steps of this invention include:

[0045] S1. Identify the attacking vehicle, the autonomous vehicle, and the background vehicle, and extract the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, as well as predict the future trajectory of the autonomous vehicle.

[0046] S2. Obtain a large-scale real-world traffic scene dataset, train a trajectory prediction network based on the large-scale real-world traffic scene dataset, and obtain the attack trajectory prediction model and the background trajectory prediction model.

[0047] S3. Based on the predicted future trajectory of autonomous vehicles, establish a motion adversarial optimization problem and solve it to obtain the position of the adversarial target and the corresponding preliminary trajectory.

[0048] S4. Retain the position of the adversarial target as the endpoint of the attack vehicle's trajectory, input the endpoint of the attack vehicle's trajectory into the attack trajectory prediction model to obtain the complete trajectory of the attack vehicle, and obtain the trajectory of the background vehicle based on the current and historical motion states of the attack vehicle, the autonomous vehicle, and the background vehicle, the complete trajectory of the attack vehicle, and the background trajectory prediction model.

[0049] In safety-critical scenarios, most accidents are caused by aggressive driving behavior. In this patent, we simplify the scenario by selecting a vehicle exhibiting aggressive driving behavior, referred to as the attacking vehicle (OV), and having it closely interact with an automated driving vehicle (AV) to cause a collision. All other vehicles besides the attacking vehicle and the automated driving vehicle are referred to as background vehicles (BV). In most cases, background vehicles do not engage in aggressive driving behavior but rather react to other vehicles.

[0050] like Figure 1As shown, this invention comprises two main parts: adversarial transfer of driving intent and intent-based motion planning. First, the adversarial transfer of driving intent is transformed into a constrained optimization problem to find feasible solutions for directly attacking autonomous vehicles. Specifically, the adversarial intent is characterized by the location of the attacking vehicle at its final future moment, also known as the adversarial target. Subsequently, guided by the adversarial target, we employ a target-based motion prediction model to plan the future motion of the attacking vehicle. Supported by large-scale real-world data, the motion prediction model learns from common scenarios and can generate realistic vehicle trajectories based on arbitrary target locations. Therefore, this invention can effectively generate rare and challenging safety-critical scenarios while ensuring realism. Figure 1 In this invention, a decoupling strategy is employed to achieve both adversarial and realistic characteristics in the generated scenarios. First, the adversarial transfer of driving intent is formulated as a constrained optimization problem to find feasible solutions for directly attacking the autonomous vehicle. Then, guided by the adversarial intent, IntSim utilizes a motion prediction model to generate realistic trajectories. Here, AV and OV represent the autonomous vehicle and the attacking vehicle, respectively.

[0051] In S1 of this invention, the process of extracting the trajectory of the autonomous vehicle is as follows:

[0052] To determine the manner of a collision with an autonomous vehicle, it is first necessary to estimate the vehicle's future trajectory. A simple solution is to use a neural network to predict the vehicle's trajectory. However, due to the black-box nature of the planner and the scarcity of vehicle trajectory samples, inaccurate predictions are likely, making it challenging to directly use a neural network to fit the trajectory of an autonomous driving system in practical applications. Therefore, we estimate the vehicle's position by sampling the autonomous vehicle's trajectory in a typical scenario. We do not require additional information (e.g., the probability of taking an action), only basic position and orientation information. This invention uses the vehicle's trajectory collected in a corresponding typical scenario closed-loop test as its predicted future trajectory.

[0053] In S2 of this invention, the training process of the trajectory prediction model is as follows:

[0054] A neural network-based trajectory prediction model is used as the basis for generating realistic trajectories. This model is trained on a large-scale real-world traffic scene dataset to capture complex interaction patterns and generate highly realistic trajectories. The neural network model employs an encoder-decoder architecture, taking contextual information as input and outputting the future trajectory. Specifically, we utilize a vectorized representation in a local coordinate system centered on each vehicle to describe the properties of the context encoding. We then calculate the trajectories of all vehicles {s}. iThe map information M is considered as distinct polylines, where each polyline contains a set of directed points, each represented by a descriptor (x, y, θ) including its two-dimensional location coordinates and orientation. For example, the historical trajectory input of the k-th vehicle in the scene is represented as... It is encoded as a feature by a multilayer perceptron (MLP).

[0055]

[0056] Simultaneously, the local and global structures of the map polylines are considered, and the input map polylines M are processed through two multilayer perceptrons (MLPs). in Encode:

[0057] M local =MLP m1 (M in M global =φ(M local M p =φ(MLP m2 ([M local M global ])),

[0058] Where φ represents the max pooling operation, and [,] represents the concatenation operation. M local and M global These represent local and global structural features, respectively. M p This represents the encoded map features. Then, a multi-head attention mechanism is used to fuse the information:

[0059] E j =MultiheadAttn(q=E j-1 , k = υ = [S p M p ]+CE),

[0060] Where E j E represents the output of the j-th attention module and the input of the (j+1)-th attention module. 0 Will be initialized as vehicle code We introduce a learnable category encoding (CE) to distinguish and encode differences between different vehicle types and map features. Through an attention mechanism, different features interact fully, and the attention module ultimately outputs E. pThe model implicitly encodes all the necessary information for generating future trajectories. This trajectory prediction model is used simultaneously for predicting the trajectories of both the attacking vehicle and all background vehicles; that is, it generates both attack trajectory prediction and background trajectory prediction models during training. During the inference phase, the attacking vehicle prediction is used in generating its complete trajectory. Simultaneously, this prediction model is used to jointly predict the trajectories of all background vehicles. The model's input is the current and historical motion states of all vehicles and the obtained trajectory of the attacking vehicle (not the trajectory in the dataset), and its output is the future motion of all background vehicles, enabling it to effectively respond to the adversarial behavior of the attacking vehicle. The training of both models is based on a large-scale real-world traffic scene dataset. When training the prediction model for the attacking vehicle, the trajectories of vehicles in the large-scale real-world traffic scene dataset are used. Since the real-world dataset represents a general scenario, the vehicle trajectories in the large-scale real-world traffic scene dataset are not aggressive. Similarly, the prediction model for all background vehicle trajectories is trained using non-aggressive future trajectories from the real-world dataset.

[0061] The encoder in the background trajectory prediction model is the same as the encoder in the attack trajectory prediction model, through which the contextual features of each vehicle are obtained. Here, the superscript i denotes the contextual features of the i-th vehicle. In particular, a future trajectory multilayer perceptron is introduced to encode the future trajectory of the attacking vehicle. The resulting features are added to the contextual features of the attacking vehicle to generate the actual contextual features.

[0062] In the decoder of the background trajectory prediction model, the contextual features of all vehicles are input into a multi-head attention mechanism, which includes multiple attention modules that fuse information from other vehicles to update the vehicle contextual features. The contextual features of all background vehicles are then input into the multilayer perceptron of the decoder to obtain the joint predicted trajectory of all background vehicles.

[0063] Attack trajectory prediction models need to be able to generate trajectories based on given target locations. To this end, we establish a two-stage decoding scheme, where the model first predicts the target location using MLP taps, and then fills in the intermediate trajectory based on the target location. The intermediate trajectory is generated by encoding the given target location and contextual features E. p Decoding is performed by combining these methods. The decoding process can be formalized as follows:

[0064] s T =MLP goal (E p ).

[0065] s 0:T-1 =MLP traj ([E p MLP enc (s T)]),

[0066] Where s t =(x t ,y t ,θ t This includes the vehicle's position and orientation at timestamp t, where T is the maximum timestamp. MLP enc Encode the target location, while MLP goal and MLP traj These are the taps for predicting the target location and the intermediate trajectory, respectively. When predicting the actual attack trajectory, the endpoint of the initial trajectory is used as s. T Input the decoder of the attack trajectory prediction model.

[0067] In S3 of this invention, an optimization problem is used to solve for the adversarial target position and guide trajectory generation. To simulate a collision with an autonomous vehicle, we search for an attacking vehicle with adversarial intent and obtain the adversarial trajectory. We use two-dimensional vectors to describe the vehicle's position, velocity, acceleration, and jerk on the x and y axes, denoted by p, v, a, and j respectively, and θ represents the orientation. The superscript i and subscript t represent a certain attribute of vehicle i at timestamp t, for example, This indicates the location of the attacking vehicle (OV) at timestamp t. Additionally, we use d... margim (·,·) represents a function that calculates the shortest distance between a vehicle and the road boundary based on the input location attribute p and map information M (negative values ​​indicate out-of-bounds). We use π a and φ a Let represent the acceleration distributions of real-world data and generated data, respectively. We use... and Let π represent the probability density function under the corresponding distribution. Similarly, the distribution and probability density function of jerk are represented by π. j and φ j , and The time interval between two consecutive timestamps is represented by Δt.

[0068] The key to scenario simulation lies in ensuring a high degree of realism, especially for safety-critical scenarios, where the adversarial nature of autonomous vehicles needs further consideration. Realism includes physical feasibility and the plausibility of driving behavior (e.g., driving along the center line of a lane). Adversarial nature means that the trajectory of the attacking vehicle will overlap with the estimated trajectory of the autonomous vehicle at some point, leading to a collision. As mentioned above, an optimization problem is introduced to calculate the position of the adversarial target. However, to ensure that the adversarial intent is physically and realistically feasible, the optimization problem optimizes the entire trajectory, not just the position of the adversarial target.

[0069] Mathematically, the optimization problem can be written as:

[0070]

[0071] v max a max and (Δθ) max d represents the physically feasible maximum velocity, acceleration, and steering angle changes within a single time interval. thres This represents the distance threshold at which the attacking vehicle collides with the autonomous vehicle.

[0072] Then, we solve the optimization problem. To achieve an efficient solution, we further simplify the optimization problem and transform it into a two-level nested optimization problem. The inner optimization corresponds to a convex optimization problem, which can be solved efficiently by a convex optimization solver. Based on our observations, most scenarios with a high probability of traffic accidents have the following characteristics: (1) there is no separation between the autonomous vehicle (AV) and the attacking vehicle (OV) in an undriveable area; (2) the attacking vehicle collides with the autonomous vehicle with real driving behavior rather than sudden maneuvers (such as a sudden U-turn). Therefore, we incorporate the constraint equations (Eq.(S1f)) and (Eq.(S1i)) into the outer optimization problem. Accordingly, the optimization problem is transformed into:

[0073]

[0074] In the implementation, all future timestamps are iterated over. For each timestamp t... k An open-source optimization solver is used to solve the internal optimization problem. Then, the returned trajectory is checked to see if it satisfies the constraints of the external problem. The optimal solution that satisfies all constraints and obtains the optimal value according to the objective function is retained.

[0075] While the adversarial trajectory obtained by solving an optimization problem provides a feasible way to collide with autonomous vehicles, it ignores the rich knowledge of driving habits contained in large-scale real-world data. Therefore, the resulting trajectory only resembles a human driver kinematically, but lacks reasonable driving intentions and habits. To address this, we only consider the final position of the trajectory as the target reflecting the adversarial intent, and use the aforementioned target-based trajectory prediction model to fill in the intermediate trajectories. The prediction model, by utilizing and learning from a large amount of real-world driving data, captures complex interaction patterns, making the generated adversarial trajectory more realistic. This mechanism endows the attacking vehicle with a reasonable driving intent, rather than simply aiming to collide with the autonomous vehicle.

[0076] In S4 of this invention, a Transformer-based neural network is used to generate the trajectories of other vehicles. To jointly generate the trajectories of other vehicles, the context corresponding to each other vehicle needs to be encoded first, and then information exchange between them needs to be facilitated. For efficient computation, we reuse the previously encoded context features E. p However, each vehicle encodes its context in its own local coordinate system and cannot interact with other vehicles. We use learnable relative position encoding to encode the relative differences between different coordinate systems. In the global coordinate system, a two-dimensional coordinate system with origin O can be uniquely described by its origin coordinates and x-axis orientation, formalized as follows: Given two coordinate systems, Oxy and O′x′y′, their relationship can be represented by a three-dimensional descriptor. We use this descriptor to describe the relative position features and add them to the context features. Based on the relative position features, we can use an attention mechanism to facilitate information exchange between vehicles and further use MLP taps to jointly decode the trajectories of other vehicles.

[0077] The beneficial effects of this invention are as follows:

[0078] (1) This invention transforms the generation of safety-critical scenarios into a two-stage process of adversarial transfer of driving intent and intent-based motion planning. By explicitly decoupling the driving intent of the attacking vehicle from its motion planning, we use two independent modules to achieve flexible and efficient attack behavior and ensure that the motion planning conforms to the constraints of real-world behavior.

[0079] (2) This invention first describes the adversarial transfer of driving intent as an optimization problem, and obtains the optimal adversarial driving objective that leads to a collision with an autonomous vehicle based on the optimization solution. This promotes extensive exploration of diverse attack behaviors and provides an efficient solution.

[0080] (3) Furthermore, with the above-mentioned optimal adversarial driving objective as a condition, intention-based motion planning, with the help of powerful deep neural network models and large-scale real-world data, can simulate the real motion behavior of vehicles and thus obtain adversarial and realistic motion trajectories.

[0081] The method proposed in this invention effectively balances adversarialism, realism, and generative efficiency by explicitly decoupling the adversarial transfer of driving intentions of surrounding traffic participants from their motion planning. This decoupling design allows us to leverage large-scale real-world data to capture subtle motion differences that are difficult to model explicitly, thereby enhancing the realism of the simulation.

[0082] This invention proposes to formulate adversarial transfer of driving intent as a constrained optimization problem that is insensitive to the planner and can be solved efficiently. This characteristic ensures compatibility with different motion planning algorithms and promotes scalability to large-scale datasets.

[0083] Numerical and simulation experiments have already validated the proposed IntSim method. A series of open-loop and closed-loop experiments were conducted on the publicly available real-world nuScenes and Waymo datasets, using rule-based and reinforcement learning-based planners to evaluate the method. These experimental results demonstrate significant improvements in adversarial, realistic, and efficient simulation of hazardous traffic scenarios. Furthermore, by training on safety-critical scenarios generated by IntSim, the planner exhibits a stronger ability to handle challenging situations, successfully completing path planning and avoiding collisions, while maintaining good performance in normal scenarios.

[0084] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A traffic simulation method based on adversarial transfer of driving intent, characterized in that, The method includes the following steps: S1. Identify the attacking vehicle, the autonomous vehicle, and the background vehicle, and extract the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, as well as predict the future trajectory of the autonomous vehicle. S2. Obtain a large-scale real-world traffic scene dataset, train a trajectory prediction network based on the large-scale real-world traffic scene dataset, and obtain the attack trajectory prediction model and the background trajectory prediction model. S3. Based on the predicted future trajectory of autonomous vehicles, establish a motion adversarial optimization problem and solve it to obtain the position of the adversarial target and the corresponding preliminary trajectory. S4. Retain the position of the adversarial target as the endpoint of the attack vehicle's trajectory. Input the endpoint of the attack vehicle's trajectory into the attack trajectory prediction model to obtain the complete trajectory of the attack vehicle. Based on the current and historical motion states of the attack vehicle, the autonomous vehicle, and the background vehicle, and the complete trajectory of the attack vehicle and the background trajectory prediction model, obtain the trajectory of the background vehicle. The motion adversarial optimization problem is specifically as follows: in, p, v, a and j These represent the vehicle's position, velocity, acceleration, and jerk on the x and y axes, respectively. θ Indicates direction, superscript i and subscript t Represents timestamp t Vehicle i A certain attribute, in which i for ov or av, av Indicates autonomous vehicles, ov Indicates an attack on the vehicle. This indicates the location of the attacking vehicle at timestamp t. Indicates based on the input position attribute p and map information M A function to calculate the shortest distance between a vehicle and the road boundary. and Represent the acceleration distributions of real-world data and generated data, respectively. This represents the time interval between two consecutive timestamps. and , and Let represent the distribution and probability density function of the jerk, respectively. This represents the distance threshold at which the attacking vehicle collides with the autonomous vehicle. Let E represent the position of the autonomous vehicle at timestamp t, determined based on the vehicle's predetermined trajectory. This indicates the maximum speed. This indicates the maximum acceleration. This represents the maximum value of the change in orientation angle. The specific steps for solving the problem are as follows: the motion adversarial optimization problem is transformed into a two-level nested optimization problem, in which a convex optimization problem is corresponding to the interior of the two-level nested optimization problem. For each time stamp, the convex optimization problem is solved by the optimization solver, and it is determined whether the interior solution satisfies the constraints of the external problem. The solution that satisfies all constraints is retained, and the optimal solution of the motion adversarial optimization problem is determined according to the objective function of the problem.

2. The traffic simulation method based on adversarial transfer of driving intent, as described in claim 1, is characterized in that... The specific steps for training the trajectory prediction network based on a large-scale real-world traffic scenario dataset to obtain the attack trajectory prediction model are as follows: A trajectory prediction network is constructed by inputting a large-scale real-world traffic scene dataset into the network. The network includes an encoder and a decoder, and incorporates historical vehicle observation trajectories and map information from the large-scale real-world traffic scene dataset. M Consider them as distinct polylines, where each polyline contains a set of directed points. Each directed point is represented by a descriptor, representing the vehicle's two-dimensional position coordinates and orientation, respectively. Let i represent the i-th vehicle, and let i be the i-th vehicle among all vehicle trajectories. c The broken line representing the trajectory of the vehicle is as follows: Map information M The corresponding broken line is represented as , No. c The trajectory of the vehicle, corresponding to the polyline input encoder, is encoded as a feature in the multilayer perceptron. ,feature All features derived from the codes of each vehicle are denoted as ,in C The total number of vehicles, map information M Corresponding broken line Input the first map into a multilayer perceptron and encode it as local structural features. Max pooling is performed on the local structural features to output the global structural features. Local structural features and global structural features The concatenated data is then input into a second map multilayer perceptron and subjected to max pooling to obtain the encoded map features. All features obtained from the coding of each vehicle and the features of the coded map Input multi-head attention mechanism to obtain contextual features ; context features The predicted target location is obtained by inputting the target location decoding result into the target multilayer perceptron of the decoder. The predicted target location decoding result is then input into the target location encoding multilayer perceptron, which outputs the target location encoding multilayer perceptron and the context features. The intermediate trajectory is spliced ​​and input into the intermediate trajectory multilayer perceptron, and the intermediate trajectory is output. The loss is calculated based on the intermediate trajectory, and the encoder and decoder are iteratively trained to obtain the attack trajectory prediction model.

3. The traffic simulation method based on adversarial transfer of driving intent, as described in claim 2, is characterized in that... The multi-head attention mechanism includes multiple attention modules, with the last attention module outputting contextual features. The output of the intermediate attention module is: in, represents the output of the j-th attention module and the input of the (j+1)-th attention module, where CE represents the learnable category code, and q, k, and v represent the query vector, key vector, and value vector, respectively.

4. The traffic simulation method based on adversarial transfer of driving intent, as described in claim 1, is characterized in that... The specific steps for inputting the endpoint of the attack vehicle's trajectory into the attack trajectory prediction model to obtain the overall trajectory of the attack vehicle are as follows: The endpoint of the attack vehicle's trajectory is input into the decoder of the attack trajectory prediction model. The encoder of the attack trajectory prediction model outputs the actual context features. The endpoint of the attack vehicle's trajectory is used as the target position. The target position and the actual context features are input together into the trained intermediate trajectory multilayer perceptron, which outputs the actual intermediate trajectory. The actual intermediate trajectory and the endpoint of the attack vehicle's trajectory are integrated to obtain the complete trajectory of the attack vehicle.

5. A traffic simulation method based on adversarial transfer of driving intent, as described in claim 4, is characterized in that... The process of obtaining the background vehicle trajectory based on the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle, as well as the complete trajectory of the attacking vehicle and the background trajectory prediction model, specifically involves: The encoded actual context features are obtained by inputting the current and historical motion states of the attacking vehicle, the autonomous vehicle, and the background vehicle into the encoder of the background trajectory prediction model. The actual context features include context features for each vehicle obtained based on its motion state. A three-dimensional descriptor is constructed, which represents the relative position features. The three-dimensional descriptor is input into the relative position encoding multilayer perceptron of the decoder in the background trajectory prediction model to obtain the relative position encoding. The relative position encoding and the actual context features are input into the attention mechanism of the decoder in the background trajectory prediction model to output the updated context features. The updated context features are input into the multilayer perceptron of the decoder in the background trajectory prediction model to obtain the background vehicle trajectory.

6. A traffic simulation method based on adversarial transfer of driving intent, as described in claim 5, is characterized in that... The three-dimensional descriptor is represented as The three-dimensional descriptor represents the offset between the local coordinate systems of two different vehicles, wherein... , and They represent the first i The xy-axis coordinates of the origin of the vehicle's local coordinate system and x Axis orientation, , and Let x and y coordinates of the origin of the local coordinate system of vehicle j be respectively. x Axis orientation.

7. A traffic simulation method based on adversarial transfer of driving intent, as described in claim 6, is characterized in that... The attention mechanism of the decoder in the background trajectory prediction model is as follows: Among them LE i This represents the position code of all vehicles relative to the i-th vehicle. Represents the updated context features of the i-th car. q, k, and v represent the query vector, key vector, and value vector, respectively; in the attention mechanism, the output of the previous attention module corresponds to the input of the next attention module.

8. A traffic simulation method based on adversarial transfer of driving intent, as described in claim 7, is characterized in that... The position encoding of vehicle j relative to vehicle i is obtained using a relative position encoding multilayer perceptron: 。

Images