Automatic driving safety key simulation scene generation method based on adversarial generation and co-evolution

Abstract

The invention discloses an automatic driving safety key simulation scene generation method based on adversarial generation and co-evolution. The method comprises the following steps: receiving a basic traffic scene described by a natural language, generating an antagonistic element scene containing security threats by using a large language model in combination with a traffic safety knowledge base, and analyzing the antagonistic element scene into an executable scene script; constructing a multi-agent confrontation collaboration diagram based on the meta-scene, and recognizing a key background vehicle through a cross-timing attention mechanism in combination with a time mask and time decay mechanism; and performing disturbance optimization on the key background vehicle trajectory to generate an automatic driving test scene. According to the method, a scientific and systematic solution with engineering operability is provided for safety verification of the automatic driving system when the automatic driving system faces real traffic challenges such as multi-source intervention and dynamic collaborative threat, and the method has wide adaptation capability and important industrial popularization value.

Description

Technical Field

[0001] This invention relates to a method for generating autonomous driving safety-critical simulation scenarios based on adversarial generation and co-evolution, and also to a corresponding device for generating autonomous driving safety-critical simulation scenarios, belonging to the field of autonomous driving technology. Background Technology

[0002] With the rapid development of autonomous driving technology, its safety verification in real traffic environments has become a key challenge restricting its large-scale deployment and commercial application. Compared with field testing, simulation-based verification methods have become an important means of evaluating the performance of autonomous driving systems, especially the robustness of autonomous driving test scenarios, due to their controllability, repeatability, and lower testing costs. However, existing technologies still have significant shortcomings in generating test scenarios with high risk and diversity.

[0003] On the one hand, most current scenario generation methods rely on predefined threat templates or rule-driven strategies, typically requiring manual construction of rule bases or reliance on expert experience to define traffic violation behaviors. While these methods are effective in handling standardized scenarios, they often lack coverage of edge cases and complex interactions in the real world, resulting in insufficient diversity and aggressiveness of the generated scenarios. For example, traditional rule-driven methods simulate typical violations using static rules, making it difficult to describe and generate atypical or risky scenarios with ambiguous policy boundaries, thus failing to effectively expose potential vulnerabilities in the perception, prediction, and decision-making chains of autonomous driving systems.

[0004] On the other hand, existing simulation testing frameworks generally use static scripts to generate test cases, lacking the ability to dynamically model and intervene in the behavior of traffic participants. Especially in complex traffic flows, traditional adversarial methods mostly only apply perturbations to a single agent, ignoring the risk superposition effect caused by multi-vehicle cooperative interaction, making it difficult to construct realistic threat scenarios sufficient to challenge advanced autonomous driving systems. At the same time, existing methods often have a conflict between semantic logical rationality and physical executability in the scenario generation process, making it difficult to improve the system adversarial strength of the test scenario while maintaining naturalness. Summary of the Invention

[0005] The primary technical problem to be solved by this invention is to provide a method for generating autonomous driving safety-critical simulation scenarios based on adversarial generation and co-evolution.

[0006] Another technical problem to be solved by the present invention is to provide a corresponding simulation scenario generation device for autonomous driving safety.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] According to a first aspect of the present invention, a method for generating safety-critical simulation scenarios for autonomous driving based on adversarial generation and co-evolution is provided, comprising the following steps:

[0009] (1) Receive basic traffic scenarios described in natural language;

[0010] (2) Combine traffic safety knowledge base and use large language model to generate adversarial meta-scenarios containing safety threats;

[0011] (3) Parse the semantic elements of the adversarial meta-scene and generate a scene script that can be executed by the autonomous driving simulation platform;

[0012] (4) Construct a multi-agent adversarial cooperation graph based on the adversarial meta-scenario, and model the spatiotemporal correlation between the autonomous vehicle, adversarial agents and background vehicles through a cross-temporal attention mechanism in order to identify key background vehicles.

[0013] (5) The trajectory segments of the key background vehicles are perturbed and optimized to generate autonomous driving test scenarios.

[0014] Preferably, in step (2), the traffic safety knowledge base includes information on road traffic regulations, driving behavior norms and testing knowledge, and real accident and risk case data.

[0015] A preferred approach to generating adversarial meta-scenes using a large language model includes the following sub-steps:

[0016] The user-input basic scenario description P base The encoding is performed as a semantic vector, and cosine similarity matching is performed between the vector and the embedded representation of each segment in the traffic safety knowledge base.

[0017] Search function f R Selecting the basic scene description P base The k most relevant knowledge fragments form the context for enhanced hints.

[0018] The obtained context It is concatenated into the basic input prompts as semantic guidance for the generation of the large language model, in order to generate adversarial meta-scenes containing security threats.

[0019] Preferably, in step (3), the executable scene script is generated through the following sub-steps:

[0020] The text content is mapped to a structured scene script, which includes vehicle type selection, initial spatial position parameterization, motion trajectory generation strategy and embedded behavioral logic.

[0021] The generated structured configuration is converted into a domain-specific language script in the standard Scenic format to ensure compatibility with the intended autonomous driving simulation platform.

[0022] Preferably, constructing the multi-agent adversarial cooperative graph includes the following sub-steps:

[0023] Extract trajectory data of different agents in autonomous driving test scenarios and construct a temporal attention matrix;

[0024] A temporal masking mechanism is adopted to restrict each query frame to focus only on the current frame and historical frames of the same background vehicle, so as to avoid interference from future frame information; at the same time, a temporal decay mechanism is adopted to introduce an exponential decay factor γ∈(0,1] in the attention calculation.

[0025] Calculate the risk score of each background vehicle across the entire time series; sort by risk score and select the top K background vehicles to form the multi-agent adversarial cooperation graph, which will serve as the target objects for subsequent trajectory perturbation optimization.

[0026] Preferably, in the attention matrix, the first T rows represent the behavioral state of the vehicle in each time frame, the last T rows represent the behavior of the adversarial agent; the columns represent the states of all background vehicles in each time frame; and the matrix element values ​​reflect the correlation between the behavior of the vehicle or the adversarial agent and the background vehicles in a specific time frame.

[0027] Preferably, the intelligent agent includes the autonomous vehicle, the adversarial intelligent agent, and the background vehicle; wherein the autonomous vehicle is the vehicle controlled by the autonomous driving system under test, the adversarial intelligent agent is the main source of threat in the meta-scenario, and the background vehicle is a vehicle that does not engage in offensive behavior but has the potential for collaborative intervention.

[0028] Preferably, step (5), which involves perturbation optimization of the trajectory segment of the key background vehicle, specifically includes:

[0029] For the selected set of cooperative perturbation agents and their perturbable trajectory segments, the trajectory perturbation optimization process is formalized as an optimization problem with the goal of maximizing the degradation of the vehicle's behavior;

[0030] A multi-objective perturbation optimization loss function is constructed and solved using an iterative gradient update strategy: the gradient of the trajectory coordinates of each perturbation segment with respect to the loss function is calculated, the trajectory parameters are updated with a fixed step size along the direction of risk improvement, and the perturbation trajectory is projected back into the physical feasible region after each update to ensure traffic rationality. This process is repeated until convergence or the maximum number of iterations is reached.

[0031] A perturbation parameter distribution modeling mechanism is introduced, which uses interval or probability distribution to represent perturbation parameters and dynamically adjusts the sampling range to generate a trajectory perturbation population with behavioral differences.

[0032] The sample that most significantly interferes with the behavior of the autonomous driving system and has the strongest perceptual misleading effect is selected from the aforementioned trajectory perturbation family and used as the output of the autonomous driving test scenario.

[0033] Preferably, the method for generating key simulation scenarios for autonomous driving safety further includes the following steps:

[0034] The basic scenario, the meta-scenario, and the cooperative disturbance scenario are run sequentially in the autonomous driving simulation platform.

[0035] Collect collision rate, path offset distance, and control oscillation indicators;

[0036] By comparing the differences in indicators across the three scenarios, the robustness degradation trend of autonomous driving systems can be quantified.

[0037] According to a second aspect of the present invention, an apparatus for generating autonomous driving safety-critical simulation scenarios is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, enables the processor to implement the above-described method for generating autonomous driving safety-critical simulation scenarios.

[0038] Compared with existing technologies, this invention constructs an integrated testing closed loop covering scenario generation, perturbation design, simulation execution, and behavior evaluation. It combines a trajectory perturbation-based behavioral intervention mechanism, a multi-agent collaborative strategy challenge framework, and a multi-dimensional robustness evaluation system covering safety, functionality, and stability. This enables system-level performance profiling of autonomous driving systems in complex dynamic traffic scenarios. This method can not only automatically identify and construct high-risk test scenarios with explicit threat structures and potential triggering factors, but also quantify the performance degradation process of the tested system under different intervention levels, forming an attack-response-evaluation closed loop. This effectively fills the gaps in existing technologies regarding composite behavioral modeling, policy vulnerability disclosure, and quantitative evaluation methods. Ultimately, this invention provides a scientific, systematic, and engineering-operable solution for the safety verification of autonomous driving systems facing real-world traffic challenges such as multi-source intervention and dynamic collaborative threats, possessing broad adaptability and significant industrial promotion value. Attached Figure Description

[0039] Figure 1 This is a logical framework diagram of the autonomous driving test scenario constructed in an embodiment of the present invention;

[0040] Figure 2 This is a flowchart illustrating the method for generating key safety simulation scenarios for autonomous driving in an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of the structure of the autonomous driving safety key simulation scenario generation device in an embodiment of the present invention. Detailed Implementation

[0042] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0043] The technical concept of this invention lies in first receiving basic traffic scene information described in natural language, combining it with a traffic safety knowledge base, and then using a large language model to generate an adversarial meta-scenario with safety threats. Next, a multi-agent adversarial cooperative graph is constructed based on the meta-scenario, and a cross-temporal attention mechanism is used to evaluate and identify key background vehicles. Then, trajectory segments of the key background vehicles are extracted, a perturbation optimization objective function is constructed, and their trajectories are perturbed and optimized, ultimately generating a high-risk, highly interactive autonomous driving test scenario. Based on this, the generated autonomous driving test scenario is imported into an autonomous driving simulation platform for testing, data is recorded, and evaluation indicators are calculated to quantify the declining trend of system robustness, thereby achieving a robustness assessment of the autonomous driving system in complex traffic environments.

[0044] like Figure 1 As shown, the autonomous driving safety-critical simulation scenario generation method provided in this embodiment of the invention can be divided into two main parts: meta-scenario generation and evolution. In the meta-scenario generation stage, a basic scenario description is first input, such as "the vehicle is traveling straight at a constant speed in the lane." Then, a large language model is used for reasoning, combining a driving knowledge base containing road traffic regulations, driving behavior norms tests, and real accident risk cases. Next, through the description of safety-critical scenarios, such as the vehicle not seeing a pedestrian due to obstruction by a truck, and the pedestrian suddenly appearing, leading to a potential collision risk, an adversarial meta-scenario is generated. Simultaneously, executable simulation code (such as Scenic scripts) defines the various elements and behavioral logic in the scenario, such as vehicles, pedestrians, and their trajectories and behaviors, thereby constructing a meta-scenario that can run on an autonomous driving simulation platform.

[0045] During the evolution phase, embodiments of this invention introduce background traffic flow and identify key background vehicles by constructing an adversarial cooperative graph. While these vehicles are not inherently aggressive, they may amplify the threat level of the scenario under certain conditions. Through agent selection and trajectory segment selection, vehicles with potential impact on the vehicle's safety are identified, and their trajectories are optimized through perturbation. The perturbed traffic flow reveals the trajectory changes of these background vehicles, thereby creating a more challenging and realistically risky complex traffic scenario. This process simulates the dynamic interactions and potential risks among multiple traffic participants in a real traffic environment, providing a more rigorous and realistic testing environment for autonomous driving systems.

[0046] like Figure 2As shown, the method for generating key safety simulation scenarios for autonomous driving provided in this embodiment of the invention includes at least the following steps:

[0047] (1) Receive basic traffic scenarios described in natural language;

[0048] (2) Combining the traffic safety knowledge base, use a large language model to generate adversarial meta-scenes containing safety threats.

[0049] (3) Parse the semantic elements of the adversarial meta-scene and generate a scene script that can be executed by the autonomous driving simulation platform;

[0050] (4) Construct a multi-agent adversarial cooperation graph based on the adversarial meta-scenario, and model the spatiotemporal correlation between the autonomous vehicle, adversarial agents and background vehicles through a cross-temporal attention mechanism in order to identify key background vehicles.

[0051] (5) The trajectory segments of the key background vehicles are perturbed and optimized to generate autonomous driving test scenarios.

[0052] After obtaining the above-mentioned autonomous driving test scenarios, progressive testing can be performed in the autonomous driving simulation platform: the basic scenario, the meta scenario, and the cooperative disturbance scenario are run in sequence, and the collision rate, path deviation distance, and control oscillation indicators are collected in real time; the robustness degradation trend of the autonomous driving system is quantified based on the comparison of indicators of the three-stage scenarios.

[0053] The following section details the specific implementation steps of this method for generating key safety simulation scenarios for autonomous driving:

[0054] First, by introducing a large language model with natural language understanding and generation capabilities, a meta-scenario containing traffic violation behavior and threat triggering logic is constructed.

[0055] The above steps aim to reveal the potential safety vulnerabilities of the tested autonomous driving system in following traffic rules, responding to emergencies, and situations where behavioral boundaries are blurred. Its core objective is to take a regular traffic scenario described in natural language as input, combine it with well-defined and clearly coded traffic knowledge, automatically reason and generate adversarial traffic events with potential safety threats, and then transform them into structured and executable scenario scripts to achieve effective mapping and implementation from the semantic layer to the behavioral layer.

[0056] To achieve the above objectives, the autonomous driving system first receives a basic traffic scenario described in natural language provided by the user. The content can be a concise or complex situational narrative (e.g., the vehicle is traveling straight at a constant speed in the lane). Such benign descriptions are widespread in real traffic environments and usually do not contain obvious dangerous elements.

[0057] In order to uncover potential security risks from such scenarios and reveal the key factors that may trigger system vulnerabilities, this invention constructs a well-organized traffic safety knowledge base with clear rules, and uses it as an external retrieval source or contextual hint for a large language model, thereby guiding the model's reasoning and constructing a behavioral agent with adversarial features.

[0058] Specifically, this traffic safety knowledge base It consists of the following:

[0059] ① Road traffic regulations information D r It covers road safety-related regulations and provisions from multiple countries and regions, including China, the United States, and Germany. The content includes driving priority, lane changing rules, signal compliance, and yielding obligations. Corresponding constraint logic models have been established to define the boundaries of compliant behavior for traffic participants in different scenarios.

[0060] ②Driving behavior norms and test knowledge D l It includes a large number of standardized driver training and examination questions, including rule understanding, scenario judgment, emergency response, etc., and provides a wealth of typical driving behavior decision templates, reflecting the semantic differences between compliant operation and common mistakes;

[0061] ③Real accident and risk case data D c Based on collision event classification reports released by the National Highway Traffic Safety Administration and other institutions, this study summarizes and organizes various high-incidence accident patterns (such as illegal parking at red lights, obstructing overtaking, and being rear-ended while turning left), and marks the corresponding scene elements and behavioral triggering factors to support the practical feasibility of adversarial behavior.

[0062] The aforementioned knowledge content is organized in a fragmented form and embedded into the reasoning process of the language model through a retrieval-enhanced generation mechanism. Specifically, firstly, the basic scenario description P input by the user is... base The data is encoded into semantic vectors and then matched with the cosine similarity of the embedding representations of each segment in the knowledge base. The retrieval function f... R Select with P base The k most relevant knowledge fragments form the context for enhanced hints. This process can be formally represented as:

[0063]

[0064] Where ε(·) represents the text embedding function, Let C be the cosine similarity function, C be the set of knowledge fragments, and e be the cosine similarity function. i For each fragment, an embedding vector is generated, where k is a positive integer. Ultimately, the obtained context... It is concatenated into the basic input prompts to serve as semantic guidance for the generation of the large language model, thereby ensuring that the generated content is semantically reasonable and has adversarial features that reveal potential risks.

[0065] On the other hand, this invention employs multiple input prompt template structures to provide knowledge prompts, adapting to the requirements of different large language models in terms of input format and context window capabilities. For example, prompt modes such as few-shot example guidance can be flexibly applied to various tasks such as static rule reasoning and dynamic interactive reasoning. During semantic generation, the model generates content that includes not only textual descriptions of violations but also the temporal logic of the behavior, the vehicle's role in the scene, initial state parameter configuration, traffic signal status, environmental conditions, and other key dimensions. This ensures the integrity of the generated results in terms of behavioral logic and gives them clear semantic instructions during simulation execution.

[0066] In one embodiment of the present invention, after fusing basic scene descriptions and knowledge prompts, the large language model ultimately generates a natural language output with adversarial semantic features. This natural language output describes a meta-scene generated by the large language model, containing core threat events, typically including key elements such as behavioral conflict points, role relationships, and temporal triggering conditions. For example, the model might generate the following: A car is traveling at a constant speed in a lane. A pedestrian is preparing to cross the road behind a parked truck on the right. Due to the truck's obstruction, the car does not see the pedestrian, and the pedestrian only appears in the car's field of vision when the car is approaching. Even if the car is not speeding, it is difficult to effectively avoid a collision at this point.

[0067] To enable the generated natural language to be used in the simulation of autonomous driving test scenarios, this embodiment of the invention parses the semantic elements of the adversarial scenario to generate a scenario script executable by the simulation platform. The specific generation process is as follows: the text content is mapped into a structured scenario script, ensuring that the scenario script covers vehicle type selection, initial spatial position parameterization, motion trajectory generation strategy, and behavioral logic embedding; the generated structured configuration is converted into a domain-specific language script in the standard Scenic format, ensuring compatibility with a predetermined autonomous driving simulation platform (e.g., CARLA, but not limited to it), so that the generated natural language can be directly used for large-scale, reproducible simulation test tasks.

[0068] The meta-scene constructed in this invention demonstrates synergistic advantages in language generation, behavior modeling, and system feasibility. On one hand, the prompting mechanism integrating traffic rules and accident knowledge effectively enhances the semantic constraints and reasoning focus of the large language model during generation, ensuring that the generated content not only closely resembles real traffic contexts but also possesses clear behavioral triggering logic and high-risk induction structures. On the other hand, the generated scene introduces various potential unsafe elements, inducing response deviations in the tested system under boundary states without compromising the naturalness of the scene. Furthermore, the generated results possess clear structural expression and parameterization characteristics, enabling them to be parsed into standard scene scripts, compatible with mainstream simulation platforms, and laying the foundation for subsequent multi-agent perturbation optimization and system robustness evaluation.

[0069] After completing the initial meta-scenario construction, to further amplify its security threats and enhance its ability to characterize real traffic interactions, this invention proposes a method for constructing a multi-agent adversarial cooperative graph. This graph describes the potential intervention effects of various agents on the autonomous vehicle in the temporal and spatial dimensions of a traffic scenario. Specifically, the multi-agent adversarial cooperative graph introduces a background traffic flow with normal traffic flow into the meta-scenario and quantifies the behavioral correlation between the autonomous vehicle, adversarial agents, and various background vehicles in the global temporal dimension through a cross-timeframe attention mechanism. This process aims to accurately identify key background vehicles that, while having no direct attack intent, may exacerbate potential threats through collaborative behavior. These identified background vehicles, although their behavior superficially conforms to traffic rules, can indirectly influence the autonomous vehicle through specific trajectory perturbation strategies, such as visual obstruction, spatial compression, or path intervention. This not only strengthens the existing threats in the meta-scenario but may also induce decision-making biases or path degradation in the autonomous vehicle, thereby enhancing the challenge and risk disclosure capabilities of the test scenario for the autonomous driving system.

[0070] In one embodiment of the present invention, trajectory data of all participating vehicles in the test scenario are first extracted, covering three types of intelligent agents: (1) autonomous vehicle a ego (1) The vehicle controlled by the tested autonomous driving system; (2) Adversarial agent a adv (3) Background vehicle a i,i∈{1,N} Although it does not exhibit offensive behavior, it possesses the potential for coordinated intervention. The aforementioned trajectory data is sampled in the form of discrete time frame sequences. The behavioral trajectory of each vehicle within the simulation time window is composed of multi-dimensional attributes such as its spatial position, driving speed, heading angle, and behavioral state in each time frame, which are used to support subsequent intervention relationship modeling and key vehicle selection.

[0071] Building upon this, this invention further proposes a cross-temporal attention modeling mechanism, namely, constructing a temporal attention matrix M. att ∈R (2T)×(NT) The attention matrix consists of T rows representing the vehicle's behavior at each time frame, and T rows representing the adversarial agent's behavior; the columns represent the states of all N background vehicles at each time frame. The matrix element M... att (t,j) represents the time frame t of the autonomous vehicle or adversarial agent relative to the background vehicle. The behavioral correlation or intervention correlation in frame jmod T.

[0072] To ensure the rationality and causal consistency of the attention matrix in the temporal dimension, this embodiment of the invention introduces two temporal control mechanisms during the attention modeling process. These mechanisms are used to constrain the logical relationship between consecutive frames and to enhance the ability to focus on key moments. Specifically, the first mechanism is a temporal masking mechanism, which introduces a masking matrix when calculating attention weights to block the attention channel of the current time frame to future frames, thereby avoiding non-causal reverse information flow.

[0073] In one embodiment of the present invention, the masking matrix M mask The value (t,j) is defined as follows: when the j-th frame in the background vehicle trajectory is later than the current frame t in time, its corresponding value is set to -∞; otherwise, it is 0.

[0074]

[0075] This mechanism ensures that each query frame can only focus on the behavior state of the same background agent in the current frame and its historical frames, thus avoiding information leakage caused by the reversal of causal relationships.

[0076] The second mechanism is the time decay mechanism, used to strengthen the response weights of the large language model to recent keyframes. By introducing an exponential decay factor γ∈(0,1] in the attention calculation, weight decay is applied to behavioral responses from earlier frames. Specifically, for an element (t,j) of the matrix, its time decay weight is defined as:

[0077] M decay (t,j)=γ t-(j mod T) ,γ∈(0,1]

[0078] This ensures that historical behaviors closer to the current frame receive a higher attention response, thus allowing the large language model to focus on key interaction windows that are more temporally valuable for intervention.

[0079] Through the combined design of the two mechanisms mentioned above, this invention effectively enhances the temporal consistency and intervention sensitivity of the attention mechanism in the process of multi-agent trajectory modeling, providing a more accurate basis of related information for the selection of key agents for subsequent high-risk collaborative behaviors.

[0080] After constructing the attention matrix, a cumulative interaction risk score for each background vehicle is calculated based on this matrix to quantify its behavioral influence on the autonomous vehicle and adversarial agents across the entire time frame. Specifically, the attention response values ​​received by each background vehicle in all time frames are summed frame by frame, and the attention from both the autonomous vehicle and adversarial agents is integrated to obtain its total interaction importance score. A higher score indicates a higher probability of decision-making pressure, path disturbance risk, or safety boundary triggering for the autonomous vehicle under multi-time and multi-interaction conditions.

[0081] Subsequently, based on the risk scores, the top K background vehicles are selected as potential collaborative intervention nodes, forming a targeted intervention graph structure, denoted as the adversarial cooperative subgraph. This adversarial cooperative subgraph uses directed edges to represent the interaction direction and correlation strength between the selected background agents and key elements in the meta-scene, serving as the target object set for the subsequent trajectory perturbation optimization stage. By performing minor perturbations on these background agents in key trajectory segments, it is possible to effectively induce the autonomous vehicle to exhibit degenerate behaviors such as rule misjudgment, trajectory instability, response lag, or path conflict without introducing significant violations. This amplifies the threat impact of the meta-scene and increases the challenge intensity of the test scenario for the autonomous driving system.

[0082] By employing a multi-agent adversarial cooperative graph modeling mechanism, this invention achieves an evolution from the traditional single-agent triggering mode to a multi-agent cooperative behavior induction mechanism, effectively characterizing the causal path of risk events indirectly triggered by non-hostile background behaviors in complex traffic environments. This mechanism overcomes the technical bottlenecks of existing testing schemes in multi-agent relationship modeling and intervention strategy coupling, providing crucial structural support for the systematic generation and multi-dimensional performance evaluation of high-risk test scenarios, further enhancing the comprehensiveness and challenge of safety testing for autonomous driving systems.

[0083] After constructing the multi-agent adversarial cooperation graph and identifying key background agents with collaborative intervention potential, this invention further proposes a trajectory optimization method based on behavioral perturbation. This method applies strategic perturbations to these key agents, thereby generating more challenging test scenarios. It should be noted that the strategic perturbation process does not aim to directly cause collisions. Instead, it induces deviations in key modules such as perception, planning, or control of the tested autonomous driving system through physically reasonable and behaviorally concealed trajectory adjustments, significantly improving the risk disclosure capability of the test scenario.

[0084] Specifically, based on the importance scores in the attention matrix, several top-ranked background agents are selected as perturbation candidates. While these key vehicles are not the primary threat sources, their trajectories τ are highly coupled temporally with the vehicle or adversarial agents, possessing the ability to amplify the overall systemic risk through slight trajectory perturbations. This is then applied to the selected set K of cooperative perturbation agents and their corresponding perturbed trajectory segments. In this embodiment of the invention, the trajectory perturbation optimization process is formalized as a goal maximization problem, aiming to maximize its counteracting impact on the vehicle through strategic trajectory adjustments. The specific optimization objective is expressed as follows:

[0085]

[0086] in, To counteract the objective function, this study evaluates the effectiveness of the current perturbation trajectory in inducing vehicle behavior degradation. The optimization employs an iterative gradient update strategy. Specifically, for each perturbation segment, the gradient of its trajectory coordinates with respect to the loss function is calculated, and the trajectory parameters are updated in the direction of increased risk. Each update uses a fixed step size, and after execution, the perturbation trajectory is projected back into the physically feasible region to ensure the realism of the behavior and the rationality of traffic flow. This process continues until the convergence condition is met or the preset maximum number of iterations is reached.

[0087] To guide the direction of disturbances and quantify their system impact, a multi-objective disturbance optimization loss function is designed in this embodiment of the invention. Used to comprehensively assess the risk value of various disturbance strategies.

[0088]

[0089] The loss function contains the following key elements: (1) spatial compression term This is used to push the disturbed vehicle closer to the homing vehicle's path without triggering a collision, thereby compressing the maneuvering space and inducing the homing vehicle to adopt a non-optimal avoidance or emergency braking strategy. This is achieved by calculating the disturbance trajectory. With vehicle trajectory Minimize the Euclidean distance within the local time window; (2) Occlusion cooperation term This method optimizes the geometric occlusion relationship between disturbed vehicles and adversarial agents, creating perception blind spots or misjudgment risks from the vehicle's perspective. It is particularly suitable for simulating sudden traffic situations and perception weaknesses in intersection areas. This method utilizes a two-dimensional cross product |(·)×(·)| ⊥ The normalized vertical distance is obtained, thereby promoting the alignment of the perturbation agent with the adversarial agent; (3) Behavioral smoothing and naturalness regularization terms. By penalizing the second difference of the perturbation trajectory on the time axis It is used to limit drastic changes in key variables such as speed, acceleration, and trajectory curvature, prevent the generation of disturbing trajectories that do not conform to normal driving behavior, and ensure that the intervention behavior is semantically and formally reasonable for traffic participants.

[0090] To enhance the diversity of disturbance samples and the coverage of test scenarios, this embodiment of the invention further introduces a disturbance parameter distribution modeling mechanism. The disturbance parameters are represented using interval or probability distributions, and the sampling range is dynamically adjusted during the optimization process to generate a cluster of trajectory disturbances that are risk-oriented and exhibit behavioral differences. Finally, from the large number of generated disturbance instances, several autonomous driving test scenarios that most significantly interfere with the behavior of the autonomous driving system, exhibit the strongest perceptual misleadingness, and have the worst decision-making stability are selected as high-value safety test cases for output and evaluation.

[0091] Through the aforementioned trajectory perturbation optimization mechanism, this invention achieves a leap from static semantic scene generation to dynamic behavior control guidance, breaking through the limitations of traditional rule-based templates or single-unit perturbation strategies in terms of system coupling and scene complexity. It can construct highly challenging test scenarios that combine real traffic behavior characteristics, dynamic interaction structures, and safety risk expression capabilities, and has significant engineering value and research significance for the robustness verification, safety boundary exploration, and emergency strategy evaluation of autonomous driving systems.

[0092] After generating autonomous driving test scenarios with high adversarial nature and behavioral complexity, this embodiment of the invention deploys the constructed safety-critical traffic scenarios in an autonomous driving simulation platform and executes a closed-loop simulation process to systematically evaluate the behavioral stability and robustness of the tested autonomous driving system under multi-agent collaborative intervention conditions.

[0093] In the closed-loop simulation process, an architecture design that decouples scenario-driven and control strategies is adopted. This means the autonomous driving simulation platform uniformly controls scenario initialization and multi-agent trajectory injection, while the perception, decision-making, and control modules of the system under test operate as independent entities. This ensures the test environment has good versatility and can realistically reproduce the native response behavior of the system under test under natural input. This architecture design supports horizontal comparative analysis of various autonomous driving strategies, providing a unified test benchmark for system performance diagnosis and robustness verification.

[0094] In one embodiment of the present invention, the generated autonomous driving test scenario is imported into the autonomous driving simulation platform in the form of a structured scenario configuration file. This structured scenario configuration file includes the initial position and driving path of the autonomous vehicle, a customized sensor layout and control interface parameters, a description of the spatial arrangement and behavioral logic of the adversarial agent, and trajectory control functions and disturbance strategy parameters for all background vehicles. After loading the scenario, the autonomous driving simulation platform starts running in synchronous execution mode, comprehensively simulating the autonomous driving decision-making chain, including environmental perception, sensor data acquisition, perception fusion, path planning, and control execution, ensuring the end-to-end behavioral response of the tested system driven by real perception inputs. To ensure the objectivity of the testing process, the autonomous vehicle adopts a black-box control mode, meaning that no external intervention is made to its internal strategy logic; it only interacts with the simulation environment through sensor inputs and control outputs. This mechanism effectively isolates the differences between the test environment and the internal implementation of the tested system, improving the comparability and universality of the evaluation results.

[0095] During the simulation, the vehicle's state evolution data is recorded in real time at a high frequency, covering multi-dimensional behavioral information such as the vehicle's spatial position, speed, acceleration, control commands, trajectory deviation, and dynamic avoidance actions at each time frame. Simultaneously, the relative trajectory evolution of all background vehicles and adversarial agents in the scene is collected, and interaction event logs are identified and labeled, including structured semantic tags such as collision occurrence time and subject, overlapping trajectory paths, spatial compression conflicts, number of emergency braking and sharp steering actions, and rule violation behaviors.

[0096] For different types of autonomous driving control models (such as policy networks based on reinforcement learning, multimodal programming models that integrate visual and linguistic inputs, and rule engine systems based on state machines), this embodiment of the invention designs a unified set of robustness evaluation metrics. This set of metrics includes multiple dimensions such as response latency, avoidance performance score, path deviation degree, control command smoothness, task completion rate, and local safety coefficient. Through standardization and normalization, it adapts to different model input-output spaces, ensuring consistency and fairness across models during the evaluation process. The results of these metrics can be further used for system performance comparison analysis, behavioral failure attribution and localization, and safety boundary condition extraction, providing quantitative support for the stability optimization and policy iteration of autonomous driving models.

[0097] In one embodiment of the present invention, the robustness evaluation index set mainly includes three core categories of indicators to comprehensively measure the safety, functional completion capability, and control stability of the autonomous driving system under interference environments. Specifically, the first category is safety indicators, which assess the vehicle's survivability and rule compliance in high-risk scenarios. These include the number of collisions, the frequency of running red lights, the number of times the vehicle fails to stop before a stop sign, the average distance the vehicle travels out of its lane, and the frequency of high-risk events such as failure to actively avoid obstacles, reflecting the system's emergency response capability and safety boundary control level. The second category is task completion indicators, which measure whether the vehicle can complete a preset navigation task when subjected to multi-agent cooperative disturbances. These include the path endpoint arrival rate, the average route completion rate, task time, and path tracking stability, reflecting the system's performance in terms of functional continuity and task robustness. Finally, the third category is behavioral stability indicators, which assess the system's control consistency and driving behavior smoothness under continuous disturbances. These include the magnitude of longitudinal acceleration changes, the frequency of steering angle jitter, yaw speed fluctuations, the degree of control action redundancy, and the number of lane departures, reflecting the vehicle's movement rationality and operational naturalness under intervention scenarios.

[0098] All the above metrics are collected in real time by the autonomous driving simulation platform during the testing process, and a structured evaluation report is generated after each round of simulation. To improve the fairness and comparability of cross-model evaluation, the measurement direction of various metrics has been standardized: most metrics are judged based on the criterion that higher values ​​indicate worse system performance, while a few metrics are the opposite, forming a unified evaluation logic to ensure that it is applicable to horizontal performance comparison and risk attribution analysis of multiple autonomous driving models.

[0099] In addition, to support large-scale comparative evaluation of system robustness, this embodiment of the invention also designs a multi-stage testing mechanism to support multiple rounds of closed-loop testing of the same autonomous driving system under three conditions: the original basic scenario, the meta-scenario with the introduction of adversarial agents, and the evolutionary adversarial scenario with further superimposed cooperative perturbations. Based on a standardized index system, the performance degradation trend under different threat levels is analyzed.

[0100] Specifically, in the basic scenario, the vehicle can typically complete navigation tasks stably with a smooth trajectory and no collisions, indicating that it possesses basic environmental perception and path planning capabilities under ideal conditions. In the meta-scenario, with the addition of adversarial agents, slight trajectory deviations, rule misjudgments, or reaction delays may occur at critical decision moments. However, due to the timely intervention of some safety redundancy mechanisms, the collision probability can still be maintained within an acceptable range. In the finally constructed cooperative perturbation scenario, multiple agents act together in time and space on the vehicle's decision boundary, significantly compressing its drivable space and avoidance reaction window. This leads to a significant increase in the probability of trajectory conflicts, premature braking, path divergence, and even direct collisions when threats are not explicitly identified, exhibiting obvious behavioral instability and control collapse. By systematically quantifying the behavioral differences in these three scenarios, such as comparing the changes in key indicators like collision rate and path deviation distance, we can effectively reveal the vulnerabilities and strategic defects exposed by the tested system during the evolution from single threat to complex intervention. This multi-stage testing mechanism can not only measure the robustness of autonomous driving systems, but also characterize their dynamic adaptability and safety boundaries in scenarios with multiple overlapping risks, which has important engineering reference value and strategic optimization significance.

[0101] Based on the above embodiments, this invention further provides an autonomous driving safety-critical simulation scenario generation device, used to integrate multiple modules such as simulation scheduling, trajectory disturbance, behavior recording, and indicator evaluation under a unified platform to form a robust verification tool operating in a closed loop. Figure 3 As shown, the generation device includes one or more processors and a memory, which are coupled and coordinated through a hardware bus or on-chip interconnect structure. The memory stores one or more programs, which, when executed by the one or more processors, enable the processors to implement the autonomous driving safety-critical simulation scenario generation method described in any embodiment of the present invention.

[0102] In a specific embodiment, the processor is preferably a graphics processing unit (GPU) to meet the parallel modeling requirements of high-dimensional trajectory data, scene graph structures, and multi-agent behavioral logic; alternatively, it can be an FPGA (Field-Programmable Gate Array), ASIC (Application-Specific Integrated Circuit), DSP (Digital Signal Processor), or other processing components with high-performance computing capabilities, depending on deployment requirements. The memory is used to support the storage of instruction code and intermediate process data during the operation of the device, including but not limited to simulation scene structures, vehicle state trajectories, disturbance parameter sets, behavior record caches, and evaluation index statistical results. The memory can be implemented using any type of volatile or non-volatile storage medium, such as static random access memory (SRAM), flash memory, electrically erasable programmable read-only memory (EEPROM), disk arrays, solid-state drives, etc.

[0103] In one exemplary embodiment, the aforementioned generation device can be constructed using physical structures such as computer chips, embedded systems, or edge computing modules, or integrated into existing vehicle control terminals, traffic management platforms, simulation analysis systems, or testing and evaluation frameworks to support multi-round simulation operations and safety performance analysis of autonomous driving systems. Typical device forms include, but are not limited to: personal computers, laptops, tablet computers, smartphones, in-vehicle human-machine interaction devices, autonomous driving perception fusion modules, public security checkpoint inspection equipment, wearable analysis units, etc., or any combination of the above devices.

[0104] In another embodiment, the present invention also provides a computer-readable storage medium storing program instructions that, when executed by a processor, enable the system to implement the autonomous driving safety-critical simulation scenario generation method provided by the present invention. For example, the computer-readable storage medium can be a local disk, external storage card, USB device, cloud image container, or persistent database, used to deploy the autonomous driving test scenario generation logic provided by the present invention across devices or multiple systems.

[0105] In summary, this invention achieves system-level performance profiling of autonomous driving systems in complex dynamic traffic scenarios by constructing an integrated testing closed loop covering scenario generation, perturbation design, simulation execution, and behavior evaluation. This is achieved by combining a trajectory perturbation-based behavior intervention mechanism, a multi-agent collaborative strategy challenge framework, and a multi-dimensional robustness evaluation system covering safety, functionality, and stability. This method can not only automatically identify and construct high-risk test scenarios with explicit threat structures and potential triggering factors, but also quantify the performance degradation process of the tested system under different intervention levels, forming an attack-response-evaluation closed loop. This effectively fills the gaps in existing technologies regarding composite behavior modeling, policy vulnerability disclosure, and quantitative evaluation methods. Ultimately, this invention provides a scientific, systematic, and engineering-operable solution for the safety verification of autonomous driving systems facing real-world traffic challenges such as multi-source intervention and dynamic collaborative threats, possessing broad adaptability and significant industrial promotion value.

[0106] It should be noted that the above embodiments are merely illustrative examples. The technical solutions of the various embodiments can be combined, and all are within the protection scope of this invention.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0108] The method and apparatus for generating key safety simulation scenarios for autonomous driving provided by this invention have been described in detail above. Any obvious modifications made by those skilled in the art without departing from the essence of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A method for generating safety-critical simulation scenarios for autonomous driving based on adversarial generation and co-evolution, characterized in that... Includes the following steps: (1) Receive basic traffic scenarios described in natural language; (2) Combine traffic safety knowledge base and use large language model to generate adversarial meta-scenarios containing safety threats; (3) Parse the semantic elements of the adversarial meta-scene and generate a scene script that can be executed by the autonomous driving simulation platform; (4) Construct a multi-agent adversarial cooperation graph based on the adversarial meta-scenario, and model the spatiotemporal correlation between the autonomous vehicle, adversarial agents and background vehicles through a cross-temporal attention mechanism in order to identify key background vehicles. (5) The trajectory segments of the key background vehicles are perturbed and optimized to generate autonomous driving test scenarios.

2. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1, characterized in that... In step (2), the traffic safety knowledge base includes information on road traffic regulations, driving behavior norms and testing knowledge, and real accident and risk case data.

3. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1 or 2, characterized in that... Generating adversarial meta-scenes using large language models includes the following sub-steps: The user-input basic scenario description P base The encoding is performed as a semantic vector, and cosine similarity matching is performed between the vector and the embedded representation of each segment in the traffic safety knowledge base. Search function f R Selecting the basic scene description P base The k most relevant knowledge fragments form the context for enhanced hints. The obtained context It is concatenated into the basic input prompts as semantic guidance for the generation of the large language model, in order to generate adversarial meta-scenes containing security threats.

4. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1, characterized in that... In step (3), the executable scene script is generated through the following sub-steps: The text content is mapped to a structured scene script, which includes vehicle type selection, initial spatial position parameterization, motion trajectory generation strategy and embedded behavioral logic. The generated structured configuration is converted into a domain-specific language script in the standard Scenic format to ensure compatibility with the intended autonomous driving simulation platform.

5. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1, characterized in that... Constructing the multi-agent adversarial cooperative graph includes the following sub-steps: Extract trajectory data of different agents in autonomous driving test scenarios and construct a temporal attention matrix; A temporal masking mechanism is adopted to restrict each query frame to focus only on the current frame and historical frames of the same background vehicle, so as to avoid interference from future frame information; at the same time, a temporal decay mechanism is adopted to introduce an exponential decay factor γ∈(0,1] in the attention calculation. Calculate the risk score for each background vehicle across the entire time series; Sorting by risk score, the top K background vehicles are selected to form the multi-agent adversarial cooperation graph, which serves as the target for subsequent trajectory perturbation optimization.

6. The method for generating key simulation scenarios for autonomous driving safety as described in claim 5, characterized in that... In the attention matrix, the first T rows represent the behavior state of the autonomous vehicle in each time frame, and the last T rows represent the behavior of the adversarial agent; the columns represent the state of all background vehicles in each time frame; the matrix element values ​​reflect the correlation between the behavior of the autonomous vehicle or the adversarial agent and the background vehicles in a specific time frame.

7. The method for generating key simulation scenarios for autonomous driving safety as described in claim 5, characterized in that... The intelligent agents include the autonomous vehicle, the adversarial intelligent agent, and the background vehicle; wherein, the autonomous vehicle is the vehicle controlled by the autonomous driving system under test, the adversarial intelligent agent is the main source of threat in the meta-scenario, and the background vehicle is a vehicle that does not engage in offensive behavior but has the potential for collaborative intervention.

8. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1, characterized in that... In step (5), the perturbation optimization of the trajectory segment of the key background vehicle specifically includes: For the selected set of cooperative perturbation agents and their perturbable trajectory segments, the trajectory perturbation optimization process is formalized as an optimization problem with the goal of maximizing the degradation of the vehicle's behavior; A multi-objective perturbation optimization loss function is constructed and solved using an iterative gradient update strategy: the gradient of the trajectory coordinates of each perturbation segment with respect to the loss function is calculated, the trajectory parameters are updated with a fixed step size along the direction of risk improvement, and the perturbation trajectory is projected back into the physical feasible region after each update to ensure traffic rationality. This process is repeated until convergence or the maximum number of iterations is reached. A perturbation parameter distribution modeling mechanism is introduced, which uses interval or probability distribution to represent perturbation parameters and dynamically adjusts the sampling range to generate a trajectory perturbation population with behavioral differences. The sample that most significantly interferes with the behavior of the autonomous driving system and has the strongest perceptual misleading effect is selected from the aforementioned trajectory perturbation family and used as the output of the autonomous driving test scenario.

9. The method for generating key simulation scenarios for autonomous driving safety as described in claim 1, characterized in that... It also includes the following steps: The basic scenario, the meta-scenario, and the cooperative disturbance scenario are run sequentially in the autonomous driving simulation platform. Collect collision rate, path offset distance, and control oscillation indicators; By comparing the differences in indicators across the three scenarios, the robustness degradation trend of autonomous driving systems can be quantified.

10. A device for generating simulation scenarios critical to autonomous driving safety, characterized in that... It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, enables the processor to implement the autonomous driving safety key simulation scenario generation method according to any one of claims 1 to 9.

Images