Natural and confrontation type ai collaborative autonomous driving test scene generation method
By introducing NPC vehicles that combine natural and adversarial AI in autonomous driving testing, and utilizing multi-agent collaboration and dynamic adjustment strategies, the problems of single NPC behavior patterns and insufficient collaboration in existing technologies are solved, enabling the generation of more realistic and efficient test scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH OF CHINA
- Filing Date
- 2025-12-17
- Publication Date
- 2026-07-31
AI Technical Summary
In existing autonomous driving testing technologies, NPC behavior patterns are singular, making it difficult to simulate the complex environment of multiple driving styles coexisting in real traffic. The lack of coordination mechanisms leads to insufficient diversity of test scenarios and low testing efficiency.
In a simulation environment, NPC vehicles controlled by natural and adversarial AI are deployed. Through multi-agent collaboration, they are coordinated and coordinated, and the behavior strategies of the NPCs are dynamically adjusted to construct complex interactive scenarios. The NPC models are trained using imitation learning and deep reinforcement learning to achieve information sharing and spatiotemporal strategy coordination.
It significantly improves the diversity and realism of test scenarios, can proactively construct complex multi-vehicle chain risk scenarios, improves the targeting and efficiency of testing, and dynamically adjusts NPC behavior to target and uncover system defects.
Smart Images
Figure CN121956936B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of autonomous driving testing technology, specifically involving a method for generating autonomous driving test scenarios through the collaboration of natural and adversarial AI. Background Technology
[0002] With the continuous development of autonomous driving technology, how to conduct efficient and comprehensive testing and verification has become a key challenge for the industry. Simulation testing has become the mainstream solution due to its high efficiency and safety. Its core lies in generating realistic and challenging test scenarios in a virtual environment, especially simulating the complex behaviors of other traffic participants (non-player character vehicles, NPCs).
[0003] Early simulation tests commonly employed rule-driven or predefined script-based NPC control methods. These methods exhibit fixed and predictable behavior patterns, making it difficult to simulate the uncertainties and interactivity of human driving. This resulted in insufficient diversity of test scenarios and an inability to effectively expose the boundary defects of autonomous driving systems.
[0004] To improve testing effectiveness, existing technologies are beginning to incorporate artificial intelligence models to control NPC behavior. For example, imitation learning is used to generate natural NPCs that conform to human driving habits, while reinforcement learning is used to train adversarial NPCs that aim to discover weaknesses in the main vehicle system.
[0005] However, existing AI-based solutions have significant drawbacks: First, most methods use only NPCs with a single behavioral pattern (either all natural or all adversarial), which cannot simulate the complex environment of multiple driving styles coexisting in real traffic. Second, even when two types of NPCs are deployed in combination, they lack a collaborative mechanism, their behaviors are independent, and they cannot proactively construct high-risk complex scenarios involving multiple agents, such as "using the obstruction of the vehicle in front to make an incursion." Finally, existing solutions typically lack the ability to dynamically adjust based on the real-time performance of the main vehicle, resulting in a static and rigid testing process that makes it difficult to specifically uncover deep-seated system defects. Summary of the Invention
[0006] This application proposes a method for generating autonomous driving test scenarios through collaboration between natural and adversarial AI. It aims to address the problems in existing technologies, such as insufficient scenario realism due to the single behavior patterns of NPCs, difficulty in constructing complex interactive scenarios due to the lack of collaboration between intelligent agents, and low testing efficiency and relevance caused by the static and rigid testing process.
[0007] This application provides a method for generating autonomous driving test scenarios through collaboration between natural and adversarial AI, including:
[0008] The test vehicle and multiple non-player character vehicles are deployed in a simulation environment. The multiple non-player character vehicles include natural non-player character vehicles and adversarial non-player character vehicles. The natural non-player character vehicles are controlled by a first artificial intelligence model to simulate normal driving behavior, and the adversarial non-player character vehicles are controlled by a second artificial intelligence model to induce the main vehicle to take risks.
[0009] During the simulation test, based on the environmental state information of the main vehicle and each non-player character vehicle, a multi-agent cooperative approach is adopted to coordinate and coordinate the driving behavior of the natural type non-player character vehicle and the adversarial type non-player character vehicle to jointly construct the test scenario. The environmental state information includes the real-time status of the main vehicle and each non-player character vehicle, road topology, traffic rules and historical interaction information.
[0010] The system collects the planning and control information of the main vehicle, and dynamically adjusts the behavior strategy of the non-player character vehicle based on the environmental state information and the collected planning and control information of the main vehicle, so as to improve the relevance and effectiveness of the test scenario.
[0011] In some embodiments of this application, the first artificial intelligence model is obtained through imitation learning to mimic human driving behavior;
[0012] The second artificial intelligence model is trained through deep reinforcement learning to perform driving behaviors that actively create high-risk events while complying with traffic rules.
[0013] In some embodiments of this application, the imitation learning used to train the first artificial intelligence model uses vehicle trajectory information recorded in the NGSIM dataset as training data, so that the natural non-player character vehicle generates a smooth and reasonable driving trajectory.
[0014] In some embodiments of this application, the multi-agent cooperation method is implemented through a multi-agent reinforcement learning framework;
[0015] The framework treats the natural non-player character vehicles and the adversarial non-player character vehicles as independent intelligent agents. These agents share environmental state information, including road topology, traffic rules, the status of the main vehicle and all parties, and conduct collaborative training and decision-making.
[0016] In some embodiments of this application, the reward function set for the adversarial non-player character vehicle in the multi-agent reinforcement learning framework includes the following reward items: a reward item for incentivizing its behavior to cause the main vehicle to violate regulations, a reward item for incentivizing it to approach the main vehicle, and a reward item for penalizing it for causing a collision.
[0017] In some embodiments of this application, the method of coordinating and coordinating the driving behaviors of the natural-type non-player character vehicle and the adversarial non-player character vehicle using a multi-agent cooperative approach includes:
[0018] Control a natural-type non-player character vehicle to drive normally in front of the main vehicle to create a visual obstruction, while simultaneously controlling an adversarial-type non-player character vehicle to use the visual obstruction to cut into the lane in front of the main vehicle and brake suddenly.
[0019] In some embodiments of this application, the dynamic adjustment includes:
[0020] Based on the environmental state information, a trajectory prediction and collision detection method based on a kinematic model is used to predict whether a collision event will occur in the future, and a feedback instruction is generated to promote the generation of the collision event.
[0021] In some embodiments of this application, the method of using trajectory prediction and collision detection based on kinematic models to predict whether a collision event will occur in the future includes:
[0022] The main vehicle and the non-player character vehicle are modeled as rectangular bounding boxes defined by their center position, heading angle, length, and width.
[0023] Based on the current speed and acceleration of each vehicle, its trajectory in the near future is calculated using kinematic formulas.
[0024] The collision event is predicted by determining whether the bounding boxes of the main vehicle and any of the non-player character vehicles based on the motion trajectory spatially overlap.
[0025] In some embodiments of this application, when a collision event is predicted to occur, the strategy for dynamically adjusting the behavior of the non-player character vehicle includes:
[0026] The adversarial non-player character vehicle is controlled to accelerate in order to shorten its longitudinal distance from the main vehicle. In some embodiments of this application, the behavioral strategy of the adversarial non-player character vehicle is constrained by preset liability determination rules;
[0027] The liability determination rules define the reasonableness of a non-player character vehicle's behavior by setting a minimum safe distance that the vehicle must maintain when changing lanes in front of the main vehicle, thus ensuring that the behavior is recognized as a valid competitive test rather than an invalid extreme driving behavior.
[0028] In summary, the autonomous driving test scenario generation method based on the collaboration of natural and adversarial AI provided in the embodiments of this application, by integrating natural NPCs driven by imitation learning and adversarial NPCs driven by deep reinforcement learning, reproduces the real environment in which normal traffic flow and high-risk events coexist in a simulation environment, thereby breaking through the limitations of a single behavior mode and significantly improving the diversity and realism of the test scenarios. By introducing a collaborative decision-making mechanism based on multi-agent reinforcement learning (MARL), information sharing and spatiotemporal strategy coordination (such as using occlusion to enter) between natural and adversarial NPCs are realized, so that the system can actively construct complex multi-vehicle chain risk scenarios, solving the problem of isolated and uncooperative agent behavior. Furthermore, by constructing a real-time feedback closed loop, the NPC strategy is dynamically adjusted according to the master vehicle's planning and control information and the environmental state, forming an adaptive testing process of "generation-response-regeneration". Therefore, it can target and discover system boundary defects, greatly improving the efficiency and relevance of the test. Attached Figure Description
[0029] The features and advantages of this application will be more clearly understood by referring to the accompanying drawings, which are illustrative and should not be construed as limiting the application in any way. In the drawings:
[0030] Figure 1 This is a schematic diagram of a computer system to which this application applies;
[0031] Figure 2 This is a flowchart illustrating a method for generating autonomous driving test scenarios through collaboration between natural and adversarial AI, according to some embodiments of this application;
[0032] Figure 3 These are schematic diagrams of collaborative test scenarios generated by natural-type NPCs and adversarial-type NPCs in some embodiments of this application;
[0033] Figure 4 These are schematic diagrams of multi-agent reinforcement learning (MARL) frameworks in some embodiments of this application;
[0034] Figure 5 This is a schematic diagram of the closed-loop dynamic adjustment of NPC behavior strategy based on real-time feedback in some embodiments of this application. Detailed Implementation
[0035] In the following detailed description, numerous specific details of this application are illustrated by example to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that this application can be practiced without these details. It should be understood that the terms “system,” “apparatus,” “unit,” and / or “module” used in this application are one way of distinguishing different parts, elements, sections, or components at different levels in a sequential arrangement. However, these terms may be replaced with other expressions if other expressions can achieve the same purpose.
[0036] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly connected to or coupled to or communicate with other devices, units, or modules, or there may be intermediate devices, units, or modules present, unless the context explicitly indicates otherwise. For example, the term "and / or" as used herein includes any one and all combinations of one or more of the relevant listed items.
[0037] The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate that explicitly identified features, integrals, steps, operations, elements, and / or components are included, and such expressions do not constitute an exclusive list, and other features, integrals, steps, operations, elements, and / or components may also be included.
[0038] Referring to the following description and accompanying drawings, these and other features and characteristics, operating methods, functions of related structural elements, combinations of parts, and economics of manufacture of this application can be better understood, wherein the description and drawings form part of the specification. However, it is clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. It is understood that the drawings are not drawn to scale.
[0039] Various structural diagrams are used in this application to illustrate various variations of the embodiments according to this application. It should be understood that the preceding or following structures are not intended to limit this application. The scope of protection of this application is determined by the claims.
[0040] In recent years, autonomous driving technology has developed rapidly and has become an important direction for the integration of artificial intelligence and intelligent transportation. To ensure the functional safety and decision-making robustness of autonomous driving systems, efficient and comprehensive testing and verification methods have become a focus of industry attention. Traditional road testing suffers from high costs, low efficiency, and high risks, making simulation testing the mainstream approach.
[0041] Current mainstream autonomous driving simulation testing systems (such as CARLA, LGSVL, and SUMO) generally use rule-driven or trajectory-reproducible non-player character (NPC) control strategies, such as lane keeping, traffic light compliance, and fixed-path following. While these methods can ensure the correctness of traffic logic, they cannot fully simulate the complex, dynamic, and even irrational human behavior patterns in real-world roads. This results in insufficient diversity of test scenarios and makes it difficult to effectively uncover the boundary defects of autonomous driving systems.
[0042] To enhance the challenge and coverage of simulation testing, some research has begun to introduce agent control models based on reinforcement learning (RL) or imitation learning (IL) to generate adversarial scenarios. However, current research mostly focuses on the control strategies of single-type NPCs, such as all NPCs adopting adversarial RL strategies or uniformly imitating human behavior, lacking systematic research on the collaborative control of multiple types of behavioral models. This limits the behavioral diversity and realistic complexity of the generated scenarios, making it difficult to reflect the potential risks caused by various driving style interactions in real-world roads.
[0043] In view of this, this application proposes a method for generating multi-agent cooperative autonomous driving test scenarios based on natural and adversarial AI control, which aims to control NPCs in the scenario and construct more realistic and complex traffic scenarios through cooperative simulation.
[0044] This application generates complex and diverse test scenarios by deploying two types of AI-controlled NPC vehicles in a simulated traffic environment:
[0045] Natural-type NPCs: Obey traffic rules, exhibit no aggressive behavior, and maintain normal traffic conduct.
[0046] Adversarial NPCs: Through reinforcement learning or generating adversarial strategies, they actively create high-risk or extreme events that affect the driving behavior of the main vehicle.
[0047] Compared to existing single-test scenarios, this application:
[0048] 1. It innovatively combines natural and adversarial intelligent agents to simulate real driving behavior and risk-inducing behavior respectively, thereby enhancing the diversity and complexity of traffic participant behavior in the test scenario.
[0049] 2. Through policy scheduling and environmental feedback mechanisms, collaborative cooperation among different types of intelligent agents can be achieved. Multiple NPCs can construct traffic scenarios with combined attack characteristics in space and time. For example, one NPC creates an obstruction, while another NPC cuts in line, thereby more effectively exposing the potential defects of the autonomous vehicle system.
[0050] 3. An AI model applied to vehicles can dynamically adjust the behavior patterns of the NPC (NPC agent) based on the response of the main vehicle system. This method drives the evolution of the NPC agent's strategy through scenario feedback, forming a closed-loop test cycle of "generation—reaction—regeneration." This adaptive mechanism significantly improves the targeting and efficiency of testing.
[0051] Figure 1 This is a schematic diagram of a computer system to which this application applies. Figure 1 The computer system shown includes a test database, a simulation test platform, and a collaborative decision-making server that are interconnected. Among them:
[0052] The collaborative decision server generates collaborative behavior strategies for virtual traffic participants by scheduling natural NPC behavior models driven by imitation learning and adversarial NPC behavior models driven by deep reinforcement learning. Based on real-time environmental and vehicle status information obtained from the simulation test platform, it uses a multi-agent collaborative algorithm to calculate collaborative actions between NPCs (such as following, cutting in, and occlusion coordination), thereby dynamically constructing complex interactive scenarios in a given simulation environment. At the same time, by analyzing the planning and control response of the master vehicle, it adjusts the NPC strategies in real time, driving the test scenario to evolve towards higher risk or more targeted directions, ultimately forming a closed-loop adaptive test process.
[0053] As a controlled virtual environment executor, the simulation testing platform is responsible for constructing and running test scenarios in a virtual environment. This includes configuring basic environmental parameters such as roads, weather, and traffic density according to preset parameters, receiving instructions from the collaborative decision server, and driving natural and adversarial NPC vehicles to perform collaborative behaviors. Simultaneously, it collects the status information of the master vehicle and each NPC in real time and feeds it back to the collaborative decision server to support dynamic strategy adjustments, and completely records the test process data to the test database to ensure that the scenario can be accurately replayed and reproduced.
[0054] The test database is used to store data throughout the entire system operation process, supporting scenario reproduction and backtracking analysis.
[0055] At the deployment level, the collaborative decision server and simulation test platform can be deployed on a single server, server cluster, or distributed cloud environment in terms of hardware architecture; in terms of software integration, they can be embedded as functional modules into existing simulation systems (such as CARLA, LGSVL), or work together with simulation systems as independent services. Figure 1 The vehicles shown are all virtual entities in the simulation environment.
[0056] Figure 2This is a flowchart illustrating a method for generating autonomous driving test scenarios using a combination of natural and adversarial AI, according to some embodiments of this application. In some embodiments, the method for generating autonomous driving test scenarios using a combination of natural and adversarial AI is... Figure 1 The collaborative decision-making server shown executes the method, which includes the following steps:
[0057] S210, Deploy the test vehicle and multiple non-player character vehicles in a simulation environment. The multiple non-player character vehicles include natural non-player character vehicles and adversarial non-player character vehicles. The natural non-player character vehicles are controlled by a first artificial intelligence model to simulate normal driving behavior, and the adversarial non-player character vehicles are controlled by a second artificial intelligence model to induce the main vehicle to take risks.
[0058] First, the simulation test environment is deployed. This deployment process is completed within the simulation test platform and includes:
[0059] 1. Scene Environment Configuration: Import a predefined road map file (such as an OpenDrive format file) and set environmental parameters such as weather, lighting, and traffic density. In some embodiments of this application, the above configuration is achieved by calling the application programming interface (API) of the Carla simulation platform.
[0060] 2. Deployment of Traffic Participants: In the road environment, the initial state (e.g., position, speed) of the tested ego vehicle is defined. Simultaneously, multiple non-player character vehicles (NPCs) are deployed, and their behavior pattern types are assigned: natural NPCs or adversarial NPCs. In some embodiments of this application, the shape, initial position, density, and behavior pattern type of the NPC vehicles are defined using the Carla simulator API.
[0061] NPC behavior patterns include natural NPCs and adversarial NPCs.
[0062] Natural NPCs need to ensure that the generated driving trajectories are smooth and reasonable. Specifically, they need to ensure cautious and stable driving while adhering to traffic rules. The natural model is trained using imitation learning, constructing a control model that can drive the vehicle to perform natural driving behaviors through processing and training on a large amount of traffic data. The dataset can be the NGSIM dataset, which records vehicle data on highways, from which vehicle trajectory information is extracted and used for training. Adversarial NPCs, on the other hand, need to exhibit more aggressive driving behaviors while adhering to traffic rules, such as suddenly changing lanes in front of the main vehicle or suddenly braking in front of the main vehicle. The model is trained using deep reinforcement learning, and its reward function is set to guide the NPC to actively create driving scenarios unfavorable to the main vehicle while adhering to physical characteristics and traffic rules. This reward function consists of a weighted sum of multiple reward components, specifically in the following form: r adv =w1 R ego_violation +w2 R ego_distance_reduce w3 R self_collision in:
[0063] R ego_violation It is a reward component that encourages the behavior of the driver, resulting in the driver's violation of regulations;
[0064] R ego_distance_reduce It is a reward component that encourages them to approach the main vehicle to enhance the competitive aspect;
[0065] R self_collision It is a reward component that punishes itself for causing a collision or serious violation;
[0066] , , These are weighting coefficients set to balance the various reward components.
[0067] In scenario testing, liability determination is a crucial part. In interactions between the primary vehicle and the NPC (Non-NPC), if an accident occurs due to extreme driving behavior by the NPC vehicle, then testing the primary vehicle's autonomous driving system is meaningless. Therefore, this involves the issue of liability allocation. The training of the NPC model also needs to be constrained by liability determination rules; for example, how far in front of the primary vehicle should a lane change be considered adversarial driving rather than extreme driving behavior? The reward function is set with r... self_collisionThis is to avoid collisions or violations caused by problems with the NPCs themselves. S220, during the simulation test, based on the environmental state information of the main vehicle and each non-player character vehicle, a multi-agent cooperative approach is adopted to coordinate and coordinate the driving behavior of the natural type non-player character vehicle and the adversarial type non-player character vehicle to jointly construct the test scenario. The environmental state information includes the real-time status of the main vehicle and each non-player character vehicle, road topology, traffic rules, and historical interaction information.
[0068] This application requires real-time coordination of the behaviors of natural-type NPCs and adversarial NPCs within a simulated environment, enabling them to jointly construct a traffic scenario that is both realistic and challenging. This realistic and challenging scenario includes, but is not limited to, the following embodiments:
[0069] Example 1: A natural NPC is driving normally in front of the main vehicle. An adversarial NPC takes advantage of the visual obstruction created by the natural NPC to cut in front of the main vehicle and brake suddenly, creating a sudden braking risk scenario.
[0070] Example 2: Two adversarial NPCs, using a cooperative strategy, occupy positions close to the front and rear of the main vehicle, respectively, forming a pincer movement to test the main vehicle's lane-changing decision-making ability in a confined space.
[0071] Example 3: A natural-type NPC performs a lane change in front of the main vehicle, suddenly revealing an emergency-stopped vehicle that was previously obscured in front of it, in order to test the main vehicle's ability to respond to sudden unseen dangers.
[0072] Figure 3 This is a schematic diagram of the collaborative testing scenario provided in the embodiments of this application. Figure 3 The left image corresponds to the emergency braking scenario shown in Embodiment 1; Figure 3 The middle image corresponds to the vehicle pincer attack scenario shown in Embodiment 2; Figure 3 The right figure corresponds to the exposed emergency stop vehicle scenario shown in Example 3.
[0073] This application implements the cooperative scheduling and coordination of natural-type NPCs and adversarial NPCs through a multi-agent reinforcement learning (MARL) framework. In some embodiments of this application, the implementation of the multi-agent reinforcement learning (MARL) framework is as follows:
[0074] The Carla simulator is viewed as a multi-agent environment, where each type of NPC is treated as an agent with its own observation, action space, and reward function.
[0075] State space: includes road topology, traffic rules, master vehicle position and speed, state of each NPC (position, speed, intent), historical interaction information, etc.
[0076] Action space: Each NPC's acceleration, turning angle, lane change actions, etc.
[0077] Reward function: This function balances traffic realism indicators (vehicle spacing distribution, flow stability) and challenge indicators (frequency of triggering driver avoidance and emergency braking) through weighting. The specific reward function is as follows:
[0078] R total =w1 R Truth +w2 R Challenge
[0079] in:
[0080] R Truth It is a reward component based on traffic authenticity indicators, which may include related metrics such as similarity to human driving behavior, vehicle distance safety, and traffic rule compliance.
[0081] R Challenge It is a reward component based on traffic challenge indicators, which may include the frequency and intensity of behaviors such as successfully triggering the main vehicle's emergency braking and forcing the main vehicle to change lanes unplanned;
[0082] w 1, w 2 is a weighting coefficient set to balance the various rewards.
[0083] Figure 4 This is a schematic diagram of a multi-agent reinforcement learning (MARL) framework in some embodiments of this application.
[0084] Different types of NPCs have different roles:
[0085] Natural NPCs prioritize following traffic rules and simulate real human driving behavior as much as possible. Their strategies are influenced by the behavior of adversarial NPCs, which naturally leads to subsequent risk events or decision-making dilemmas.
[0086] Adversarial NPCs execute interference strategies within safe constraints, increasing the driving challenge for the main vehicle. The effect can be amplified through interaction with natural NPCs.
[0087] During training, a global training approach is adopted, where all NPCs share global information and train collaboratively. However, during execution, decisions are made based solely on local perception and necessary communication.
[0088] Different collaboration modes can be set depending on the test scenario.
[0089] 1. One-way trigger mode:
[0090] Adversarial NPCs proactively create events, while natural NPCs passively respond according to traffic flow rules. This mode features clear logic and is suitable for verifying the primary vehicle's responsiveness to typical single events.
[0091] 2. Two-way interactive mode:
[0092] Natural and adversarial NPCs influence each other strategically, making events more complex. This mode realizes the interaction between multiple vehicles, more closely resembling real traffic flow, and can test the stability of the master vehicle's decision-making and control in complex interaction scenarios.
[0093] 3. Distributed Challenge Mode:
[0094] Multiple adversarial NPCs are distributed across different road sections, creating continuous challenges through time and location coordination. This mode primarily simulates complex traffic games, exposing the shortcomings of autonomous driving algorithms under high-intensity interaction.
[0095] S230: Collect the planning and control information of the main vehicle, and dynamically adjust the behavior strategy of the non-player character vehicle based on the environmental state information and the collected planning and control information of the main vehicle, so as to improve the relevance and effectiveness of the test scenario.
[0096] During the test scenario operation, the collaborative decision server continuously acquires the real-time status data of the main vehicle and each NPC through the data interface provided by the simulation test platform, and converts it into feedback signals, thereby realizing the real-time adjustment of NPC behavior and the adaptive optimization of the test scenario, forming a closed loop of "perception-decision-adjustment". Figure 5 This is a schematic diagram of the closed-loop dynamic adjustment of NPC behavior strategy based on real-time feedback provided in some embodiments of this application.
[0097] like Figure 5 As shown, the dynamic adjustment includes:
[0098] 1. Data Acquisition
[0099] The collaborative decision-making server obtains real-time status data of the master vehicle and each NPC vehicle, including position, speed, acceleration, and heading angle, by calling the application programming interface (API) of the simulation test platform. Simultaneously, it acquires environmental data such as road topology and traffic signals.
[0100] 2. Event Prediction and Implementation
[0101] The collaborative decision-making server predicts whether a critical event will occur in the near future based on pre-defined test objectives (such as triggering a collision or close-range intrusion). Specifically, a kinematic model can be used for trajectory prediction: the vehicle is modeled as a rectangular bounding box defined by its center position, heading angle, length, and width. The future trajectory is calculated based on its current speed and acceleration, and collision prediction is achieved by determining whether the predicted bounding boxes of the main vehicle and the NPC overlap.
[0102] 3. Feedback signal generation
[0103] Based on the event prediction results, the collaborative decision-making server generates structured strategy adjustment instructions through back-engineering to advance the generation of key scenarios. For example, it generates high-level instructions such as "increase the speed of adversarial NPCs," "shorten the longitudinal distance between a specific NPC and the main vehicle," or "increase the lateral deviation probability of NPCs." It should be noted that these instructions do not directly control the vehicle's actuators (such as the accelerator and steering wheel), but rather serve as high-level strategy guidelines, inputting into the NPC's AI control model, which then autonomously decides and generates specific driving behaviors.
[0104] 4. Strategy Execution and Closed Loop
[0105] The collaborative decision server sends feedback signals to each non-player character vehicle (NPC) through a low-latency communication interface to drive them to update their behavior strategies, thereby achieving dynamic adjustment and adaptive evolution of the test scenario.
[0106] To ensure the reproducibility and analyzability of the test scenarios, some embodiments of this application further include a data recording and playback module.
[0107] The data recording and playback module is used to continuously record dynamic data of the main vehicle, NPC and environment in autonomous driving test scenarios, and supports subsequent playback to realize scenario reproduction, testing and other functions.
[0108] The data collected includes:
[0109] Main vehicle information: position, speed, acceleration, steering angle.
[0110] NPC information: vehicle status, control commands, decisions, and behavior tags (adversarial or natural).
[0111] Environmental information: road topology, traffic light signals, and weather conditions.
[0112] The data is stored in JSON file format. During subsequent scenario playback, the contents of the JSON file can be read to pinpoint the time and specific details of the problem that occurred during the test.
[0113] In some embodiments of this application, the data may be persistently stored in a test database.
[0114] One embodiment of this application constructs a single interference scenario in the CARLA simulation platform based on the collaborative construction of natural and adversarial NPCs. Specifically:
[0115] In the CARLA simulation platform, a two-way four-lane highway is set up with medium traffic density and sunny weather. The main vehicle cruises at a speed of 80 km / h in the right lane.
[0116] The natural type NPC (vehicle A) is controlled by an imitation learning model and maintains normal car-following behavior.
[0117] The adversarial NPC (vehicle B) is controlled by a deep reinforcement learning model, whose reward function includes terms that approach the host vehicle and induce it to brake.
[0118] During the test, vehicle B made a sudden cut in front of vehicle A, forcing the main vehicle to decelerate abruptly. This scenario verifies the main vehicle's braking and stability control capabilities under sudden circumstances.
[0119] Another embodiment of this application describes the creation of complex scenarios based on multi-agent collaborative manufacturing within the CARLA simulation platform. Specifically:
[0120] In urban scenarios (including intersections and traffic lights), traffic flow is high-density.
[0121] The two natural-type NPCs obeyed the traffic light rules and maintained normal driving in front of the main vehicle.
[0122] One of the opposing NPCs used the traffic light to suddenly change lanes in front of and to the right of the main vehicle, while another opposing NPC approached at high speed from behind.
[0123] Two opposing NPCs coordinate their actions to put the main vehicle in a dilemma of being "blocked in front and pressured behind," thus testing the main vehicle's lane-changing and evasion strategies.
[0124] Another embodiment of this application constructs a real-time feedback-driven dynamic adjustment scenario in the CARLA simulation platform. Specifically:
[0125] On suburban roads, the main vehicle travels along a planned route. The system uses a real-time feedback module to detect the vehicle's acceleration and predict its trajectory.
[0126] If the main vehicle exhibits an overly conservative strategy (such as significantly slowing down in advance), the intensity of the adversarial NPC behavior will automatically decrease to maintain the rationality of the scene.
[0127] If the main vehicle exhibits aggressive driving (such as attempting to overtake quickly), the strategy of the opposing NPC will automatically adjust, increasing the level of interference.
[0128] This scenario demonstrates the adaptive capability of this application, namely, that the NPC strategy is dynamically adjusted in real time according to the master vehicle, thereby exploring the system boundary under different test objectives.
[0129] In summary, the autonomous driving test scenario generation method based on the collaboration of natural and adversarial AI provided in the embodiments of this application, by integrating natural NPCs driven by imitation learning and adversarial NPCs driven by deep reinforcement learning, reproduces the real environment where normal traffic flow and high-risk events coexist in a simulation environment, thereby breaking through the limitations of single behavior patterns and significantly improving the diversity and realism of test scenarios. By introducing a collaborative decision-making mechanism based on multi-agent reinforcement learning (MARL), information sharing and spatiotemporal strategy coordination (such as using occlusion to enter) between natural and adversarial NPCs are realized, so that the system can actively construct complex multi-vehicle chain risk scenarios, solving the problem of isolated and uncooperative agent behavior. Furthermore, by constructing a real-time feedback closed loop, the NPC strategy is dynamically adjusted according to the master vehicle's planning and control information and the environmental state, forming an adaptive testing process of "generation-response-regeneration". Therefore, it can target and discover system boundary defects, greatly improving the efficiency and relevance of testing. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described devices and modules can be referred to the corresponding descriptions in the foregoing device embodiments, and will not be repeated here.
[0130] Although the subject matter described herein is provided in the general context of execution on a computer system in conjunction with an operating system and applications, those skilled in the art will recognize that other implementations can also be executed in conjunction with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will understand that the subject matter described herein can be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframes, etc., and can also be used in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may reside on both local and remote memory storage devices.
[0131] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0132] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this application and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this application should be included within the protection scope of this application. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.
Claims
1. A natural type and confrontation type AI cooperative automatic driving test scene generation method, characterized in that, include: The test vehicle and multiple non-player character vehicles are deployed in a simulation environment. The multiple non-player character vehicles include natural non-player character vehicles and adversarial non-player character vehicles. The natural non-player character vehicles are controlled by a first artificial intelligence model to simulate normal driving behavior, and the adversarial non-player character vehicles are controlled by a second artificial intelligence model to induce the main vehicle to take risks. During the simulation test, based on the environmental state information of the main vehicle and each non-player character vehicle, a multi-agent cooperative approach is adopted to coordinate and coordinate the driving behavior of the natural type non-player character vehicle and the adversarial type non-player character vehicle to jointly construct the test scenario. The environmental state information includes the real-time status of the main vehicle and each non-player character vehicle, road topology, traffic rules and historical interaction information. The multi-agent collaboration method is implemented through a multi-agent reinforcement learning framework. The framework treats the natural non-player character vehicle and the adversarial non-player character vehicle as independent agents. The agents share environmental state information, including road topology, traffic rules, the status of the main vehicle and the status of each party, and perform collaborative training and decision-making. In the multi-agent reinforcement learning framework, the reward function set for the adversarial non-player character vehicle includes the following reward items: a reward item for incentivizing its behavior to cause the main vehicle to violate the rules, a reward item for incentivizing it to approach the main vehicle, and a reward item for penalizing it for causing a collision. The method of coordinating and coordinating the driving behaviors of the natural-type non-player character vehicles and the adversarial-type non-player character vehicles using a multi-agent cooperative approach includes: Control a natural non-player character vehicle to drive normally in front of the main vehicle to create a visual obstruction, while controlling an adversarial non-player character vehicle to use the visual obstruction to cut into the lane in front of the main vehicle and brake suddenly. The system collects the planning and control information of the main vehicle, and dynamically adjusts the behavior strategy of the non-player character vehicle based on the environmental state information and the collected planning and control information of the main vehicle, so as to improve the relevance and effectiveness of the test scenario. The dynamic adjustment includes: based on the environmental state information, using a trajectory prediction and collision detection method based on a kinematic model to predict whether a collision event will occur in the future, and generating feedback instructions to promote the generation of the collision event; The method of using a kinematic model-based trajectory prediction and collision detection to predict whether a collision event will occur in the future includes: modeling the main vehicle and the non-player character vehicles as rectangular bounding boxes defined by their center position, heading angle, length, and width; calculating their future short-term motion trajectories based on the current speed and acceleration of each vehicle using kinematic formulas; and determining whether the predicted rectangular bounding boxes of the main vehicle and any of the non-player character vehicles based on the motion trajectories spatially overlap to complete the prediction of the collision event. When a collision event is predicted, the strategy for dynamically adjusting the behavior of the non-player character vehicle includes: controlling the acceleration of the adversarial non-player character vehicle to shorten its longitudinal distance from the main vehicle; The behavioral strategy of the non-player character vehicle in the competitive mode is constrained by a preset liability determination rule. The liability determination rule defines the rationality of the behavior by setting a minimum safe distance that the non-player character vehicle in the competitive mode must meet when changing lanes in front of the main vehicle, so as to ensure that the behavior is recognized as a valid competitive test rather than an invalid extreme driving behavior.
2. The method according to claim 1, characterized in that: The first artificial intelligence model was trained through imitation learning to mimic human driving behavior; The second artificial intelligence model is trained through deep reinforcement learning to perform driving behaviors that actively create high-risk events while complying with traffic rules.
3. The method according to claim 2, characterized in that: The imitation learning used to train the first artificial intelligence model uses vehicle trajectory information recorded in the NGSIM dataset as training data, so that the natural non-player character vehicle can generate a smooth and reasonable driving trajectory.