Automatic driving key scene generation method and system based on nested game algorithm

Abstract

The invention provides an automatic driving key scene generation method and system based on a nested game algorithm, relates to the technical field of automatic driving, and aims to solve the problems that a single game model is adopted in the prior art, strategy updating is not clear, calculation complexity is high, the convergence speed is low, and interpretability is poor. The method comprises the following steps: constructing an automatic driving antagonism test environment, respectively constructing game models of two stages for vehicle driving, determining a game player set, an action space, a revenue function and a constraint condition, and performing optimization test on the automatic driving antagonism test environment by using a nested anti-fact regret minimization algorithm. And obtaining an optimized adversarial test environment, performing a simulation test on the optimized adversarial test environment in a simulation environment, and generating a key safety scene for testing the safety of automatic driving. The problems in the prior art are solved, the representativeness and diversity of the generated scene are improved, and the efficiency and accuracy of automatic driving safety verification are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automatic driving, and particularly relates to an automatic driving key scene generation method and system based on a nested game algorithm. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] In recent years, with the rapid development of artificial intelligence technology and high-precision sensors, and the strong support of relevant policies, automatic driving technology has made significant progress. However, to ensure the safe and reliable operation of automatic driving systems in actual complex traffic environments, full testing is indispensable.

[0004] Currently, automatic driving testing mainly adopts two methods: real vehicle testing and simulation testing. Although real vehicle testing can directly verify system performance and safety, it is costly, time-consuming, and difficult to cover all dangerous scenarios; simulation testing, although it can efficiently generate scenarios, has problems such as dependence on sample data, lack of interpretability, and low testing efficiency.

[0005] In the prior art, accelerating testing is a key method to break through the bottleneck of automatic driving technology. Existing acceleration testing technologies usually optimize scenario generation through reinforcement learning or game theory methods, but still have certain deficiencies: single game model is difficult to handle high-dimensional action space, resulting in high computational complexity and slow convergence speed; lack of effective constraints on scenario rationality and diversity, which may generate test cases that do not conform to actual traffic rules; the strategy updating mechanism is not transparent, making it difficult to trace the generation logic of the test scenario. SUMMARY

[0006] To overcome the deficiencies of the prior art, the present application provides an automatic driving key scene generation method and system based on a nested game algorithm. An adversarial testing environment is constructed, and a nested game model is constructed in two stages in the adversarial testing environment. The counterfactual regret minimization algorithm is used for optimization testing to optimize the adversarial strategy and improve the probability of occurrence of safety-critical events in testing. At the same time, a driving risk field model is introduced as a criterion for scenario generation to ensure the rationality and effectiveness of the generated scenarios. While improving the efficiency of key scene generation, the representativeness and diversity of the generated scenarios are also significantly improved.

[0007] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions: The first aspect of the present application discloses an automatic driving key scene generation method based on a nested game algorithm, comprising: constructing an automatic driving adversarial testing environment; Based on the automatic driving adversarial test environment, two-stage game models are constructed for vehicle driving to determine the player set, action space, payoff function and constraint condition of the game; Based on the game model, the nested counterfactual regret minimization algorithm is used to optimize the test of the automatic driving adversarial test environment, and the optimized adversarial test environment is obtained; The optimized adversarial test environment is simulated and tested in a simulation environment to generate key safety scenarios for testing the safety of automatic driving; The nested counterfactual regret minimization algorithm is used to optimize the test of the automatic driving adversarial test environment, and the nested counterfactual regret minimization algorithm includes two coupled minimization algorithm modules, and the specific process is as follows: Initialize the selection probability of the upper and lower minimization strategies, the counterfactual regret value and the current iteration number of the upper and lower layers; Based on the selection probability of the upper minimization algorithm strategy, update the counterfactual regret value and strategy of the upper minimization algorithm, and continuously iterate to obtain the upper average sub-strategy; Based on the upper average sub-strategy, the action space is iteratively optimized, and the counterfactual regret value and strategy of the lower minimization algorithm are updated.

[0008] As a further technical solution, an automatic driving adversarial test environment is constructed, and the specific process is as follows: A tested vehicle is set as a defender, and multiple traffic participants are set as attackers; Set multiple typical road scenes such as highways, urban intersections and ramp merging as test environments; A 3*3 grid space matrix is established around the tested vehicle, and the position information of other traffic participants is obtained according to the relative position with the tested vehicle.

[0009] As a further technical solution, a two-stage game model is constructed, wherein the game model is a defender-attacker interaction model, and the two-stage specific formulas are as follows: ; ; Wherein, , is the first-stage player set, and players 1,2 represent the defender and the attacker respectively, the defender is a vehicle to be tested, and the attacker includes all vehicles in the 3*3 matrix; , is the sequence set of the first-stage player , is the sequence set of the defender, is the sequence set of all attackers; , is the first-stage player a set of payoff functions of the first stage players; , a set of constraints of the implementation strategy of the first stage players ; , is a set of second stage players, and players 1,2 represent a defender and an attacker respectively; , a set of sequences of the second stage players , a set of sequences of the defenders, a set of sequences of the attackers; , a set of payoff functions of the second stage players ; , a set of constraints of the implementation actions of the second stage players .

[0010] As a further technical solution, the payoff functions of the first stage and the second stage agents are as follows: ; ; wherein, is a first stage sub-player, is the number of first stage sub-players, is the payoff of the first stage sub-player; represents the risk payoff of the second stage attacker adopting strategy.

[0011] As a further technical solution, the risk payoff of the second stage attacker is obtained by driving risk field model, and the formula is as follows: ; wherein, and are the distances of the attacker and the defender in the vertical and horizontal directions respectively; represents the kinetic energy of the attacker ; is the attention range of the driver to the risk, which is related to the following distance of the driver; is the free flow vehicle spacing, used to represent the maximum influence range of the risk; and are the gradient adjustment coefficients in the longitudinal and transverse directions respectively.

[0012] As a further technical solution, the selection probability of the upper and lower layer minimization strategies, the counterfactual regret value and the current iteration number of the upper and lower layers are initialized, and the specific process is as follows: In the action space, the action selection probability weight on the leaf node, the number of iterations and the regret value are initialized and set, and the counterfactual value of each information set is recorded; Optimize the action space under the leaf node, and search for conflict operations that exceed the constraints according to the abnormal returns of each information set operation. If a conflict operation is found, set the corresponding selection probability to 0.

[0013] As a further technical solution, based on the selection probability of the upper layer minimization algorithm strategy, the upper layer minimization algorithm counterfactual regret value and strategy are updated and iterated to obtain the upper layer average sub-strategy, and the specific process is: Calculate the counterfactual value based on the current strategy; Calculate the counterfactual regret value according to the counterfactual value; Cumulative counterfactual regret value, update the upper layer sub-strategy; Iterate continuously until the upper layer reaches the Nash equilibrium state to obtain the average sub-strategy.

[0014] As a further technical solution, the average sub-strategy calculation formula is: ; Among them, represents the total number of iterations of the counterfactual regret minimization algorithm; is the information set, which contains the rules of the game and the historical actions taken by the players, and the behavior set that can be taken under the information set is denoted as A(I); represents the probability of reaching the information set when the player acts according to the strategy in the th iteration of the counterfactual regret minimization algorithm; is a strategy function that defines the probability of the player selecting action when in information set in the th iteration.

[0015] As a further technical solution, based on the upper layer average sub-strategy, the action space is iteratively optimized, and the lower layer minimization algorithm strategy is updated, and the specific process is: Based on the attack action of the upper layer, a random action is sampled according to the lower layer sub-strategy selection probability; Calculate the return value under the current action; Calculate the real-time regret value and cumulative regret value under the current action, and update the lower layer sub-strategy; Iterate continuously until the lower layer reaches the Nash equilibrium state, and combine the average sub-strategies of the upper and lower layers to obtain the final strategy.

[0016] The second aspect of the present application discloses an automatic driving key scene generation system based on a nested game algorithm, comprising: An intelligent adversarial test construction module is configured to construct an adversarial test environment for automatic driving. Based on the adversarial test environment for automatic driving, two-stage game models are constructed for vehicle driving, respectively, to determine a player set, an action space, a reward function and a constraint condition of the game. Based on the game model, a nested counterfactual regret minimization algorithm is used to optimize the test of the adversarial test environment for automatic driving, to obtain an optimized adversarial test environment. A key safety scene generation module is configured to simulate and test the optimized adversarial test environment in a simulation environment, to generate a key safety scene for testing the safety of automatic driving.

[0017] The above one or more technical solutions have the following beneficial effects: In the embodiment, the hierarchical optimization architecture of the nested counterfactual regret minimization algorithm (Nested-CFR) is used to decompose the high-dimensional action space into discrete scene selection (upper layer) and continuous parameter optimization (lower layer), effectively solving the slow convergence problem caused by the "dimension disaster" of the traditional method, and improving the efficiency of generating safety scenes.

[0018] In the embodiment, based on the game model of the defender-attacker and the dynamic interaction mechanism of the grid space matrix, complex scenes including rear-end collision, lane-changing conflict and cooperative oppression can be systematically generated.

[0019] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated herein by reference. They are shown by way of illustration and do not imply any limitation on the application.

[0021] Figure 1 This is a flowchart of the method for generating key scenarios for autonomous driving based on a nested game theory algorithm according to Embodiment 1 of the present invention. Figure 2 This is a diagram illustrating the accelerated testing process of Embodiment 1 of the present invention; Figure 3 This is a schematic diagram illustrating the test effect of Embodiment 1 of the present invention. Detailed Implementation

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] Example 1 This embodiment discloses a method for generating key scenarios for autonomous driving based on a nested game theory algorithm.

[0026] To more clearly illustrate this embodiment, the process of generating key autonomous driving scenarios based on nested game theory algorithms can be specifically described as follows: like Figure 1 As shown, this embodiment provides a method for generating key scenarios for autonomous driving based on a nested game theory algorithm, including: S1. Construct an adversarial testing environment for autonomous driving; Based on the autonomous driving adversarial test environment, a two-stage game model is constructed for vehicle driving to determine the player set, action space, payoff function and constraints. Based on the game theory model, a nested counterfactual regret minimization algorithm is used to optimize the adversarial test environment for autonomous driving, resulting in an optimized adversarial test environment. S2. Simulate the optimized adversarial testing environment in a simulation environment to generate key safety scenarios for testing autonomous driving safety.

[0027] like Figure 1As shown, in step S1, an autonomous driving adversarial test environment is constructed; based on the autonomous driving adversarial test environment, two-stage game models are constructed for vehicle driving, and the player set, action space, payoff function and constraints of the game are determined; based on the game model, the nested counterfactual regret minimization algorithm is used to optimize the autonomous driving adversarial test environment to obtain the optimized adversarial test environment.

[0028] S1-1. Construct an adversarial testing environment for autonomous driving.

[0029] In this embodiment, (1) one test vehicle is set as the defender and multiple traffic participants are set as attackers.

[0030] (2) Set up a variety of typical road scenarios, such as highways, urban intersections and ramp merging, as the test environment.

[0031] (3) Establish a 3×3 grid spatial matrix around the test vehicle and obtain the position information of other traffic participants based on their relative positions to the test vehicle.

[0032] Specifically, a test vehicle, referred to as the defender, and multiple traffic participants, referred to as attackers, engage in dynamic game theory in various typical road scenarios, such as highways, urban intersections, and ramp merging. The parameters are shown in Table 1. To simulate the perceptual limitations of real-world driving environments, a 3×3 grid spatial matrix is ​​designed around the test vehicle. Within this area, the positional information of other traffic participants can be obtained based on their relative position to the test vehicle, enabling real-time control and risk assessment of the vehicle.

[0033] Table 1 Parameters in the scenario

[0034] S1-2. Based on the autonomous driving adversarial test environment, construct two-stage game models for vehicle driving, and determine the player set, action space, payoff function and constraints of the game.

[0035] In this embodiment, a structured adversarial mechanism is established to generate challenging test scenarios. The model divides traffic participants into two adversarial factions: the defenders aim to ensure their own safety or minimize the cost of danger, while the attackers aim to maximize the danger to the defenders through collaborative strategies. The model transforms a multiplayer game into a broad form of two-player zero-sum game, with different types of players represented by their respective virtual agents. These agents collect information from players within the group and provide them with optimal strategies. Players have a perfect memory of their own actions and observations. In game theory models, as more players join, the size of the combinatorial action space in the game tree grows exponentially, causing the "curse of dimensionality." Furthermore, the computation time of the algorithm increases with the number of iterations. In autonomous driving testing, excessively long computation times significantly reduce testing efficiency and may lead to deviations between the results and expected values. Therefore, this problem is decomposed into two nested, linked sub-problems.

[0036] (1) Construct a two-stage game model.

[0037] Vehicle control is decomposed into two stages: the first stage handles high-level decisions, while the second stage handles more specific execution details. This significantly reduces the complexity of the decision space, improves computational efficiency, and allows for more precise control of the test vehicle. Furthermore, the two-stage decision-making approach aligns better with driving habits, making the game theory algorithm's strategy more interpretable and facilitating rule constraints and debugging, enabling the algorithm to generate reasonable driving strategies more efficiently. This method can be applied to complex driving environments; nested control enhances control flexibility and better adapts to different traffic conditions, including intersections and multiple highways. The testing process is as follows: Figure 2 As shown.

[0038] A two-stage game theory model is constructed, where the game theory model is a defender-attacker interaction model, and the specific formula is as follows: ; (1) in, , is the first phase of the player set. Players 1 and 2 represent the defender and the attacker, respectively. The defender is a vehicle to be tested, and the attacker includes all vehicles in the 3*3 matrix. , Players in Phase 1 A set of sequences, It is a set of sequences of defenders. It is a set of sequences from all attackers; , Players in Phase 1 A set of payoff functions; , Players in Phase 1 A set of constraints for the implementation strategy; This is the second phase of the player group, where players 1 and 2 represent a defender and an attacker, respectively. , Players in the second stage A set of sequences, It is a set of sequences of defenders. It is a set of sequences belonging to the attacker; , Players in the second stage A set of payoff functions; , Players in the second stage The set of constraints for the implementation of actions.

[0039] (2) Determine the player set, action space, payoff function and constraints for the first stage of the game.

[0040] In the first phase, the player set: the game model consists of two participants, a defender and an attacker, each forming their own agent. The attacker includes all attacking vehicles at this moment. Action space: a sequence refers to a series of ordered actions performed by an agent at a given node in the game tree. (Note: The last part is a typo and can be left as is.) For intelligent agents In the stage The set of actions available in the system. (Agent) The basic sequence of actions corresponds to the physical resources allocated to the task, and is composed of a set. Definition. The action space in the first stage includes: changing lanes to the left, changing lanes to the right, accelerating, braking, and holding.

[0041] To form an effective sequence, each set of basic operations within each piece of information is indexed, ensuring that each information set has the same physical resources. An information set is a group of nodes in a game tree, which players cannot distinguish due to the opponent's incomplete information. To rationally configure the environment, it is assumed that the attacker has a perfect memory of their own state and actions, and that they possess perfect information about the allocation of physical attack resources through conventional intelligence. It is also assumed that the attacker cannot determine whether the defender will engage in defensive behavior before launching a physical attack; this can be understood as the attacker lacking the ability to predict the defender's future behavior, and the attacker cannot determine whether the attack will succeed, only returning a reference value.

[0042] The total payoff in the first phase of the game is set as follows: ,in Represents the sequence pairs of attacker and defender, where , For any sequence of actions that does not constitute a complete behavioral path, its reward is defined as zero. This means that in the corresponding phase, if neither the defender nor the attacker participates in an effective game, neither side gains any benefit. Since this is a two-player zero-sum game model, the defender's payoff is opposite to the attacker's, i.e. .

[0043] Specifically, the first-stage game model agent The profit function Defined as: (2) in, It is a sub-player, For the number of child players, For the benefit of the sub-players.

[0044] The payoff of a sub-player is determined by its relative position to the defender, which is obtained through a 3×3 dimension monitoring matrix. For example, in an empirical analysis of autonomous driving accident reports in California, Hu et al. pointed out that 97% of accidents involved interaction between two vehicles, with the test vehicle and other traffic participants constituting the main cause of the accident; the remaining 3% of accidents involved single-vehicle or vehicle-pedestrian collisions. Furthermore, the study showed that rear-end collisions were the most common type of accident, accounting for 62%. According to collision statistics from 2010 to 2017, the main causes of serious accidents included: sudden lane changes (approximately 17%), speeding (12.8%), and rear-end collisions (11.2%). These findings indicate that vehicles in the same or adjacent lanes in front of the test vehicle are more likely to cause serious traffic disruption. Therefore, in the model design, different levels of payoff functions are set for each sub-player of the attacker, so that their payoff is positively correlated with the intensity of the disruption they may cause to the defender and the probability of an accident.

[0045] The constraints of the first stage, It is the set of constraints that implement the strategy, where, It is with players The relevant set of constraints. Specifically, at each time step, the positions of each sub-player within the detection matrix are mutually exclusive, ensuring that no infighting occurs within the alliance.

[0046] (3) Determine the player set, action space, payoff function and constraints for the second stage game.

[0047] In the second phase, the player set is not the same as the set of players in the first phase, but only represents the currently selected player, allowing a single attacker to participate in the game and form a true two-player game with the defender.

[0048] In each iteration, the agent receives an observation and generates an action. The action is a three-dimensional vector. In the first phase, the player must choose from a discretized action space (left lane change, right lane change, acceleration, braking, holding). In the second phase, the selected action is parameterized and adjusted. The second-phase action space includes: throttle range 0-10. For example, continuous adjustment of the braking range. This allows the player to handle more specific execution details, choose from a wider range of actions, and control the test vehicle more precisely and coherently. The use of nested game theory greatly simplifies the size of the game tree, making the information within the game tree more compact and efficient.

[0049] Designing the payoff function for the second stage of the game In this study, parameters such as the mass, relative position, real-time speed, speed threshold, and steering angle of the autonomous vehicle during operation, as well as other factors that may directly affect the vehicle's risk level, were considered. To address these multidimensional parameters, this embodiment employs a driving risk field model (reference) to measure the risk impact posed by an attacker to a defender.

[0050] Define the attacker's payoff function at this stage as follows: (3) in, This indicates that the attacker used [the following] in the second phase. Risk and return of the strategy.

[0051] At the time point Define attacker against the defender The risk constituted, i.e., the risk field model formula, is: (4) in, and attackers With defenders Distance in the vertical and horizontal directions; Indicates attacker The kinetic energy is defined as: , and attackers The mass and instantaneous velocity; The driver's range of risk awareness is related to the driver's following distance. The free-flowing vehicle spacing is used to represent the area of ​​maximum risk impact; and These are the gradient adjustment coefficients for the longitudinal and lateral directions, respectively, assuming the vehicle moves along... If traveling in that direction, then , , , .

[0052] The risk value obtained by formula (4) is the profit value of the current action, which represents the profit of the current strategy.

[0053] The amplitude of the risk field is primarily determined by the spatial distance from obstacles, exhibiting an inverse function distribution—the closer the distance, the stronger the field force. When other dynamic traffic participants are introduced into the scenario, the distribution of the risk field is not only affected by spatial distance but also significantly modulated by their dynamic behavioral factors such as speed and acceleration, thus presenting more complex spatiotemporal variation characteristics. This dynamic risk field can sensitively reflect the impact of different participants' strategy differences on system safety, thereby providing a valid basis for payoff function modeling.

[0054] The constraints for the second phase, It mainly refers to the security interaction constraints between the attacker and the defender.

[0055] Through the above steps, the autonomous driving safety verification model is decoupled into two sub-problems that are optimized in a collaborative manner, thus constructing a nested decision architecture.

[0056] S1-3. Based on the game theory model, the nested counterfactual regret minimization algorithm is used to optimize the adversarial test environment for autonomous driving, resulting in the optimized adversarial test environment.

[0057] Counterfactual regret minimization is a classic algorithm for solving large-scale, wide-form games. Its core idea is to gradually reduce the "regret value" of the agent in game decisions through iterative training, eventually approximating the Nash equilibrium strategy. It has good theoretical and practical convergence speed.

[0058] like Figure 2 As shown, a nested counterfactual regret minimization algorithm is used to implement nested control of the test vehicle. Vehicle control is decomposed into two stages: the first stage handles high-level decisions, while the second stage handles more specific execution details. This significantly reduces the complexity of the decision space, improves computational efficiency, and allows for more precise control of the test vehicle. Furthermore, the two-stage decision-making approach is more in line with driving habits, making the game theory algorithm's strategy more interpretable and facilitating rule constraints and debugging, enabling the game theory algorithm to generate reasonable driving strategies more efficiently. This method can be applied to complex driving environments; nested control improves control flexibility and can better adapt to different traffic environments, including intersections and multiple highways.

[0059] In this embodiment, the nested counterfactual regret minimization algorithm includes two coupled minimization algorithm modules at the top and bottom.

[0060] The upper-layer CFR focuses on scene type selection, enabling the vehicle to construct diverse and representative test scenarios through discrete action selection. The lower-layer CFR, based on specific scenarios, refines the dynamic parameters in the continuous action space. The upper and lower layers interact through a dynamic feedback mechanism, achieving an effective trade-off between achieving global optimal risk and the realism of the test scenario. The algorithm begins with the specific steps of applying the CFR algorithm to autonomous driving test scenarios: (1) Initialize the selection probability, counterfactual regret value and current iteration number of the minimization strategy in the upper and lower layers.

[0061] Throughout the algorithm, An information set contains the rules of the game and the players' historical actions. The set of actions that can be taken is denoted as ; This indicates the action chosen by the player; These are nodes that represent the player's action history. It is a player index. Player The cumulative arrival probability, Player The cumulative probability of the opponent arriving.

[0062] leaf node The probability weights for each action selected are initialized to 1. This means that the probability of selecting any action from the action space is equal. The number of iterations is initialized to 0, and the regret value is... It is initialized to 0, and it records each information set. The counterfactual value.

[0063] Specifically, initialize the selection probabilities of the upper and lower sub-policies. , .

[0064] Initialize the current iteration count of both the upper and lower layers. .

[0065] Initialize the cumulative counterfactual regret values ​​of the upper and lower layers. .

[0066] Optimize the optional operation space under leaf nodes.

[0067] The strategy searches for conflicting operations that exceed constraints based on the abnormal returns of operations under each information set. If a conflicting operation is found, its corresponding selection probability is set to 0, which means that the operation is ignored.

[0068] (2) Based on the selection probability of the upper-level minimization algorithm strategy, update the counterfactual regret value and strategy of the upper-level minimization algorithm, and continuously iterate to obtain the upper-level average sub-strategy.

[0069] 1) Calculate the counterfactual value based on the current strategy.

[0070] In this embodiment, a set of attack combinations is randomly sampled based on the probability selection of the upper-level sub-strategy.

[0071] First, calculate the revenue generated by this set of actions.

[0072] Counterfactual value is defined as excluding the player All other players are selected according to the strategy combination of actions (i.e., without considering the player). Reaching the node The expected return (based on the probability of) is given by the formula: (5) in, This indicates that other players, based on strategy, have a history of h. The probability of taking an action. Indicates when player When the destination node is reached The game payoff obtained is equal to the total payoff of all sub-players within the agent reaching the finish line node. The sum of each party's gains at that time.

[0073] Second, calculate the counterfactual value.

[0074] Each information set The counterfactual value is calculated using the following formula: (6) Players The counterfactual value of action a is given by the formula: (7) 3) Calculate the counterfactual regret value based on the counterfactual value.

[0075] Players Actions performed on information set F The counterfactual regret value is calculated using the following formula: (8) in, This indicates that the counterfactual reward at each step is based on the current strategy. Under the decision, players In information set and select action The reward received at that time.

[0076] If the player i The generated strategy satisfies when hour, If the outcome is positive, then the strategy is considered to minimize regret. Wherein, This indicates that all regret values ​​are non-negative.

[0077] 4) Accumulate counterfactual regret values ​​and update the upper-level sub-strategy.

[0078] The strategy is updated based on the level of regret for selected actions in the game's historical data, thereby adjusting the strategy for future action selection.

[0079] The policy probability is updated using regret values. The regret matching algorithm randomly selects actions based on an action probability distribution proportional to positive regret values. The regret value calculated for each action 'a' in each iteration is stored in the set of available actions. In the next iteration In this context, the formula for calculating the update strategy is: (9) The regret matching algorithm employs a virtual self-play method, minimizing the regret value through iterative training. The formula for updating the current policy in the next iteration is as follows: ; 5) Iterate continuously until the upper layer reaches the Nash equilibrium state and obtains the average sub-policy.

[0080] Update the average strategy portfolio, after After the nth iteration, the average policy is defined as: (10) in, This represents the total number of iterations in the counterfactual regret minimization algorithm; An information set contains the rules of the game and the players' historical actions. Let A(I) be the set of actions that can be taken. This indicates that in the counterfactual regret minimization algorithm, the... In the next iteration, the player According to strategy During the action, upon arrival at the information set The probability of; It's a strategy function that defines the player. In the In the next iteration, when in the information set When selecting an action The probability of.

[0081] Repeat steps 2) through 5) until the policy converges. The upper layer reaches Nash equilibrium, forms an average sub-policy, and inputs it into the lower layer.

[0082] Through multiple iterations of the upper-level CFR, an approximate Nash equilibrium strategy is given, from which a set of typical scenarios representing high risk are generated.

[0083] (3) Based on the upper-level average sub-policy, iteratively optimize the action space and update the counterfactual regret value and policy of the lower-level minimization algorithm.

[0084] 1) Calculate the reward value under the current action.

[0085] Based on the attack actions of the upper layer, an action is randomly sampled according to the probability selection of the lower-layer sub-policy. Calculate the benefit value caused by this action. .

[0086] 2) Calculate the real-time regret value and cumulative regret value under the current action, and update the next-level sub-policy.

[0087] Calculate the real-time regret value for this action. Cumulative regret value Update the lower-level sub-policy .

[0088] 3) Iterate continuously until the lower layer reaches Nash equilibrium, then merge the average sub-policies of the upper and lower layers to obtain the final policy.

[0089] Increase the number of iterations in the lower layer The lower level reaches Nash equilibrium, and the average sub-strategies are merged. Average sub-strategy To form the final strategy .

[0090] The lower-level CFR iteratively optimizes the continuous parameter space within a given scenario framework, continuously adjusting the attacker's strategy to maximize the interference effect on the tested system, and finally converges to the optimal combination of test strategies.

[0091] The above steps not only improve the efficiency and accuracy of autonomous driving testing, but also significantly enhance the interpretability and scalability of testing strategies, providing theoretical and algorithmic support for building a systematic safety verification process.

[0092] like Figure 1 As shown, in step S2, the intelligent adversarial test environment is simulated and tested in the simulation environment, and key security scenarios are generated.

[0093] An intelligent adversarial test environment was constructed based on the Carla simulation platform. Carla's built-in Traffic Manager autonomous driving algorithm was used to control both the background vehicle and the test vehicle, evaluating the safety of the autonomous driving algorithm in the simulated environment. When the target vehicle enters the preset test area, control is switched from the autonomous driving algorithm to a virtual agent. Through continuous iteration of the nested CFR algorithm, key scenarios are generated. To ensure the rationality of vehicle behavior, driver style is extracted from natural data and used in conjunction with the strategy generated by the virtual agent to control the selected vehicle to perform adversarial actions. The experimental variables and parameter settings are shown in Table 2. Here, [A, B, K, L, R] represent discrete actions "acceleration," "braking," "maintaining," "left turn," and "right turn," respectively, and [0.0~1.0] represent continuous actions "throttle parameter."

[0094] Table 2 Variable and Parameter Settings

[0095] like Figure 3 This example illustrates a typical high-risk testing scenario where two dangerous situations occurred consecutively during the test: In the first phase, the test vehicle was driving normally when a vehicle on the left changed lanes into the middle lane, prompting the test vehicle to slow down and avoid it. Simultaneously, a vehicle on the right approached, creating a spatial blockade and preventing the test vehicle from changing lanes to the right to avoid the collision. In the second phase, the target vehicle in front suddenly slowed down and cut into the left lane, while vehicles behind rapidly approached, creating a rear-end collision. Ultimately, due to the path closure caused by the coordinated pressure from multiple vehicles, the test vehicle failed to brake effectively in time, resulting in a collision.

[0096] In this embodiment, the game theory-based autonomous driving scenario generation method constructs an intelligent adversarial testing environment and utilizes a counterfactual regret minimization algorithm. Through a cumulative regret mechanism, it updates the policy probability in real time, gradually increasing the selection weight of the safety critical state to systematically generate complex "corner scenarios." The invention designs a nested counterfactual regret minimization algorithm, decomposing the testing process into two stages, effectively reducing the computational complexity brought about by the high-dimensional action space, thereby accelerating the algorithm's convergence speed and significantly improving testing efficiency. The introduction of a continuous action space setting allows for more detailed and flexible control of the test vehicle, enabling more accurate simulation of the vehicle's dynamic behavior in real traffic environments.

[0097] Example 2 This embodiment discloses an autonomous driving key scenario generation system based on a nested game theory algorithm, including: The intelligent adversarial testing module is used to build an adversarial testing environment for autonomous driving. Based on the autonomous driving adversarial test environment, a two-stage game model is constructed for vehicle driving to determine the player set, action space, payoff function and constraints. Based on a game theory model, a nested counterfactual regret minimization algorithm is used to optimize the adversarial testing environment for autonomous driving, resulting in an optimized adversarial testing environment. The nested counterfactual regret minimization algorithm comprises two coupled minimization algorithm modules: initializing the selection probabilities, counterfactual regret values, and current iteration counts of the minimization strategies in both layers; updating the counterfactual regret value and strategy of the upper-layer minimization algorithm based on the selection probability of the upper-layer minimization algorithm strategy, and iterating continuously to obtain the upper-layer average sub-strategy; and iteratively optimizing the action space based on the upper-layer average sub-strategy, updating the counterfactual regret value and strategy of the lower-layer minimization algorithm. The critical safety scenario generation module is used to simulate and test the optimized adversarial test environment in a simulation environment, generating critical safety scenarios for testing autonomous driving safety.

[0098] The method steps in Implementation Example 1 are implemented by providing a system for generating key scenarios for autonomous driving based on a nested game theory algorithm. Example 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0099] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps described above.

[0100] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0101] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for generating key scenarios for autonomous driving based on nested game theory algorithms, characterized in that, include: Construct an adversarial testing environment for autonomous driving; Based on the autonomous driving adversarial test environment, a two-stage game model is constructed for vehicle driving to determine the player set, action space, payoff function and constraints. Based on the game theory model, a nested counterfactual regret minimization algorithm is used to optimize the adversarial test environment for autonomous driving, resulting in an optimized adversarial test environment. The optimized adversarial testing environment is simulated and tested in a simulation environment to generate key safety scenarios for testing autonomous driving safety. Among them, a nested counterfactual regret minimization algorithm is used to optimize the autonomous driving adversarial test environment. The nested counterfactual regret minimization algorithm consists of two coupled minimization algorithm modules, and the specific process is as follows: Initialize the selection probability, counterfactual regret value, and current iteration count of the minimization strategy in both the upper and lower layers; Based on the selection probability of the upper-level minimization algorithm policy, update the counterfactual regret value and policy of the upper-level minimization algorithm, and continuously iterate to obtain the upper-level average sub-policy; Based on the upper-level average sub-policy, the action space is iteratively optimized, and the counterfactual regret value and policy of the lower-level minimization algorithm are updated.

2. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 1, characterized in that, The specific process for constructing an adversarial testing environment for autonomous driving is as follows: One test vehicle is designated as the defender, and multiple traffic participants are designated as attackers; The test environment was set up with various typical road scenarios, including highways, urban intersections, and ramp merging. A 3×3 grid spatial matrix is ​​established around the test vehicle, and the position information of other traffic participants is obtained based on their relative positions to the test vehicle.

3. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 1, characterized in that, A two-stage game theory model is constructed, which is a defender-attacker interaction model. The specific formulas for the two stages are as follows: ； ； in, , is the first phase of the player set. Players 1 and 2 represent the defender and the attacker, respectively. The defender is a vehicle to be tested, and the attacker includes all vehicles in the 3*3 matrix. , Players in the first stage A set of sequences, It is a set of sequences of defenders. It is a set of sequences from all attackers; , Players in Phase 1 A set of payoff functions; , Players in Phase 1 A set of constraints for the implementation strategy; This is the second phase of the player group, where players 1 and 2 represent a defender and an attacker, respectively. , Players in the second stage A set of sequences, It is a set of sequences of defenders. It is a set of sequences from the attacker; , Players in the second stage A set of payoff functions; , Players in the second stage The set of constraints for the implementation of actions.

4. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 3, characterized in that, The profit functions for the agents in the first and second stages are as follows: ； ； in, They are the first-stage sub-players. The number of sub-players in the first phase. For the benefits of the sub-players in the first phase; This indicates that the second-stage attacker used Risk and return of the strategy.

5. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 4, characterized in that, The risk-reward ratio for the second-stage attacker is obtained using a driving risk field model, as follows: ； in, and attackers With defenders Distance in the vertical and horizontal directions; Indicates attacker kinetic energy; The driver's range of risk awareness is related to the driver's following distance. The free-flowing vehicle spacing is used to represent the area of ​​maximum risk impact; and These are the gradient adjustment coefficients for the longitudinal and lateral directions, respectively.

6. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 1, characterized in that, The initialization process for the selection probabilities, counterfactual regret values, and current iteration counts of the minimization strategies in both the upper and lower layers is as follows: In the action space, the action selection probability weights, iteration counts, and regret values ​​at the leaf nodes are initialized, and the counterfactual values ​​of each information set are recorded. Optimize the action space under the leaf nodes, and search for conflicting operations that exceed the constraints based on the abnormal returns of operations under each information set. If a conflicting operation is found, set its corresponding selection probability to 0.

7. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 1, characterized in that, Based on the selection probability of the upper-level minimization algorithm's policy, the counterfactual regret value and policy of the upper-level minimization algorithm are updated, and this process is iterated continuously to obtain the upper-level average sub-policy, specifically as follows: Calculate the counterfactual value based on the current strategy; Calculate the counterfactual regret value based on the counterfactual value; Accumulate counterfactual regret values ​​and update the upper-level sub-strategy; The process continues iteratively until the upper layer reaches a Nash equilibrium state, resulting in the average sub-policy.

8. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 7, characterized in that, The formula for calculating the average sub-strategy is: ； in, This represents the total number of iterations in the counterfactual regret minimization algorithm; An information set contains the rules of the game and the players' historical actions. Let A(I) be the set of actions that can be taken. This indicates that in the counterfactual regret minimization algorithm, the... In the next iteration, the player According to strategy During the action, upon arrival at the information set The probability of; It's a strategy function that defines the player. In the In the next iteration, when in the information set When selecting an action The probability of.

9. The method for generating key autonomous driving scenarios based on nested game theory algorithms as described in claim 1, characterized in that, Based on the upper-level average sub-policy, the action space is iteratively optimized, and the lower-level minimization algorithm policy is updated. The specific process is as follows: Based on the attack actions of the upper layer, a probability is randomly sampled according to the probability of the lower layer sub-policy; Calculate the reward value for the current action; Calculate the real-time regret value and cumulative regret value under the current action, and update the next-level sub-policy; The process continues iterating until the lower layer reaches a Nash equilibrium. The average sub-policies of the upper and lower layers are then merged to obtain the final policy.

10. A system for generating key scenarios for autonomous driving based on a nested game theory algorithm, characterized in that, include: The intelligent adversarial testing module is used to build an adversarial testing environment for autonomous driving. Based on the autonomous driving adversarial test environment, a two-stage game model is constructed for vehicle driving to determine the player set, action space, payoff function and constraints. Based on a game theory model, a nested counterfactual regret minimization algorithm is used to optimize the adversarial testing environment for autonomous driving, resulting in an optimized adversarial testing environment. The nested counterfactual regret minimization algorithm comprises two coupled minimization algorithm modules: initializing the selection probabilities, counterfactual regret values, and current iteration counts of the minimization strategies in both layers; updating the counterfactual regret value and strategy of the upper-layer minimization algorithm based on the selection probability of the upper-layer minimization algorithm strategy, and iterating continuously to obtain the upper-layer average sub-strategy; and iteratively optimizing the action space based on the upper-layer average sub-strategy, updating the counterfactual regret value and strategy of the lower-layer minimization algorithm. The critical safety scenario generation module is used to simulate and test the optimized adversarial test environment in a simulation environment, generating critical safety scenarios for testing autonomous driving safety.

Images