Safety key scene enhancement generation method fusing traffic rule hard constraint and dynamic feasible region

Abstract

The invention provides a safety key scene enhancement generation method fusing traffic rule hard constraints and a dynamic feasible region, and relates to the technical field of automatic driving safety tests.The method comprises the steps that a multi-strategy initial data set is constructed, pre-training an optimal feasible value network of the automatic driving vehicle through offline reinforcement learning by utilizing the multi-strategy initial data set, and generating a dynamic feasible region; a traffic rule is represented in a formalized mode through linear sequential logic, and a rule hard constraint and feasibility guide dual-action mask mechanism is constructed; and controlling the confrontation background vehicle to generate a compliance confrontation scene by maximizing a rule-feasible region dual-guided confrontation objective function, feeding back the generated effective key scene to an initial data set, and realizing data-driven feasible region dynamic updating. According to the method, by combining rule formalization constraint and feasible region reachability analysis, the scene is ensured to be compliant and close to a system capability boundary.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving safety testing technology, specifically to a method for enhancing and generating safety-critical scenarios by integrating hard constraints of traffic rules and dynamic feasible domains. This method is applicable to the generation and testing of adversarial safety-critical scenarios for autonomous driving systems. Background Technology

[0002] Verifying the safety of autonomous driving systems in complex traffic environments faces significant challenges. The complexity of real-world traffic environments and the scarcity of adversarial scenarios make it difficult for traditional testing methods to cover extreme edge scenarios, resulting in significant deficiencies in the robustness of autonomous driving algorithms in unknown environments. Safety-critical scenario generation technology, by proactively constructing highly adversarial, low-probability, and high-hazard driving scenarios, can systematically expose the potential defects of autonomous driving systems. This approach not only fills the coverage gaps of traditional testing but also accelerates technological iteration by quantifying safety boundaries, providing crucial support for the breakthrough of autonomous driving systems' reliability from limited scenarios to all scenarios.

[0003] Regarding traffic rule constraints, the reinforcement learning algorithms used in current mainstream methods mainly rely on random exploration strategies. Their action space lacks explicit traffic rule constraints, which leads to agents easily engaging in violations such as running red lights and changing lanes illegally in order to maximize rewards. At the same time, rule constraints are often simplified into sparse reward terms rather than hard constraints, causing agents to tend to optimize the former in the trade-off between efficiency and safety, thus weakening the long-term value of rule compliance.

[0004] In terms of dynamic adaptation of feasible domain constraints, existing methods mostly drive scenario optimization with indicators such as collision rate and minimum distance, ignoring the feasible response boundary of autonomous driving system. This causes the generated scenario to deviate from the actual collision avoidance capability, resulting in a large number of invalid test cases of "unavoidable collisions". In addition, the existing feasible domain is initially generated offline, and the feasible range is limited, which seriously reduces the effectiveness and discrimination of the test. Summary of the Invention

[0005] In view of this, the present invention provides a method for enhancing and generating safety-critical scenarios by integrating hard constraints of traffic rules and dynamic feasible domains. By combining formal constraints of rules with reachability analysis of feasible domains, the method ensures that the scenarios are both compliant and close to the system capability boundaries.

[0006] Therefore, the present invention provides the following technical solution: This invention provides a method for generating safety-critical scenarios that integrates hard constraints of traffic rules and dynamic feasible regions, including: A multi-policy initial dataset is constructed, and the optimal feasible value network of autonomous vehicles is pre-trained through offline reinforcement learning using the multi-policy initial dataset to generate a dynamic feasible region. Traffic rules are formally represented using linear temporal logic, and a dual action masking mechanism of hard rule constraints and feasibility guidance is constructed. By maximizing the adversarial objective function guided by both rules and feasible domain, the adversarial background vehicle generates compliant adversarial scenarios, and the generated effective key scenarios are fed back to the initial dataset, thereby achieving data-driven dynamic updating of the feasible domain.

[0007] Furthermore, a multi-policy initial dataset is constructed, including: The autonomous driving simulation platform uses a built-in rule model to generate normal traffic data for autonomous vehicles. The real-time travel trajectory generated by the Transfuser perception-control fusion strategy; The trajectory generated by the autonomous vehicle approaching the critical collision avoidance state is induced by the adversarial background vehicle policy trained by the PPO with the near-end policy optimization, in order to expand the boundary samples of potential conflicts.

[0008] Furthermore, the optimal feasible value network for the autonomous vehicle is pre-trained via offline reinforcement learning using the aforementioned multi-policy initial dataset, including: The autonomous driving system is modeled as a Markov decision process using a constrained safety reinforcement learning framework. ,in For state space, For the action space, Let be the state transition probability. For the reward function, To constrain the violation function, It is a bounded cost function. Discount factor; A constraint violation function is constructed based on the minimum Euclidean distance between the autonomous vehicle and all background vehicles to quantify collision risk. The formula is as follows: ; in, Indicates a constraint violation function. Indicates the status of autonomous vehicles. This represents the Euclidean distance between the autonomous vehicle and the background vehicle. For a predefined safe distance threshold, N This represents the total number of background vehicles in the scene. M The constant term makes the function positive in dangerous conditions and negative in non-dangerous conditions, thus accurately characterizing the collision risk boundary; Based on Hamilton-Jacobi Reachability theory, construct an optimal state feasibility value function. With action feasible value function , respectively, represent the state of the autonomous vehicle. Or in status Execute action Then, the minimum and maximum constraint values ​​that can be maintained through the optimal strategy are expressed by the following formula: ; ; Optimal Feasibility Function Network for Autonomous Vehicles Based on Offline Dataset and Offline training; After training, the feasible region is determined by the optimal feasible value: if the optimal feasible value of the autonomous vehicle state is... If the state has an optimal strategy to avoid collisions, it is considered to be in a feasible region; otherwise, it is considered to be in an infeasible region.

[0009] Furthermore, based on offline datasets, the optimal feasible value function network for autonomous vehicles is... and Offline training included: a two-layer fully connected network with 64 neurons per layer, using the ReLU activation function and the Adam optimizer; the input was the state of an autonomous vehicle. ,action and the state of the autonomous vehicle at the next moment. The data is trained using a weighted sample loss function based on empirical replay data. A sample reweighting mechanism is introduced to address the sparsity of collision negative samples, enhancing the contribution of negative samples to the model. The formula is as follows: ; ; in, The loss function represents the state feasibility value. Loss function representing the feasibility value of an action, sample-weighted loss function , For input; This is an offline dataset; Used to balance the weights of positive and negative samples; For feasible state values Function parameters, Action Feasibility Value The parameters of the function; As a discount factor, Indicates in the dataset The expected value of the following.

[0010] Furthermore, a dual action masking mechanism is constructed, combining hard rule constraints and feasibility guidance, including: Receive background vehicle current status information Based on the policy network, a set of candidate control actions are output. By using a vehicle kinematics model to perform forward simulation of candidate actions, the predicted state after the action is executed is obtained. It includes position, attitude, and velocity information; Traffic element information is extracted from the simulation engine of the map and autonomous driving simulation platform to generate structured label data that can be used for predicate logic judgment, including: solid lane markings. Road boundaries Real-time traffic light status Intersection area markings ;in, Indicates a solid line. Indicates road boundary information, Indicates traffic lights, This indicates that the traffic light is red. Indicates the intersection area; Based on the linear time-series logic formula of the built-in traffic rules, the input action is predicted and the state is evaluated. The core rules include prohibiting crossing solid traffic lines, prohibiting driving out of the road boundary, and prohibiting running red lights. Construct an action mask based on the output of the rule logic engine. All candidate actions that do not meet traffic rules are set to 0, while legal actions are kept as 1; Illegal actions are filtered out from the candidate set based on the mask to form a subset of legal actions. This is for use in subsequent strategy optimization and execution.

[0011] Furthermore, the linear sequential logic formula includes: By calculating the vehicle bounding box and solid line in the adversarial background The geometric intersection is used to determine whether the line crosses the solid line. The system uses the road boundary returned by the API of the autonomous driving simulation platform to determine whether the center coordinates of the adversarial background vehicle are located outside the polygon. By simultaneously returning the traffic light status and vehicle prediction status Intersection area Boundary determination.

[0012] Furthermore, by maximizing the adversarial objective function guided by both rules and feasible domains, the system controls the generation of compliant and challenging key scenarios in an adversarial context, including: A rule-based, feasibility-based dual-guided adversarial strategy training framework is adopted, which optimizes the PPO structure based on proximal strategy, and the state of the adversarial background vehicle is input. The state of the target autonomous vehicle Local map features and feasible region information The policy network employs a multilayer perceptron structure. The objective function is updated using a PPO pruning structure construction strategy, and the update magnitude is controlled by the importance sampling ratio and pruning factor. ; in For importance sampling ratio, For the rule-feasibility joint advantage function, As the clipping factor, Representation strategy The expected value of the following.

[0013] Furthermore, the rule-feasibility dual-guided adversarial strategy training employs a designed combined advantage function that includes: Combining the action compliance mask output by traffic rules, a three-layer action control logic is formed: if the action is compliant and within the feasible region, positive rewards drive strategy exploration; if the action is compliant but outside the feasible region, a feasibility function penalizes and guides avoidance; if the action violates the rules, it is blocked from participating in strategy updates; and a combined advantage function... It is expressed as follows: ; in It is an indicator function, indicating that it is 1 when both the agent's current state and the next state are within the feasible region; if and only if and That is, the actions of the background vehicles are within the feasible domain. Otherwise, it is 0; The standard reward advantages are represented as follows: ; The parameter is strategy, Rewards are immediate feedback signals to the agent's actions in reinforcement learning. Indicating in strategy The state-action value function for executing action a from state s. Indicating in strategy The state value function of the next state s; The feasibility advantage is represented as:

[0014] In the formula The viewpoint transformation function represents the constraint violation values ​​before and after the autonomous vehicle's operation. The absolute state of the adversarial vehicle is converted into the relative state of the autonomous vehicle. Indicates the state of the vehicle in the opposing background at the next moment. After viewpoint transformation The relative state obtained afterward with respect to the autonomous vehicle Below, the optimal value function related to constraint violation, Indicates the current state of the vehicle in the opposing background. After viewpoint transformation The relative state obtained afterward with respect to the autonomous vehicle Below, the optimal value function related to constraint violation; The rule-feasibility dual-guided adversarial strategy training adopts the designed rule-feasibility dual-guided goal-oriented reward function as follows: ; Among them, Indicates the opposing background vehicle and the target point of confrontation The distance; To counteract collision warnings between background vehicles and autonomous vehicles; Rewards for completing the combat mission and reaching the desired area; This represents a traffic rule violation by a vehicle in a confrontational context. These are distance rewards, collision penalties, task completion rewards, and violation penalties, with adjustable weighting coefficients.

[0015] Furthermore, policy training also includes: Screening for vehicles in potential adversarial scenarios, with screening criteria including: The European distance to autonomous vehicles is less than or equal to 25 meters; Located in the area in front of or to the side of an autonomous vehicle; The current state complies with traffic rules, avoiding the involvement of adversarial vehicles that violated the rules in the initial state.

[0016] Furthermore, the generated valid key scenarios are fed back into the initial dataset to achieve data-driven dynamic updates of the feasible domain, including: After each first preset round of strategy training, trajectories that meet the following conditions are selected and fed back into the initial dataset; The reinjected data is used to fine-tune the feasible value network in a second preset round, updating the feasible domain. Repeat the steps of policy training, trajectory backfeeding, and feasible domain update until the feasible domain covers a critical state that exceeds the preset percentage of the autonomous vehicle, then stop updating.

[0017] Advantages and positive effects of the present invention: (1) In this invention, the feasible value function of autonomous vehicle (AV) is trained by initial offline data to characterize the initial feasible domain boundary. Combined with key scenario backfeeding, the data-driven dynamic evolution of the feasible domain is realized, continuously approaching the latest capability boundary of the system, and significantly accelerating the generation of key scenarios.

[0018] (2) In this invention, an action filtering mask that cannot be relaxed is constructed based on linear temporal logic (LTL), so that the adversarial background vehicle (CBV) always complies with regulations before adversarial behavior in the strategy exploration, thus completely avoiding distorted or invalid violation samples.

[0019] (3) In this invention, by designing a combined advantage function of rule-feasible domain dual guidance, a CBV adversarial strategy generation method integrating traffic rules and AV maximum feasible domain constraints is proposed, outputting compliant and critically feasible adversarial trajectories, forming a test data closed loop of "offline feasible domain → compliant adversarial generation → key scenario backfeeding → feasible domain update". Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the overall technical process for generating safety-critical scenarios that integrate hard constraints of traffic rules and dynamic feasible domains in this embodiment of the invention. Figure 2 This is a flowchart of traffic rule hard constraint determination and action filtering based on linear temporal logic (LTL) in an embodiment of the present invention; Figure 3 To visualize the comparison of the generation effects of AV trajectory and CBV adversarial trajectory at various times in the unprotected left turn scenario of this invention, in which (a) no hard constraints of traffic rules (algorithm ①) - CBV leaves the road boundary, and the scenario has no practical reference value; (b) no dynamic feasible domain constraints (traditional algorithm ②) - the distance between CBV and AV exceeds the safety threshold, and the scenario is conservative and has no testing significance; (c) the method of this invention - CBV enters the critical collision avoidance zone of AV in compliance with regulations, inducing AV to make extreme turns to avoid obstacles. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] This invention proposes a method for generating safety-critical scenarios that integrates hard constraints of traffic rules with dynamic feasible domains. First, a multi-policy initial dataset is constructed based on the Carla simulator. An optimal feasible value network (AV) is pre-trained using offline reinforcement learning to accurately characterize its feasible domain range. Second, traffic rules are formally represented using LTL, embedding semantic information such as traffic lights and road boundaries into the action space of CBVs (Carla Flow Variants), constructing a dual action masking mechanism of hard constraints and feasibility guidance. Finally, by maximizing the adversarial objective function guided by both rules and feasible domains, CBVs are controlled to generate compliant and challenging critical scenarios. The generated effective critical scenarios are then fed back into the initial dataset, achieving data-driven dynamic updates of the feasible domain and overcoming the limitation of static feasible domains on scenario coverage.

[0025] like Figure 1As shown, a CBV (Confrontational Traffic Vehicle) adversarial strategy generation method integrating hard constraints of traffic rules and dynamic feasible domains is used for generating key adversarial scenarios in AV (Autonomous Driving) safety testing. In specific implementation, the open-source Carla Simulator platform (Carla Simulator is an open-source autonomous driving simulation platform built on Unreal Engine, specifically designed for autonomous driving R&D. It provides highly customizable environments, sensor simulations, traffic scene generation, and flexible API interfaces, supporting the entire development process from algorithm testing to complex system verification) is used to build the basic simulation environment. Town05 (including complex intersections and ramps) and Town02 (including multi-lane straight roads and simple intersections) are selected as test scenarios, covering typical road types such as straight roads, intersections, and ramps. The vehicles in the scenario are divided into three categories: ①AV (Vehicle Under Test), which uses Carla's built-in rule model and Transfuser perception-control fusion model (Transfuser is an end-to-end perception-control fusion strategy for autonomous driving. Its core idea is to directly output vehicle control signals (such as steering, throttle / brake) through deep fusion of multimodal sensor data (such as cameras, LiDAR, etc.)) as the benchmark; ②CBV (Adversarial Vehicle), which is controlled by the "rule-feasible domain dual guidance" adversarial strategy trained by this invention; ③BV (Other Vehicles), which uses Carla's built-in IDM model to construct real traffic flow.

[0026] Specifically, the following steps are included: S1. Initial generation of the dynamic feasible region.

[0027] S11. Diverse Trajectory Acquisition Strategies. To improve the coverage of critical boundary states, random initialization was performed in Town05 and Town02 scenarios. Three diverse trajectory acquisition strategies were used for data acquisition, with each strategy running 1000 simulations under various initialization conditions at a sampling frequency of 10Hz. The initial datasets constructed are as follows: ① Normal AV passage data generated by the Carla built-in rule model; ② Real passage trajectories generated using the Transfuser perception-control fusion strategy; ③ Trajectories generated by the CBV strategy trained by Proximal Policy Optimization (PPO) to induce the AV to approach the critical collision avoidance state, used to expand the boundary samples of potential conflicts.

[0028] S12. Modeling the Autonomous Driving System. To construct the initial feasible region, this invention models the autonomous driving system as a Markov decision process using a constrained safety reinforcement learning framework. ,in For state space, For the action space, Let be the state transition probability. For the reward function, To constrain the violation function, It is a bounded cost function. This is the discount factor. The core objective of this framework is to optimize the strategy and maximize rewards while satisfying safety constraints.

[0029] S13. Constraint Function Calculation. A constraint violation function is constructed based on the minimum Euclidean distance between the AV and all background vehicles to quantify the collision risk. The formula is as follows: ; in This represents the Euclidean distance between AV and BV. A predefined safe distance threshold is used, where N is the total number of background vehicles in the scene, and M is a constant term. This ensures the function is positive in dangerous situations and negative in non-dangerous situations, thus accurately characterizing the collision risk boundary. In this embodiment, the predefined safe distance threshold... Through numerical simulation calibration, the constant term M is set to 18, and collision samples and critical samples are marked for subsequent sample weight adjustment during network training.

[0030] S14. Feasibility Value Network Training. To quantify the obstacle avoidance probability of the AV in future states, an optimal state feasibility value function is constructed based on Hamilton-Jacobi Reachability theory. With action feasible value function , respectively representing the state of AV Or in status Execute action Then, the minimum and maximum constraint values ​​that can be maintained through the optimal strategy are determined. The formula is as follows: ; ; Optimal Feasibility Function Network for AV Based on Offline Dataset and Offline training. Both use a two-layer fully connected network with 64 neurons per layer, ReLU activation function, and Adam optimizer (learning rate). ), weight decay Input is in AV mode. ,action and the next state To improve the stability of the value function training and address the model bias caused by the sparsity of collision negative samples, a weighted sample loss function based on empirical replay data is used for training. Furthermore, a sample reweighting mechanism is introduced to address the problem of sparse collision negative samples, enhancing the contribution of negative samples to the model. The specific formula is as follows: ; ; Where the sample weighted loss function , For input; This is an offline dataset; This is used to balance the weights of positive and negative samples; for Function parameters, for The parameters of the function; As a discount factor, in this embodiment .

[0031] After training, the feasible region is determined by the optimal feasible value: if the optimal feasible value of the AV state is... If a collision avoidance strategy exists in a given state, it is considered to belong to the feasible region; otherwise, it is considered to be in the infeasible region. In this embodiment, training is iterated for 200 rounds until the loss on the validation set stabilizes, thus obtaining a converged result. and Network parameters , The final set of the maximum feasible region output by offline training is This means that AV has an optimal obstacle avoidance strategy in the critical state, which belongs to the feasible region and is used to guide the generation of subsequent BV adversarial strategies.

[0032] S2, Implementation of hard constraint mechanism for traffic rules To ensure that CBV does not violate traffic rules when generating adversarial strategies, a formal logical rule modeling mechanism is adopted. Rules are defined as hard constraints with enforceable force, directly eliminating any violations. This mechanism differs from the soft constraints in the system; it is a hard rule constraint module that is neither slack nor learnable.

[0033] The formal modeling of rules is based on LTL (Linguistic Transformation), precisely expressing the semantic constraints of traffic rules through a combination of atomic predicates and temporal operators. Atomic predicates describe the state at a specific moment, such as "the vehicle is on the left side of the solid line"; temporal operators represent the relationship between the state and time, with the core using the global operator G, meaning "true at all future moments". If an action or state sequence violates the logical formula corresponding to the rule, the resulting state will be judged as an "illegal state," and the action will be prohibited from entering the policy optimization path. The overall process of hard constraint rules is as follows: Figure 2 As shown, the specific steps are as follows: S21. Candidate action generation and prediction. Receive current BV state information. Based on the policy network, a set of candidate control actions are output. By using a vehicle kinematics model to perform forward simulation of candidate actions, the predicted state after the action is executed is obtained. It contains dynamic information such as position, attitude, and velocity.

[0034] S22. Environmental Semantic Information Extraction. Traffic element information is extracted from maps and the Carla simulation engine to generate structured label data that can be used for predicate logic judgments, including solid lane markings. Road boundaries Real-time traffic light status Intersection area markings wait.

[0035] S23. Traffic Rule Logic Judgment. Based on the built-in LTL formulas for traffic rules, the input action prediction state is evaluated and judged. Core rules include prohibiting crossing solid traffic lines, prohibiting leaving the road boundary, and prohibiting running red lights. The specific logic is as follows: ① Do not cross the solid line: ; Among them, atomic predicates Obtain lane line type information from the Carla Maps API and mark solid lines. , Calculate the CBV bounding box and solid line The geometric intersection of the points is considered "crossed" if the area is greater than 0.

[0036] ② It is forbidden to drive outside the road boundary: ; Among them, atomic predicates Obtain road boundary information from the Carla Maps API , The ray casting method is used to determine whether the center coordinates of the CBV are outside the road boundary polygon returned by the Carla Map API; if they are outside, it is True.

[0037] ③ Running red lights is prohibited: ; Among them, atomic predicates Get the current traffic light from the Carla sensor API The light is red. Based on vehicle predicted status Intersection area The boundary judgment determines whether to enter when the light is red.

[0038] in By calculating the CBV bounding box and solid line The geometric intersection is used to determine whether the line crosses the solid line. Determine whether the CBV center coordinates are outside the polygon by using the road boundary returned by the Carla API. By simultaneously returning the traffic light status and vehicle prediction status Intersection area The boundary determination, based on the above logical matching judgment, outputs the rule evaluation label. .

[0039] In this embodiment, the current state of the CBV is received, the policy network outputs 10 candidate control actions, the predicted state is obtained by forward simulation of the vehicle kinematics model for 500ms, and the Carla API is called to extract traffic element information, such as lane line type, road boundary, traffic light status, etc., to determine compliance.

[0040] S24, Action mask generation.

[0041] Construct an action mask based on the output of the rule logic engine. All candidate actions that do not comply with traffic rules are set to 0, while legal actions are kept at 1. Illegal actions are then filtered from the candidate set using a mask to form a subset of legal actions. This information is used for subsequent strategy optimization and execution. In this embodiment, a binary action mask is constructed based on the compliance judgment result to achieve "hard removal" of non-compliant actions. Compliant actions are marked as 1, and non-compliant actions are marked as 0, thus filtering a subset of compliant actions. .

[0042] S3. Generation and optimization of CBV adversarial strategies that integrate rules and feasible domains. This invention proposes a CBV (Confrontational Traffic Violation) adversarial strategy generation method that integrates traffic rules and AV (Action Capability) maximum feasible region constraints. By introducing a dual action constraint mechanism of rules and feasible regions in the strategy optimization, it ensures that the generated scenario is both critical and compliant. The specific details are as follows: S31. Potential CBV Screening. To ensure that adversarial agents have the physical basis and traffic justification for applying policy perturbations, and to ensure the physical justification and timeliness of adversarial behavior, the following conditions must be met: ① Distance constraint: Euclidean distance from the AV ≤ 25 meters; ② Orientation constraint: Located in front of or to the side of the AV, excluding atypical threats such as rear-side collisions. In this embodiment, it is located in a fan-shaped area within ±90° of the AV's heading angle, excluding low-risk rear-side collision scenarios; ③ Compliance constraint: The current state complies with traffic rules, avoiding CBVs that violate traffic rules in the initial state from participating in the adversarial process.

[0043] S32, PPO dual-guided CBV policy training. A rule-feasibility dual-guided adversarial policy training framework based on the PPO structure is adopted, with CBV states as input. Target AV status Local map features (target point information, traffic information, etc.) and feasible region information The policy network employs a 3-layer MLP structure: Input layer → Hidden layer 1 (128 neurons, ReLU) → Hidden layer 2 (64 neurons, ReLU) → Output layer. Network training utilizes a combination of rule-feasibility-guided advantage function and goal-oriented reward. ① Conventional reward advantage. Generalized advantage estimation (GAE) is used. Depicting the basic adversarial benefits The specific calculation is as follows: .

[0044] ② Feasibility advantages. Feasibility advantages The calculation method relates AV feasibility to CBV strategy, through the viewpoint transformation function. Mapping the CBV state to the AV space is specifically represented as follows:

[0045] In the formula The constraint violation values ​​are those before and after time AV. Convert the absolute state of CBV to the relative state of AV.

[0046] ③ Fusion Rule-Feasibility Joint Advantage Function. Combining the "action compliance mask" output by traffic rule judgments, a three-layer action control logic is formed: if the action is compliant and within the feasible region, positive rewards drive strategy exploration; if the action is compliant but exceeds the feasible region, a feasibility function penalty guides avoidance; if the action violates the rules, it is blocked from participating in strategy updates. The final joint advantage function is expressed as follows:

[0047] in It is an indicator function, indicating that it is 1 when both the agent's current and next states are within the feasible region. It is true if and only if... and That is, the CBV action is within the feasible region. Otherwise, it is 0.

[0048] ④ Goal-oriented reward function. The reward function design considers collision, compliance, task completion, etc., integrating multi-dimensional goals to guide CBV to approach the critical scenario without violating regulations. The design is as follows: ; Among them, Indicates the distance between CBV and the adversarial target point; For CBV and AV collision indicators; Rewards for completing the combat mission and reaching the desired area; This represents a violation of CBV traffic rules; These are adjustable weighting coefficients. In this embodiment, the weights... (Distance Reward) (Collision penalty) (Rewards for completing the task) (Penalties for violations).

[0049] S33, Policy Update. A PPO pruning structure is used to construct the policy update objective function. The update magnitude is controlled by the importance sampling ratio and pruning factor to ensure policy iteration stability and avoid training oscillations caused by excessive policy update magnitude. ; in For importance sampling ratio, For the rule-feasibility joint advantage function, The pruning factor is updated every 10 rounds, for a total of 1000 rounds of training.

[0050] S34. Dynamic Update of Feasible Domain. To overcome the limitation of the maximum feasible domain in the initial offline computation, the generated effective adversarial scenarios are fed back into the initial dataset, and the feasible domain range is updated periodically to gradually improve scenario coverage. In specific implementation, a closed-loop data update mechanism is constructed: ① Effective adversarial trajectory selection: After every 100 rounds of policy training, trajectories that meet the following conditions are selected and fed back into the initial dataset, such as AV entering a critical state, CBV being compliant throughout, and no collisions. ② Feasibility domain fine-tuning: The feasibility value network is fine-tuned 50 times according to the process

[0008] -

[0019] based on the reinjected data, and the feasibility domain is updated. ; ③ Iterative optimization: Repeat "policy training, trajectory backfeeding, feasible domain update" until the feasible domain covers more than 95% of the critical state, then stop updating.

[0051] In the above embodiments, a closed-loop process of "simulation environment setup → initial feasible domain modeling → rule hard constraint construction → dual-guided strategy training → feasible domain dynamic iteration" is adopted to achieve the three-in-one generation of CBV adversarial strategies that integrate "compliance, criticality, and feasibility". While ensuring system security and controllability, it significantly improves the realistic rationality of adversarial examples and is suitable for intelligent driving system testing needs such as closed scenario generation and security performance adversarial evaluation.

[0052] To verify the effectiveness of this invention, the visualization effects generated by the three methods were selected in the above simulation environment under a typical scenario of unprotected left turns: ① the feasible region-guided PPO method (ruleless and non-dynamic feasible region); ② the method with traffic rule constraints (no dynamic feasible region); ③ the method proposed in this invention.

[0053] Appendix Figure 3The comparison of the generation effects of CBV adversarial scenarios in dynamic scenarios of unprotected left turns is shown: Figure (a) is the scenario generated by algorithm ①. Due to the lack of hard constraints of traffic rules, the CBV often ignores road boundaries and lane rules in pursuit of adversarial rewards, deviating from the normal road range, and the scenario loses its realistic reference value; Figure (b) is the scenario generated by algorithm ②. Without the introduction of dynamic feasible domain constraints, it is unable to perceive the obstacle avoidance boundary of the AV. The CBV and AV always maintain a distance far exceeding the safety threshold. The scenario is too conservative and has no practical significance for the performance testing of key AV scenarios; Figure (c) is the key scenario generated by the present invention. Under the premise of strictly adhering to traffic rules, the CBV accurately enters the critical collision avoidance area of ​​the AV, effectively inducing the AV to enter the extreme turning obstacle avoidance state, intuitively demonstrating the core advantages of the scenario generated by the present invention, which combines compliance and criticality.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for safety critical scenario generation that blends traffic rule hard constraints with dynamic feasible region, characterized in that, The application relates to a method for dynamically updating a feasible region of an autonomous vehicle. The method comprises the following steps: Constructing a multi-policy initial dataset and pre-training an optimal feasible value network of the autonomous vehicle by offline reinforcement learning using the multi-policy initial dataset to generate a dynamic feasible region; Formalizing traffic rules in a linear temporal logic form and constructing a dual-action mask mechanism guided by rule hard constraints and feasibility; 2. The method for generating safety critical scenarios by fusing traffic rules hard constraints and dynamic feasible region according to claim 1, wherein, By maximizing a rule-feasibility dual-guided adversarial objective function, the adversarial background vehicle is controlled to generate a compliant adversarial scene, and the generated effective key scene is fed back to the initial dataset to realize data-driven dynamic updating of the feasible region. The multi-policy initial dataset is constructed, including: Normal passing data of an autonomous vehicle generated by a rule model built in an autonomous driving simulation platform; Real passing trajectories generated by a perception-control fusion strategy of a Transfuser; 3. The method of claim 1, wherein Trajectories generated by an adversarial background vehicle strategy trained by a proximal policy optimization (PPO) to induce the autonomous vehicle to approach a critical collision avoidance state, used to expand boundary samples of potential conflicts. Modeling an autonomous driving system as a Markov decision process through a safety-augmented reinforcement learning framework with constraints wherein is a state space, is an action space, is a state transition probability, is a reward function, is a constraint violation function, is a bounded cost function, is a discount factor; The optimal feasible value network of the autonomous vehicle is pre-trained by offline reinforcement learning using the multi-policy initial dataset, including: ； wherein, represents a constraint violation function, represents a state of the autonomous vehicle, represents the Euclidean distance between the autonomous vehicle and the background vehicle, is a predefined safety distance threshold, N is the total number of background vehicles in the scene, M is a constant term, making the function positive in dangerous states and negative in non-dangerous states, to precisely depict the collision risk boundary; Based on Hamilton-Jacobi Reachability theory, construct an optimal state feasibility value function. With action feasible value function , respectively, represent the state of the autonomous vehicle. Or in status Execute action Then, the minimum and maximum constraint values ​​that can be maintained through the optimal strategy are expressed by the following formula: ； ； Optimal feasible value function network for autonomous vehicle based on offline dataset and Offline training; After the training, the feasible region is determined by the optimal feasible value: if the optimal feasible value of the state of the autonomous vehicle is determined to exist an optimal strategy to avoid the collision, and belongs to the feasible region, otherwise, it belongs to the infeasible region.

4. The method of claim 3, wherein, Offline dataset-based optimal feasible value function network for autonomous vehicles and Offline training, including: using a two-layer fully connected network, 64 neurons per layer, RELU activation function, Adam optimizer; input is the state of the autonomous vehicle , action and the next time state of the autonomous vehicle Data is trained using a weighted sample loss function based on experience replay data, and a sample reweighting mechanism is introduced to address the collision negative sample sparsity problem, enhancing the contribution of negative samples to the model, which is expressed in the formula: ； ； wherein, is a loss function representing state feasibility values, is a loss function representing action feasibility values, a sample weighted loss function , is an input; is an offline dataset; is used to balance positive and negative sample weights; is a state feasibility value is a parameter of the function, is an action feasibility value is a parameter of the function; is a discount factor, denotes the mathematical expectation over the dataset .

5. The method of claim 1, wherein A constraint violation function is constructed based on the minimum Euclidean distance between the autonomous vehicle and all background vehicles to quantify collision risks, and the formula is as follows: Receiving background vehicle current state information Outputting a set of candidate control actions based on the policy network Forward simulating the candidate actions by using a vehicle kinematics model to obtain predicted states after action execution Including position, attitude, and velocity information Extract traffic element information from a map and a simulation engine of an autonomous driving simulation platform to generate structured label data for predicate logic judgment, including: solid lane line label , road boundary , real-time traffic light state , intersection area mark ; wherein, represents a solid line, represents road boundary information, represents a traffic light, represents a red light state of the traffic light, represents an intersection area. The dual-action mask mechanism guided by rule hard constraints and feasibility is constructed, including: Building action masks from rule logic engine output All candidate actions that do not satisfy the traffic rules are set to 0, and legal actions are kept as 1. Screening illegal actions from the candidate set according to the mask to form a legal action subset for subsequent policy optimization execution.

6. The method of claim 5, wherein, The linear sequential logic formula includes: By calculating the vehicle bounding box and solid line in the adversarial background The geometric intersection is used to determine whether the line crosses the solid line. The system uses the road boundary returned by the API of the autonomous driving simulation platform to determine whether the center coordinates of the adversarial background vehicle are located outside the polygon. By simultaneously returning the traffic light status and vehicle prediction status Intersection area Boundary determination.

7. The method of claim 1, wherein According to the linear temporal logic formula of the built-in traffic rules, the input action prediction state is evaluated and judged, and the core rules include prohibition of crossing traffic lines, prohibition of driving out of the road boundary and prohibition of running a red light; Adopt a rule-feasibility dual guidance adversarial policy training framework based on a proximal policy optimization (PPO) structure, input the state of the adversarial background vehicle , the state of the target autonomous vehicle , local map features, and feasible region information ; the policy network adopts a multilayer perceptron structure; By maximizing the rule-feasibility dual-guided adversarial objective function, the adversarial background vehicle is controlled to generate a compliant and challenging key scene, including: ； where is the importance sampling ratio, is the rule-feasibility joint advantage function, is the clipping factor, denotes the mathematical expectation under the policy .

8. The method of claim 7, wherein, A strategy update objective function is constructed by adopting a PPO clipping structure, and the update amplitude is controlled by an importance sampling ratio and a clipping factor: The action compliance mask output combined with the traffic rules determines a three-layer action control logic: if the action is compliant and within the feasible region, the positive reward drives the policy exploration; if the action is compliant but beyond the feasible region, the feasibility function penalty guides the avoidance; if the action is non-compliant, it is shielded from participating in the policy update; and the advantage function is combined is represented as follows: ； where is an indicator function, which is 1 when both the current and next state of the agent are within the feasible region; and 0 otherwise. and i.e., the adversarial background vehicle action is within the feasible region, ; otherwise, 0. represents the regular reward advantage term, which is represented as: ; represents the policy with parameters . represents the reward, which is the immediate feedback signal to the agent’s action in reinforcement learning, represents the state-action value function of performing action a in state s under policy . represents the state value function of state s under policy . is the feasibility advantage term, which is represented as: wherein is the constraint violation value for the autonomous vehicle at the previous time instant, the viewpoint transformation function converts the absolute state of the adversarial background vehicle into a relative state of the autonomous vehicle; denotes the state of the adversarial background vehicle at the next time instant is transformed by the viewpoint transformation function to obtain a relative state with respect to the autonomous vehicle is the optimal value function associated with the constraint violation, at the previous time instant, the viewpoint transformation function denotes the state of the adversarial background vehicle at the current time instant is transformed by the viewpoint transformation function to obtain a relative state with respect to the autonomous vehicle is the optimal value function associated with the constraint violation. The rule-feasibility dual-guided adversarial strategy training adopts a designed rule-feasibility dual-guided combined advantage function, including: ； wherein, wherein represents the distance to the target point represents the distance to the target point represents the distance to the target point represents the distance to the target point represents the distance to the target point represents the distance to the target point represents the distance to the target point 9. The method of claim 7, wherein, The rule-feasibility dual-guided adversarial strategy training adopts a designed rule-feasibility dual-guided goal-oriented reward function, which is: Before the strategy training, the following steps are further included: Potential adversarial background vehicles are screened, and the screening conditions include: The Euclidean distance from the autonomous vehicle is less than or equal to 25 meters; Located in the front or lateral area of the autonomous vehicle; 10. The method of claim 1, wherein The current state satisfies the traffic rules, so as to avoid the initial state of the background adversarial vehicle participating in the confrontation. The generated effective key scene is fed back to the initial dataset to realize data-driven dynamic updating of the feasible region, including: After the first preset round of strategy training, the trajectories satisfying the following conditions are selected and backfilled into the initial dataset; The backfilled data is used to fine-tune the feasible value network for the second preset round, and the feasible region is updated; The steps of strategy training, trajectory backfilling and feasible region updating are repeated until the feasible region covers more than a preset percentage of critical states of the autonomous vehicle, and the updating is stopped.

Images