Automatic driving confrontation test scene generation method and system based on strategy switching

Abstract

The invention belongs to the technical field of automatic driving test, and provides an automatic driving confrontation test scene generation method and system based on strategy switching, and the method comprises the steps: obtaining an automatic driving simulation scene; setting positions and initial speeds of a target vehicle and an intelligent body vehicle in the acquired simulation scene; in combination with the set position and initial speed of the target vehicle, the running state of the target vehicle is optimized by adopting a multi-agent near-end strategy, and the target vehicle is controlled; determining a current traffic state according to the control driving state of the target vehicle; in combination with the dynamic quantitative index of the artificial risk potential field, sensing the traffic flow risk of the intelligent body vehicle in the determined current traffic state in real time; and according to the obtained traffic flow risk level, automatically switching a defense strategy and an attack strategy of the agent vehicle, and driving the multi-agent vehicle to cooperatively generate an automatic driving confrontation test scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automatic driving test, and particularly relates to an automatic driving adversarial test scene generation method and system based on strategy switching. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] Automatic driving technology has broad prospects in improving road traffic safety, relieving traffic congestion, and realizing intelligent travel. In order to ensure the safety and robustness of automatic driving in complex environments, virtual simulation-based testing and verification methods have become the core direction of research and industry; among them, adversarial testing methods expose potential defects of the tested system by constructing high-risk driving situations, which have been widely used in automatic driving test systems.

[0004] However, the existing automatic driving adversarial testing methods have the following shortcomings in scene generation: (1) Traditional testing relies on preset rules or artificially constructed scenes, which is difficult to cover extreme situations that may occur in real traffic, resulting in a lack of test samples and insufficient scene coverage; (2) Although the existing scene generation methods based on reinforcement learning have certain adaptive ability, the strategy learning efficiency is low, the reward signal is unstable, and the generated behavior lacks traffic rationality, making it difficult to effectively balance "natural driving" and "adversariality".

[0005] Although the rule-driven model can reproduce the natural driving trajectory of real vehicles to some extent, it cannot simulate challenging and goal-oriented adversarial behaviors, which leads to a lack of complexity and decision-making pressure in the test scene. Therefore, the existing methods have not fully integrated the generation mechanism of natural driving strategies and adversarial behaviors, and lack an efficient scene generation framework that can adaptively switch behavior strategies according to environmental risks in a multi-agent dynamic game environment. SUMMARY

[0006] To solve the above problems, the present application proposes an automatic driving adversarial test scene generation method and system based on strategy switching, which adopts a multi-agent reinforcement learning strategy (FAP-MAPPO) with a multi-strategy adaptive switching mechanism that integrates reinforcement learning and rule modeling mechanisms. It overcomes the problems of lack of naturalness and single adversarial vehicle strategy in existing automatic driving test scene generation methods, improves the test coverage, risk exposure ability, and evaluation representativeness of automatic driving, and efficiently generates safety-critical test scenes with natural rationality and risk resistance in a simulation environment.

[0007] According to some embodiments, the first aspect of the present application provides a method for generating an automatic driving confrontation test scene based on policy switching, which adopts the following technical solution: A method for generating an automatic driving confrontation test scene based on policy switching, comprising: Obtaining an automatic driving simulation scene; Setting the positions and initial speeds of a target vehicle and an agent vehicle in the obtained simulation scene; Combining the set positions and initial speeds of the target vehicle, using multi-agent proximal policy optimization to control the running state of the target vehicle; Determining the current traffic state according to the controlled running state of the target vehicle; Combining the dynamic quantitative indicators of the artificial risk potential field, and real-time sensing the traffic flow risk of the agent vehicle in the determined current traffic state; According to the obtained traffic flow risk level, automatically switching the defense strategy and attack strategy of the agent vehicle, and driving the multi-agent vehicle to cooperatively generate an automatic driving confrontation test scene.

[0008] As a further technical limitation, in the process of automatically switching the defense strategy and attack strategy of the agent vehicle, a density-based hierarchical triggering mechanism is adopted, and when several agents simultaneously meet the policy switching condition, a three-level priority judgment mechanism is adopted.

[0009] Further, the three-level priority judgment mechanism includes a risk-dominance principle mechanism for preferentially switching the agent with the highest directional risk value, a motion state constraint mechanism for agents in deceleration state not participating in policy switching, and a time-space sequence mechanism for determining the switching priority based on the time sequence of entering the policy switching region when the risk values are equal.

[0010] As a further technical limitation, the artificial risk potential field includes at least a composite potential field of a trigger-based gravitational potential field based on the policy switching region and an agent vehicle repulsive potential field; the reward function is designed with the aid of the artificial risk potential field, and the reward function includes a collision target vehicle reward function and a potential energy change reward function of the artificial potential field.

[0011] As a further technical limitation, the automatic switching of the defense strategy and attack strategy of the agent vehicle is performed in the local policy switching region based on situational awareness, rectangular regions are set before and after the center line of the target vehicle, the set rectangular regions are used as policy switching regions, when other agent vehicles enter the policy switching region, the local state of the agent vehicle is re-evaluated, and whether the switching between the defense strategy and attack strategy of the agent vehicle is needed is determined according to the speed of the agent vehicle and the relative distance between the agent vehicle and the target vehicle.

[0012] As a further technical limitation, in the process of driving the multi-agent vehicle to cooperatively generate the adversarial test scene of automatic driving, the running state and the loss function of the agent vehicle are obtained from the experience pool according to the strategy type, the obtained loss function is back propagated to update the network parameters, the strategy decision of the agent vehicle is obtained based on the local observation, the traffic flow around the target vehicle is disturbed according to the obtained strategy decision of the agent vehicle, the traffic interaction between the target vehicle and the agent vehicle is constructed, the test and verification of the adversarial test target are realized, and the generation of the adversarial test scene of automatic driving is completed.

[0013] According to some embodiments, the second aspect of the present application provides an automatic driving adversarial test scene generation system based on strategy switching, which adopts the following technical solution: An automatic driving adversarial test scene generation system based on strategy switching, comprising: An acquisition module configured to acquire an automatic driving simulation scene; A setting module configured to set the positions and initial speeds of a target vehicle and an agent vehicle in the acquired simulation scene; An optimization module configured to adopt multi-agent proximal policy optimization to control the running state of the target vehicle in combination with the set positions and initial speeds of the target vehicle; A determination module configured to determine a current traffic state according to the controlled running state of the target vehicle; A perception module configured to perceive the traffic flow risk of the agent vehicle in the determined current traffic state in real time in combination with the dynamic quantitative indicators of the artificial risk potential field; A generation module configured to automatically switch the defense strategy and the attack strategy of the agent vehicle according to the obtained traffic flow risk level, and drive the multi-agent vehicle to cooperatively generate the adversarial test scene of automatic driving.

[0014] According to some embodiments, the third aspect of the present application provides a computer readable storage medium, which adopts the following technical solution: A computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps in the automatic driving adversarial test scene generation method based on strategy switching according to the first aspect of the present application.

[0015] According to some embodiments, the fourth aspect of the present application provides an electronic device, which adopts the following technical solution: An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor implements the steps in the automatic driving adversarial test scene generation method based on strategy switching according to the first aspect of the present application when executing the program.

[0016] According to some embodiments, a fifth aspect of the present application provides a computer program product, adopting the following technical solutions: A computer program product comprising software code, the program in the software code performing the steps in the method for generating an automatic driving adversarial test scene based on policy switching according to the first aspect of the present application.

[0017] Compared with the prior art, the present application has the following beneficial effects: The present application adopts multi-agent reinforcement learning with a multi-policy adaptive switching mechanism, fusing reinforcement learning and rule modeling mechanism, fusing multi-agent proximal policy optimization algorithm (Flexible Adversarial Policy-switching MAPPO) and an environment perception model based on driving safety field (Driving Safety Field, DSF), so that the agent can dynamically switch behavior strategies according to the current traffic state; by introducing a dynamic quantitative index based on an artificial risk field, the traffic flow risk of the region where the agent is located is perceived in real time, and the dynamic quantitative index based on the artificial risk field is used as a criterion for policy switching, so as to realize the switching and scheduling of adversarial actions and natural driving behavior patterns; a dynamic density multi-type background vehicle modeling mechanism is introduced to enhance the naturalness, complexity and coverage of the test scene; the problems of insufficient naturalness and single adversarial vehicle strategy of existing automatic driving test scene generation methods are overcome, and the test coverage, risk exposure ability and evaluation representativeness of automatic driving are improved, and a safe and critical test scene with natural rationality and risk resistance is efficiently generated in a simulation environment. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which form a part of this embodiment, are used to provide a further understanding of the embodiment, and the schematic embodiments of the embodiment and their descriptions are used to explain the embodiment, and do not constitute an improper limitation on the embodiment.

[0019] Figure 1 A flowchart of the method for generating an automatic driving adversarial test scene based on policy switching in the first embodiment of the present application; Figure 2 A detailed step schematic diagram of the method for generating an automatic driving adversarial test scene based on policy switching in the first embodiment of the present application; Figure 3 A policy switching region schematic diagram in the first embodiment of the present application; Figure 4 A risk perception region schematic diagram in the first embodiment of the present application; Figure 5 A scene generation interaction flowchart in the first embodiment of the present application; Figure 6This is a schematic diagram illustrating the scenario generation test adversarial effect in Embodiment 1 of the present invention; Figure 7 This is a structural block diagram of the autonomous driving adversarial test scenario generation system based on strategy switching in Embodiment 2 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0023] In this invention, terms such as "upper," "lower," "left," "right," "front," "back," "vertical," "horizontal," "side," and "bottom" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only to facilitate the description of the structural relationships of the various components or elements of this invention and do not specifically refer to any component or element in this invention. They should not be construed as limiting the invention.

[0024] In this invention, terms such as "fixed connection," "connected," and "linked" should be interpreted broadly, indicating a fixed connection, an integral connection, or a detachable connection; a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can determine the specific meaning of these terms in this invention based on the specific circumstances, and they should not be construed as limitations on the invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1 Embodiment 1 of this invention introduces a method for generating autonomous driving adversarial test scenarios based on strategy switching.

[0027] like Figure 1 The method for generating autonomous driving adversarial test scenarios based on policy switching, as shown, includes: Obtain autonomous driving simulation scenarios; Set the position and initial speed of the target vehicle and the agent vehicle in the acquired simulation scenario; In combination with the set position and initial speed of the target vehicle, the running state of the target vehicle is optimized by using multi-agent proximal policy optimization, and the target vehicle is controlled; According to the control driving state of the target vehicle, the current traffic state is determined; In combination with the dynamic quantitative index of the artificial risk potential field, the traffic flow risk of the agent vehicle in the determined current traffic state is perceived in real time; According to the obtained traffic flow risk level, the defense strategy and the attack strategy of the agent vehicle are automatically switched, and the multi-agent vehicle is driven to cooperatively generate an automatic driving confrontation test scene.

[0028] The embodiment fuses the multi-agent proximal policy optimization algorithm and the environment perception model based on the driving safety potential field, so that the agent can dynamically switch the behavior strategy according to the current traffic state; by introducing the dynamic quantitative index based on the artificial risk potential field, the system can perceive the traffic flow risk of the region where the agent is located in real time, and use the index as the criterion for switching the strategy, to realize the switching and scheduling of the confrontation action and the natural driving behavior mode; the modeling mechanism of the multi-type background vehicle based on dynamic density is introduced, to further enhance the naturalness, complexity and coverage of the test scene.

[0029] The embodiment introduces the following three improvements on the basis of the traditional MAPPO algorithm: (1) Strategy switching controller module A dynamic evaluation system based on an artificial risk potential field is constructed, the DSF model is used to perform real-time risk perception on the environment around the agent, and a regional risk index is calculated as a quantitative criterion. When the local risk value exceeds a preset threshold, the controller automatically drives the agent to switch from the defense strategy (such as the IDM + MOBIL model) to the attack strategy (based on MAPPO training), so as to trigger the confrontation test behavior of the tested vehicle at a critical moment.

[0030] (2) Heterogeneous strategy cooperative training mechanism A unified value function network structure is used to support the cooperative training and experience sharing of the attack and defense strategies, to solve the sample deviation and unstable convergence problem of the traditional reinforcement learning in the multi-strategy mixed training, and to improve the learning efficiency and strategy generalization ability.

[0031] (3) Improved artificial potential field model The gravity potential field module is designed as a trigger type dynamic activation mechanism, which only participates in the control decision under the condition of meeting the strategy switching condition, thereby significantly reducing unnecessary lateral guidance interference. Meanwhile, the traditional static lane line potential field structure is cancelled, the potential field modeling process is simplified, and the system calculation complexity is reduced. The design can effectively inhibit the invalid lane changing behavior of the agent frequently occurring in the training stage, accelerate the convergence process of the reward function, and enhance the stability and training efficiency of the generated scene.

[0032] It should be noted that the ego vehicle refers to a target vehicle controlled by the automatic driving system to be verified, which is in the core position of the simulation test environment; the target vehicle has autonomous perception, path planning and action execution capability, and its behavior mode is determined by the pre-trained reinforcement learning model, representing the real decision response level of the tested system in a complex traffic environment.

[0033] The multi-agent vehicles deployed in the environment interact with the ego vehicle, and all the multi-agent vehicles are divided into offensive multi-agent vehicles and defensive multi-agent vehicles according to the strategy function; The offensive multi-agent is trained based on the multi-agent proximal policy optimization (MAPPO) algorithm, and has the ability to actively identify the position of the ego vehicle, approach and block the running track of the ego vehicle. Under the premise of controllable risk, through the cooperative disturbance strategy of multi-agent, the ego vehicle is induced to trigger the decision boundary behavior or enter the potential safety failure state; the “centralized training + distributed execution” architecture is adopted, the global information is shared in the training stage to improve the cooperative ability, and the local observation is independently decided in the execution stage, so as to realize the collective interference effect under the decentralized control, thereby enhancing the diversity, antagonism and effectiveness of the generated test scene.

[0034] The defensive multi-agent is mainly used to build the background flow of the traffic environment, and realizes natural and continuous vehicle motion based on the intelligent driver model (IDM) and the lane changing model (MOBIL). Such multi-agent follows a rule-driven strategy and does not have the active interference function for the ego vehicle, and its behavior is limited to acceleration, deceleration or uniform speed within a reasonable acceleration range, simulating the natural driving behavior of typical vehicles in real traffic.

[0035] In order to improve the authenticity and dynamics of the traffic game process, the embodiment adopts a strategy adaptive switching mechanism based on risk perception, so that the above two types of agents can run cooperatively in the same simulation environment; the agent adjusts the strategy mode in real time according to the risk level of the region: in the low-risk area, the defensive strategy is executed to maintain the natural traffic flow state; while in the high-risk area, the risk potential field index (such as the local risk evaluation value based on the Driving Safety Field) is introduced to activate the offensive strategy, trigger the multi-vehicle cooperative disturbance behavior, and then expose the potential decision boundary and safety response capability of the tested system.

[0036] To improve the training efficiency and the diversity of the test environment, the embodiment proposes a dynamic density multi-type background vehicle modeling mechanism. In the reinforcement learning training phase, by configuring a combination of "low-density background traffic + a small number of attacking intelligent agents", the complexity of the scene is controlled, the disturbance noise of strategy training is reduced, and the convergence efficiency is improved. In the simulation test phase, "high-density background traffic" is introduced to create a complex dynamic traffic environment and increase the decision-making difficulty of the measured vehicle, thereby more effectively evaluating its safety performance and system robustness.

[0037] Based on the above intelligent agent structure and strategy scheduling mechanism, the embodiment provides an automatic driving system-oriented adversarial test scene generation method as shown in Figure 2 The specific implementation steps are as follows: (1) Start the simulator and initialize the environment Run the simulation simulator to generate a lane high-speed environment with a lane width of and a length of Generate one measured vehicle and Intelligent agent vehicles in the environment, and randomly set the positions and initial speeds of these vehicles.

[0038] (2) Add algorithm control of the measured vehicle's actions To allow the measured vehicle to have better obstacle avoidance performance and prevent the MAPPO algorithm model from overfitting due to the measured vehicle's decision-making model being too simple, the actions of the measured vehicle are controlled by the pre-trained reinforcement learning model.

[0039] (3) Generate an improved artificial potential field for intelligent agents in the simulation environment The embodiment adopts a simulation environment containing five intelligent agents, one measured vehicle (Ego Vehicle). Each intelligent agent uses different control methods according to its role and strategy state to achieve high-fidelity traffic interaction modeling; the measured vehicle uses a control strategy trained by the Proximal Policy Optimization (PPO) algorithm, whose behavior represents the measured object of the automatic driving system, and is used to evaluate its safety and robustness in complex traffic environments; all background vehicles have strategy switching capabilities and can dynamically switch between defensive strategies and offensive strategies according to the perceived risk level in the environment: It should be noted that, by default, the background vehicle adopts a rule-based natural driving behavior; wherein the longitudinal acceleration is controlled by an intelligent driver model (IDM, Intelligent Driver Model), and the lateral lane-changing behavior follows a minimizing overall braking induced by lane changes (MOBIL, Minimizing Overall Braking Induced by Lane Changes) model, which requires the vehicle to only perform a lane-changing operation under the premise of meeting the adjacent vehicle acceleration safety constraint, ensuring the rationality and safety of the driving behavior; when a specific risk threshold condition is met, the background vehicle will switch to an offensive strategy trained by a multi-agent proximal policy optimization algorithm (MAPPO), forming a challenging interference behavior to the measured vehicle, thereby triggering potential safety risk scenarios.

[0040] To enhance the applicability and effectiveness of the strategy switching mechanism in real traffic scenarios, the embodiment adopts a local strategy switching region design based on situational awareness.

[0041] In a typical three-lane highway scenario, the strategy switching is no longer dependent on fixed time intervals or random triggering events, but is determined based on the spatial structure and local risk level; specifically: as shown in Figure 3 , a rule-based rectangular region is set in front and behind the center line of the ego vehicle as the strategy switching area. When other intelligent agent vehicles enter these areas, their local state is re-evaluated, and whether their strategy needs to be switched from a defensive strategy to an offensive strategy is determined based on their speed, position, and relative distance from the ego vehicle.

[0042] The boundaries of the switching region are set according to Chinese highway traffic experience data. According to the driving direction of the ego vehicle, the fast lane usually has a speed limit of 100-120 km / h and a safe following distance of 70-90 meters; the middle lane has a speed limit of 90-110 km / h and a safe distance of 50-70 meters; and the slow lane has a speed limit of 60-90 km / h and a safe distance of 30-50 meters. Accordingly, the strategy switching region is set as a rectangle extending in the longitudinal direction: 60 meters in front and behind the middle lane, 40 meters in the slow lane, and 80 meters in the fast lane, respectively named as the medium-speed, slow-speed, and high-speed strategy triggering areas to reflect the real driving reaction space and vehicle dynamics constraints.

[0043] To quantify the risk level around the measured vehicle, the embodiment constructs a Figure 4 risk matrix as shown in , which divides the area around the ego vehicle into nine sub-areas, each with lane space boundaries and risk assessment rules: ;

[0044] wherein, denotes the length of the longitudinal policy trigger zone, denote the left and right lane width, respectively.

[0045] Each sub-region ( denotes the longitudinal position, denotes the lateral position) is divided into risk zones as shown in Table 1: Table 1. Risk zone division matrix

[0046] To effectively evaluate the dynamic threat posed by vehicles in each region to the target vehicle, the "driving safety field" model (DSF) introduced in this embodiment uses the kinetic field module therein to model the interaction risk. In the DSF, the agent at the position has a potential risk impact on the target vehicle , which is represented by the kinetic potential , and the calculation formula is:

[0047] wherein, denotes the relative distance of the agent to the evaluation point , i.e., the center point of the risk matrix; denotes the included angle between the direction of the agent and the direction of the ego vehicle; denotes the speed of the agent ; , denote the road condition factor and the virtual mass of the vehicle, respectively; denote the model parameters, respectively.

[0048] The total risk value of each region is obtained by aggregating the risk contributions of all vehicles within it ; to achieve adaptive switching between natural driving and aggressive behavior, this embodiment sets a risk threshold . When the risk value of any region > , it indicates that the agent in that region poses a high-risk threat to the target vehicle, and the policy switching should be triggered to execute the offensive strategy to attack the target vehicle; otherwise, the defensive strategy should be continued to execute to simulate the behavior of a natural driving vehicle.

[0049] To solve the strategy conflict caused by multiple vehicles entering the high-risk zone at the same time, this embodiment adopts a hierarchical triggering mechanism based on density.

[0050] Define the normalized density index within the observation range of the ego vehicle ; wherein, is the number of vehicles in the switching zone; is the area of the monitoring zone, which is set as a rectangle with a length of 100 meters in the longitudinal direction and a width of one lane (3.75 meters) in the lateral direction by default.

[0051] According to the traffic flow characteristics of the expressway, the density threshold is divided as follows:

[0052] Combined with the risk matrix division (A-I), the composite risk value of each direction is calculated respectively:

[0053] wherein the weighting coefficients are designed as follows: represents the sensitivity of the driver's attention to the front; = represents the sensitivity of emphasizing the lateral conflict in the lane changing process; represents the reduction of the impact of backward risk in the strategy switching.

[0054] When multiple agents simultaneously meet the switching conditions, a three-level priority determination mechanism is adopted to realize the strategy switching control of space sensitivity and risk perception, which significantly enhances the response ability and generation ability of the autonomous driving system to typical risk scenarios; wherein the three-level priority determination mechanism includes: (1) Risk Dominant Principle: Prioritize switching the agent with the highest composite directional risk value; (2) Dynamic State Constraint: Agents in deceleration state (acceleration ) do not participate in switching; (3) Space Sequence: If the risk values are equal, the switching priority is determined according to the time sequence of entering the switching zone.

[0055] This embodiment constructs an improved artificial potential field model containing attractive and repulsive fields. By superimposing the two potential fields, a composite potential field is formed to drive multiple agents to cooperatively generate challenging confrontation test scenarios.

[0056] This embodiment introduces a strategy switching mechanism based on local environmental perception. The agent only changes from natural driving mode to offensive strategy when entering a specific "strategy switching area". It can be based on conditions such as space division rules or risk threshold judgment.

[0057] On this basis, the gravity field adopts a trigger design: only when the agent is located in the policy switching area and meets the policy switching condition, the gravity field is activated; the measured vehicle or its surrounding area is set as the potential energy minimum point of the gravity field; the gravity potential energy is calculated according to the relative position relationship between the agent and the target point , so as to exert an attractive force on the agent and guide it to move towards the measured vehicle.

[0058] The agent outside the policy switching area is not affected by the gravity field and continues to execute the rule-based natural driving strategy (such as the defense strategy), maintaining normal driving behavior. When the gravity field is activated, the agent generates adversarial actions in combination with the MAPPO algorithm, gradually approaching the measured vehicle in dynamic interaction, and thus forming a potential high-risk interaction scenario.

[0059] The form of the gravity potential energy function is as follows:

[0060] Vehicles are dynamic agents in the traffic environment, and factors such as vehicle braking, steering performance, and road structure parameters can affect the risk level of other vehicles. Therefore, a vehicle repulsive field is constructed as follows

[0061] wherein, is the geometric center coordinate of other agents in the road coordinate system. The risk coefficient will be higher when the vehicle is closer to other agents; is the dangerous potential energy switching threshold when the vehicle approaches other agents; is the lateral coefficient of the repulsive field; is the longitudinal coefficient of the repulsive field.

[0062] The improved artificial potential field for assisting in designing the reward function is obtained by superimposing the gravity field and the agent vehicle repulsive field, that is, ; wherein, .

[0063] (4) Algorithm reward function design In this embodiment, the potential energy difference of the improved artificial potential field between the agent's previous and subsequent time steps is used as part of the reward function. The design of the reward function mainly considers two factors: one is the agent's performance in adversarial interaction with the measured vehicle and collision, and the other is the agent's ability to make reasonable path planning according to the potential energy change of the artificial potential field, so that the agent's behavior is closer to reality. The designed reward function R is composed of the collision measured vehicle reward and the potential energy change reward function of the improved artificial potential field: Collision reward:

[0064] Potential change reward of artificial potential field model:

[0065] wherein, is the artificial potential energy of the moment, is the artificial potential energy of the moment, is the artificial potential energy of the moment, is the artificial potential energy of the moment, potential change reward of artificial potential field model is the difference between the artificial potential energy of the moment and the moment before, helping the agent to find the lowest point of the artificial potential energy, that is, the vicinity of the measured vehicle. If the agent collides with the measured vehicle, the collision reward is 120.

[0066] (5) Interaction between agent and environment The embodiment integrates and processes the behavior data of each agent to construct action vector and observation vector , wherein each element respectively represents a specific operation instruction executed by the corresponding agent and environment perception data obtained, including the motion state and spatial position of the surrounding vehicles and other key information.

[0067] During the system running process, the environment module first outputs the global state information, and after the quantitative processing of the dangerous potential energy, the potential energy distribution map and the risk heat map of each agent are generated. The strategy selector dynamically switches between the offensive strategy and the defensive strategy according to the preset risk threshold, and outputs the optimal control instruction through the neural network model. After each agent executes the instruction, the environment state is updated synchronously to form a closed loop feedback.

[0068] The embodiment generates interaction as shown in Figure 5 , in the state updating stage, the system will comprehensively evaluate the potential change value and the collision detection result to calculate the instant reward, wherein the potential difference reward reflects the path planning effect, and the collision reward and punishment mechanism is used to regulate the interaction behavior. All interaction data are classified and stored according to the strategy type, when the sample quantity reaches the training threshold, the system automatically starts the model optimization program to realize the continuous improvement of the agent behavior.

[0069] (6) MAPPO algorithm model training During the training process, the agent samples data from the experience pool according to the strategy type, calculates the loss function and reversely propagates to update the network parameters. After sufficient training, the agent can be independent of the central controller in the execution stage, and make strategy decisions based on local observation, realizing the distributed confrontation test goal.

[0070] During the test process, as shown in Figure 6 a plurality of trained agents construct complex traffic interactions without violating traffic rules, induce the tested vehicle to be in a high-risk decision state, and thus realize the testing and verification of the decision boundary of the tested vehicle.

[0071] The embodiment adopts multi-agent reinforcement learning with a multi-strategy adaptive switching mechanism, fuses a multi-agent proximal policy optimization algorithm (i.e., Flexible Adversarial Policy-switching MAPPO) and an environment perception model based on a driving safety field (Driving Safety Field, DSF), so that the agent can dynamically switch the behavior strategy according to the current traffic state; by introducing a dynamic quantitative index based on an artificial risk field, the traffic flow risk of the region where the agent is located is perceived in real time, and the dynamic quantitative index based on the artificial risk field is used as a criterion for strategy switching, so as to realize the switching and scheduling of the adversarial action and the natural driving behavior mode; a dynamic density multi-type background vehicle modeling mechanism is introduced to enhance the naturalness, complexity and coverage of the test scene; the problems of insufficient naturalness and single adversarial vehicle strategy of the existing automatic driving test scene generation method are overcome, the test coverage, risk exposure ability and evaluation representativeness of automatic driving are improved, and a safe and critical test scene with natural rationality and risk resistance is efficiently generated in a simulation environment.

[0072] Embodiment two The embodiment two of the present application introduces an automatic driving adversarial test scene generation system based on strategy switching.

[0073] As shown in Figure 7 an automatic driving adversarial test scene generation system based on strategy switching, comprising: an acquisition module configured to acquire an automatic driving simulation scene; a setting module configured to set the positions and initial speeds of a target vehicle and an agent vehicle in the acquired simulation scene; an optimization module configured to combine the set positions and initial speeds of the target vehicle, adopt a multi-agent proximal policy optimization algorithm, and optimize the running state of the target vehicle to control the target vehicle; a determination module configured to determine a current traffic state according to the controlled running state of the target vehicle; a perception module configured to combine a dynamic quantitative index based on an artificial risk field to perceive the traffic flow risk of the agent vehicle in the determined current traffic state in real time; The generating module is configured to automatically switch the defense strategy and the attack strategy of the intelligent vehicle according to the obtained traffic flow risk level, and drive the multi-intelligent vehicle to cooperatively generate the adversarial test scene of the automatic driving.

[0074] The detailed steps are the same as those of the method for generating the adversarial test scene of the automatic driving based on the strategy switching provided in Embodiment 1, and will not be described herein.

[0075] Embodiment 3 The embodiment 3 of the present application provides a computer readable storage medium.

[0076] A computer readable storage medium has a program stored thereon, and the program is executed by a processor to implement the steps in the method for generating the adversarial test scene of the automatic driving based on the strategy switching according to the embodiment 1 of the present application.

[0077] The detailed steps are the same as those of the method for generating the adversarial test scene of the automatic driving based on the strategy switching provided in Embodiment 1, and will not be described herein.

[0078] Embodiment 4 The embodiment 4 of the present application provides an electronic device.

[0079] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor, and the processor implements the steps in the method for generating the adversarial test scene of the automatic driving based on the strategy switching according to the embodiment 1 of the present application when executing the program.

[0080] The detailed steps are the same as those of the method for generating the adversarial test scene of the automatic driving based on the strategy switching provided in Embodiment 1, and will not be described herein.

[0081] Embodiment 5 The embodiment 5 of the present application provides a computer program product.

[0082] A computer program product includes software code, and a program in the software code executes the steps in the method for generating the adversarial test scene of the automatic driving based on the strategy switching according to the embodiment 1 of the present application.

[0083] The detailed steps are the same as those of the method for generating the adversarial test scene of the automatic driving based on the strategy switching provided in Embodiment 1, and will not be described herein.

[0084] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, the methods can be tangibly embodied in a machine-readable storage medium having stored thereon instructions that can be used to program a computer to perform any of the methods. The software implementation can be initialized by loading and executing a set of instructions arranged to perform one of the methods into the computer's memory. Alternatively, hard-wired circuitry can be used in place of, or in combination with, software instructions. Thus, the

[0085] The present application is described in reference to the flowchart and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 means for performing the function specified by the flowchart and / or block diagram block or blocks.

[0086] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 means for performing the function specified by the flowchart and / or block diagram block or blocks.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 means for performing the function specified by the flowchart and / or block diagram block or blocks.

[0088] While preferred embodiments of the application have been described, modifications and alterations thereto will occur to those skilled in the art upon reading the preceding description. In particular, it will be apparent to those skilled in the art that parts can be added to, or substituted for, parts of the described embodiment. It is therefore desired to be secured to the appended claims as they follow.

[0089] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0090] The above description is merely that of the preferred embodiments of the present application and is not intended to limit the application. One skilled in the art can readily devise variations and modifications without departing from the spirit and scope of the present application as defined by the following claims.

Claims

1. A method for generating autonomous driving adversarial test scenarios based on policy switching, characterized in that, include: Obtain autonomous driving simulation scenarios; Set the positions and initial speeds of the target vehicle and the agent vehicle in the acquired simulation scenario; Based on the set position and initial speed of the target vehicle, a multi-agent proximal strategy is adopted to optimize the operating state of the target vehicle and control the target vehicle. Determine the current traffic status based on the target vehicle's controlled driving status; By combining dynamic quantitative indicators of artificial risk potential field, the traffic flow risk of intelligent agent vehicles in the current traffic state can be perceived in real time. Based on the obtained traffic flow risk level, the defense and offense strategies of intelligent agent vehicles are automatically switched, driving multiple intelligent agent vehicles to collaboratively generate adversarial test scenarios for autonomous driving.

2. The method for generating autonomous driving adversarial test scenarios based on policy switching as described in claim 1, characterized in that, In the process of automatically switching between the defense and attack strategies of intelligent vehicles, a density-based hierarchical triggering mechanism is adopted. When several intelligent agents simultaneously meet the strategy switching conditions, a three-layer priority determination mechanism is used.

3. The method for generating autonomous driving adversarial test scenarios based on policy switching as described in claim 2, characterized in that, The three-layer priority determination mechanism includes a risk-dominant principle mechanism that prioritizes switching the agent with the highest directional risk value, a motion state constraint mechanism that agents in a deceleration state do not participate in policy switching, and a spatiotemporal order mechanism that determines the switching priority based on the time order of entering the policy switching region when risk values ​​are equal.

4. The method for generating autonomous driving adversarial test scenarios based on policy switching as described in claim 1, characterized in that, The artificial risk potential field includes at least a composite potential field of a gravitational potential field based on a trigger-based gravity mechanism in the policy switching region and a repulsive potential field of the intelligent vehicle; the artificial risk potential field is used to assist in the design of a reward function, which includes a reward function for the collision target vehicle and a reward function for the potential energy change of the artificial potential field.

5. The method for generating autonomous driving adversarial test scenarios based on policy switching as described in claim 1, characterized in that, Automatic switching between defensive and offensive strategies for intelligent agent vehicles is performed in context-aware local policy switching zones. Rectangular areas are set before and after the centerline of the target vehicle, and these rectangular areas serve as policy switching zones. When other intelligent agent vehicles enter the policy switching zone, the local state of the intelligent agent vehicles is reassessed, and the speed of the intelligent agent vehicles and the relative distance between the intelligent agent vehicles and the target vehicle determine whether it is necessary to switch between the defensive and offensive strategies of the intelligent agent vehicles.

6. The method for generating autonomous driving adversarial test scenarios based on policy switching as described in claim 1, characterized in that, In the process of driving multi-agent vehicles to collaboratively generate autonomous driving adversarial test scenarios, the operating state and loss function of the agent vehicles are obtained from the experience pool according to the policy type. The obtained loss function is backpropagated to update the network parameters. The policy decisions of the agent vehicles are obtained based on local observations. The traffic flow around the target vehicle is disturbed according to the obtained policy decisions of the agent vehicles. Traffic interaction between the target vehicle and the agent vehicles is constructed to realize the testing and verification of the adversarial test target and complete the generation of autonomous driving adversarial test scenarios.

7. A system for generating autonomous driving adversarial test scenarios based on policy switching, characterized in that, include: The acquisition module is configured to acquire autonomous driving simulation scenarios; The setup module is configured to set the position and initial speed of the target vehicle and the agent vehicle in the acquired simulation scenario. The optimization module is configured to combine the set position and initial speed of the target vehicle and use a multi-agent proximal strategy to optimize the operating state of the target vehicle and control the target vehicle. The determination module is configured to determine the current traffic state based on the controlled driving state of the target vehicle; The perception module is configured to combine dynamic quantitative indicators of artificial risk potential field to perceive the traffic flow risk of intelligent agent vehicles in the determined current traffic state in real time. The generation module is configured to automatically switch between defensive and offensive strategies of intelligent agent vehicles based on the obtained traffic flow risk level, driving multiple intelligent agent vehicles to collaboratively generate adversarial test scenarios for autonomous driving.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the autonomous driving adversarial test scenario generation method based on policy switching as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the autonomous driving adversarial test scenario generation method based on policy switching as described in any one of claims 1-6.

10. A computer program product, comprising software code, characterized in that, The program in the software code executes the steps of the autonomous driving adversarial test scenario generation method based on policy switching as described in any one of claims 1-6.

Images