Personified automatic driving simulation test scene construction method

Abstract

The invention provides an anthropomorphic automatic driving simulation test scene construction method. The method comprises the steps that all vehicles in a traffic scene are divided into two types of intelligent agents including a test vehicle and an environment vehicle, the traffic scene is modeled into a Markov decision process, the test vehicle is a vehicle controlled by an automatic driving algorithm, and the environment vehicle is a vehicle controlled by a GAIL-GRU driving strategy model; a Markov decision process is utilized to extract a driving track from the human driving data set, and an expert track data set is generated; and training a GAIL-GRU driving strategy model by using the expert track data set, realizing interaction between an environment vehicle and a test vehicle by using the trained GAIL-GRU driving strategy model, and constructing a simulation test scene. The automatic driving simulation test scene constructed by the invention can effectively expose decision defects of an automatic driving algorithm in a complex interaction situation, has good anthropomorphism and relatively high risk scene coverage, and provides support for a decision control simulation test of a high-level automatic driving vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method for constructing anthropomorphic autonomous driving simulation test scenarios. Background Technology

[0002] The verification of autonomous driving technology heavily relies on simulation testing of complex traffic scenarios, but existing scenario generation methods face significant technical bottlenecks. First, traditional rule-driven methods generate scenarios through manually defined logic, which, while ensuring basic safety logic, struggle to adapt to the complexity of dynamic interactions. Second, methods based on real data reconstruction, while preserving the natural characteristics of driving behavior, are limited by data acquisition costs and the long-tail distribution problem. Dangerous scenarios constitute a very small percentage of real road data; direct data-driven approaches easily lead to insufficient coverage of high-risk scenarios, while data augmentation techniques may compromise the authenticity of the original behavioral logic. Furthermore, existing trajectory generation models generally suffer from deficiencies in multimodal behavioral representation capabilities.

[0003] While current methods based on generative adversarial networks (GANs) and reinforcement learning have improved scene generation capabilities to some extent, they still suffer from key shortcomings. The pattern collapse problem is particularly prominent in GANs: the discriminator overfits the distribution of expert data, forcing the generator into local optima, enabling it to reproduce only limited scene patterns and failing to generate diverse interactive behaviors. Furthermore, most models neglect the temporal continuity of driving behavior; in terms of dynamic interaction adaptability, existing methods mostly use static behavior labels, unable to dynamically adjust strategies based on real-time scene states. These limitations make it difficult for existing methods to construct test scenarios that combine high risk, behavioral diversity, and physical plausibility, thus hindering the efficiency of safety verification for autonomous driving algorithms.

[0004] Currently, there is an urgent need for a novel generative framework that integrates multimodal behavioral representation and temporal dependency modeling for autonomous driving simulation testing scenarios, in order to overcome the technical bottlenecks in autonomous driving simulation testing. Summary of the Invention

[0005] The embodiments of the present invention provide a method for constructing an anthropomorphic autonomous driving simulation test scenario, so as to effectively support the decision-making and control simulation test of autonomous vehicles.

[0006] To achieve the above objectives, the present invention adopts the following technical solution.

[0007] A method for constructing an anthropomorphic autonomous driving simulation test scenario includes:

[0008] All vehicles in the traffic scenario are divided into two types of intelligent agents: test vehicles and environment vehicles. The traffic scenario is modeled as a Markov decision process. The test vehicles are vehicles controlled by the autonomous driving algorithm, and the environment vehicles are vehicles controlled by the GAIL-GRU driving strategy model.

[0009] We use Markov decision processes to extract driving trajectories from human driving datasets and then use these driving trajectories to generate expert trajectory datasets.

[0010] Construct a GAIL-GRU driving strategy model, and train the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model.

[0011] The environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model are deployed in a static environment. The interaction between the environmental vehicle and the test vehicle is realized using the trained GAIL-GRU driving strategy model to construct a simulation test scenario.

[0012] Preferably, the process of classifying all vehicles in the traffic scenario into two categories of intelligent agents—test vehicles and environment vehicles—and modeling the traffic scenario as a Markov decision process includes:

[0013] All vehicles in the traffic scenario are divided into two types of intelligent agents: test vehicles and environment vehicles. Test vehicles are vehicles controlled by autonomous driving algorithms, while environment vehicles are vehicles controlled by the GAIL-GRU driving strategy model.

[0014] In a traffic scenario, each vehicle aims to maximize the reward during the driving process, namely driving efficiency and safety. This invention models the traffic scenario as a Markov decision process, which is represented by tuples (S,A1,A2,π1,π2,T,R1,R2).

[0015] State set: S represents the set of states of all vehicles in the current traffic scenario, including vehicle speed and position;

[0016] Action set: A1 represents the set of actions of the test vehicle in the traffic scene, and A2 represents the set of environmental vehicle states in the traffic scene. The lateral and longitudinal acceleration of the vehicle are used as the vehicle's actions.

[0017] Policy set: π1 represents the policy set of the test vehicle, π2 represents the policy set of the environment vehicles, and the policy function determines the action selected by the vehicle in the current state;

[0018] Transition probability function: T represents the state transition model that calculates the next state based on the current state and action, and determines the probability of transitioning to the next state given the current state and action;

[0019] Reward functions: R1 and R2 represent the reward functions of the test vehicle and the environment vehicle, respectively. R1(s,a) represents the immediate reward obtained by the test vehicle after it performs action a from state s.

[0020] Preferably, the step of extracting driving trajectories from human driving datasets using Markov decision processes and generating expert trajectory datasets using these driving trajectories includes:

[0021] This study utilizes Markov decision processes to extract driving trajectories from human driving datasets. Tuples (S, A1, A2, π1, π2, T, R1, R2) are used to describe the interaction process in traffic scenarios. Vehicle driving trajectories and multi-dimensional features are extracted. Multiple agents take actions A1 and A2 in state S and make decisions π1 and π2 according to the policy. They then transition to the next state through the transition function T and obtain rewards R1 and R2. Based on vehicle behavior, driving trajectories are divided into straight-ahead, left lane change, and right lane change, and labels are assigned to construct an expert trajectory dataset.

[0022] The state characteristics of each vehicle are defined as shown in Table 1:

[0023] Table 1

[0024]

[0025]

[0026] Preferably, the construction of the GAIL-GRU driving strategy model, which involves training the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model, includes:

[0027] Construct a GAIL-GRU driving strategy model. This Triple-GAIL driving strategy model includes a generator, a selector, and a discriminator. The generator contains a policy network π. θ Value Network V θ The selector C is parameterized by α. α Vehicle pseudo-behavior labels are generated based on the scene state and then input into the generator π, which is parameterized by θ. θ In the process, the generator generates vehicle actions based on the scene state and pseudo-behavior labels, and finally, the discriminator D, parameterized by ψ, generates the actions. ψ To determine whether the state-label-action pair originates from an expert trajectory, the selector and generator work together against the discriminator, forming a game process that minimizes α and θ and maximizes ψ. The calculation formula is as follows:

[0028]

[0029] Policy Network π θ The input data includes behavior labels c t and vehicle status s t The output data is the vehicle's acceleration action a. t For the policy network π θA value function based on state-label-action is adopted. (c t ,s t ,a t )Evaluate the current behavior label c t and vehicle status s t Next, execute action a t The expected return, which is estimated by the value network;

[0030] A vehicle trajectory discriminator is established. This discriminator maps state-label-action pairs to scores to distinguish between expert trajectories and trajectories generated by driving strategies. In the update step, the discriminator D... ψ The discriminator scores the generated state-label-action pairs and the state-label-action pairs from the expert trajectory, with the goal of scoring the input from policy π. θ The state-label-action pair outputs a low score, with input from expert trajectory π. E The state-label-action pair outputs a high score, and the loss function and gradient update of the discriminator with parameter ψ are shown in the following equation:

[0031]

[0032] In the formula L discrim The cross-entropy loss of the discriminator, This represents the expected value of the expert data distribution. D represents the expectation D on the data distribution of the agent. ψ (s,a,c) represents the probability that the data (s,a,c) belongs to expert data, ω represents the weighting coefficient, and log(1―D) ψ (s,a,c)) represents the negative example loss of the generated data, logD ψ (s,a,c) represents the positive example loss:

[0033]

[0034] In the formula N is the gradient of the discriminator loss. e The sample size of the expert data. Let N represent the number of samples generated, and T represent the number of samples generated. j This represents the length of the j-th trajectory. ω represents the generated sample, and ω represents the weight coefficient.

[0035] Preferably, the step of training the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model includes:

[0036] The sliding time window method was used to extract 500 straight-line trajectories, 500 left lane-change trajectories, and 500 right lane-change trajectories from the expert trajectory dataset. The offset trajectories were divided into time series of length k as training data for the GAIL-GRU driving strategy model. The GAIL-GRU driving strategy model was trained using the training data.

[0037] The environment is reset using time-series data with behavioral labels. Each time-series data has a fixed real behavioral label. The agent runs a policy based on the real behavioral label, generates and collects trajectories. The generated trajectories are fed into the selector to generate pseudo-behavioral labels. The trajectories generated by the generator and selector are used as pseudo data, and the expert trajectories are used as real data. The discriminator scores the two types of data and generates a reward as optimization feedback for the selector and generator. The discriminator network is updated through gradient ascent, the selector network is updated through gradient descent, and the policy network and value network are updated using the proximal policy optimization algorithm.

[0038] When the discriminator can no longer effectively distinguish between expert trajectories and generated trajectories, that is, when the discriminator output approaches 0.5, the adversarial training has converged, the training of the GAIL-GRU driving strategy model ends, and the trained GAIL-GRU driving strategy model is obtained.

[0039] Preferably, the step of deploying the environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model in a static environment, and using the trained GAIL-GRU driving strategy model to realize the interaction between the environmental vehicle and the test vehicle to construct a simulation test scenario includes:

[0040] The environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model are deployed in a static environment. The simulation is started, and the control strategy for each group of vehicles is selected. The environmental vehicle in the scenario is controlled by the GAIL-GRU driving strategy model, and the test vehicle is controlled by the autonomous driving algorithm under test.

[0041] The initial scene state of a group of vehicles is randomly sampled from the constructed expert trajectory as the simulation environment. The simulation environment extracts a set of state features for each vehicle from the expert trajectory dataset as shown in Table 1, and inputs the state features into the GAIL-GRU driving strategy model and the autonomous driving algorithm.

[0042] The selector of the GAIL-GRU driving strategy model outputs behavior labels, the generator generates lateral and longitudinal accelerations based on the state-label pair, and the autonomous driving algorithm generates lateral and longitudinal accelerations based on the state.

[0043] The GAIL-GRU driving strategy model and autonomous driving algorithm output the longitudinal and lateral acceleration values ​​of the vehicle as the vehicle's actions, and the simulation environment updates the position and velocity of all vehicles in the environment.

[0044] The above processing procedure is repeated at a frequency of 25Hz until a collision occurs or 250 time steps are reached, thus establishing a simulation test scenario containing a driving process of up to 10 seconds. The performance of the autonomous vehicle is then tested using the simulation test scenario.

[0045] As can be seen from the technical solutions provided by the embodiments of the present invention above, the autonomous driving simulation test scenario constructed by the present invention can effectively expose the decision-making defects of autonomous driving algorithms in complex interactive situations, has good anthropomorphism and high coverage of risk scenarios, and provides support for the decision-making and control simulation test of high-level autonomous vehicles.

[0046] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram illustrating the implementation principle of a method for constructing an anthropomorphic autonomous driving simulation test scenario, provided in an embodiment of the present invention.

[0049] Figure 2 The flowchart illustrates a method for constructing an anthropomorphic autonomous driving simulation test scenario, as provided in an embodiment of the present invention.

[0050] Figure 3 This is a schematic diagram of a traffic scenario definition method based on Markov decision process provided in an embodiment of the present invention.

[0051] Figure 4 This is a schematic diagram of a vehicle trajectory simulation learning method based on GAIL-GRU provided in an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram of a simulation scene construction method based on GAIL-GRU vehicle trajectory generation provided in an embodiment of the present invention. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0056] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0057] This invention provides a method for constructing an anthropomorphic autonomous driving simulation test scenario. The main process includes defining and extracting traffic scenarios based on Markov decision processes, constructing a trajectory learning model based on Generative Adversarial Imitation Learning with Gated Recurrent Units (GAIL-GRU), training the model based on real driving datasets, and constructing an autonomous driving simulation scenario.

[0058] The implementation principle of the anthropomorphic autonomous driving simulation test scenario construction method provided in this embodiment of the invention is as follows: Figure 1 As shown, the specific processing flow is as follows: Figure 2As shown, the processing steps include the following:

[0059] Step S1: Divide all vehicles in the traffic scenario into two categories of intelligent agents: test vehicles and environment vehicles. Model the traffic scenario as a Markov decision process. Test vehicles are vehicles controlled by the autonomous driving algorithm, while environment vehicles are vehicles controlled by the GAIL-GRU driving policy model.

[0060] All vehicles in the traffic scenario are divided into two types of intelligent agents: test vehicles and environment vehicles. Test vehicles are controlled by autonomous driving algorithms, while environment vehicles are controlled by the GAIL-GRU driving strategy model. By constructing a simulation test scenario through the interaction of these two types of vehicles, the performance of autonomous driving algorithms can be tested.

[0061] During the journey, both the test vehicle and the environment vehicle continuously assess various characteristics of surrounding vehicles and determine their actions based on these characteristics. This entire process consists of a series of decisions made by the vehicle, ultimately forming a trajectory. Each vehicle in the traffic scenario aims to maximize the rewards during the driving process, namely driving efficiency and safety. Figure 3 This is a schematic diagram of a traffic scenario based on a Markov decision process, provided as an embodiment of the present invention. Therefore, the present invention models the traffic scenario as a Markov decision process, using tuples (S, A1, A2, π1, π2, T, R1, R2) to represent the above Markov decision process, as follows: Figure 3 As shown.

[0062] State set: S represents the set of states of all vehicles in the current traffic scenario, including vehicle speed, position, and other vehicle states.

[0063] Action set: A1 represents the action set of the test vehicle in the traffic scene, and A2 represents the state set of the environmental vehicle in the traffic scene. This invention uses the lateral and longitudinal acceleration of the vehicle as the action of the vehicle.

[0064] Policy set: π1 represents the policy set of the test vehicle, and π2 represents the policy set of the environment vehicles. The policy function determines the action chosen by the vehicle in the current state. In this invention, the action of the test vehicle is determined by the test autonomous driving algorithm, while the action of the environment vehicles is controlled by the GAIL-GRU driving policy model. Human driving trajectories are extracted from the HighD human driving dataset and used as model input. After training, the GAIL-GRU driving policy model is obtained. The GAIL-GRU driving policy model can interact with autonomous vehicles by adopting driving behaviors similar to those of human drivers.

[0065] Transition probability function: T represents the state transition model that calculates the next state based on the current state and action, and determines the probability of transitioning to the next state given the current state and action.

[0066] Reward functions: R1 and R2 represent the reward functions of the test vehicle and the environment vehicle, respectively. R1(s,a) represents the immediate reward obtained by the test vehicle after it performs action a from state s.

[0067] Step S2: Extract driving trajectories from the human driving dataset using Markov decision processes, and generate an expert trajectory dataset using the driving trajectories.

[0068] To extract driving trajectories from human driving datasets and construct expert trajectories, vehicle state features need to be extracted. Vehicle driving trajectories and multi-dimensional features are extracted from the human driving datasets, and driving trajectories are divided into straight-ahead, left lane change, and right lane change categories based on vehicle behavior, with labels assigned to each category, thus constructing an expert trajectory dataset. The expert trajectory dataset consists of real human driving data from the human driving datasets and is used to train the GAIL-GRU driving strategy model.

[0069] Tuples (S, A1, A2) are needed to extract vehicle trajectories and multi-dimensional features. Simultaneously, tuples (S, A1, A2, π1, π2, T, R1, R2) are used to describe the interaction process in the traffic scenario. Multiple agents take actions A1 and A2 in state S and make decisions π1 and π2 according to the policy. They then transition to the next state through the transition function T and receive rewards R1 and R2.

[0070] The state characteristics of each vehicle are defined in the table below.

[0071]

[0072]

[0073] Step S3: Construct the GAIL-GRU driving strategy model. Train the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model.

[0074] The structure of a GAIL-GRU driving strategy model provided in this embodiment of the invention is as follows: Figure 4 As shown, the Triple-GAIL driving strategy model comprises three key components: a generator, a selector, and a discriminator.

[0075] The Triple-GAIL driving strategy model consists of three components: a generator, a selector, and a discriminator. The generator comprises a policy network π. θ Value Network V φ Two core components. First, there's the α-parameterized selector C. α Vehicle pseudo-behavior labels are generated based on the scene state and then input into the generator π, which is parameterized by θ.θ In the process, the generator generates vehicle actions based on the scene state and pseudo-behavior labels, and finally, the discriminator D, parameterized by ψ, generates the actions. ψ Determine whether the state-label-action pair originates from an expert trajectory. In this process, the selector and generator work together against the discriminator, forming a game-theoretic process that minimizes α and θ and maximizes ψ, as shown below. Figure 4 As shown. The calculation formula is as follows:

[0076]

[0077] Policy Network π θ The input data includes behavior labels c t and vehicle status s t The output data is the vehicle's acceleration action a. t For the policy network π θ A value function based on state-label-action is adopted. Evaluation of current behavior label c t and vehicle status s t Next, execute action a t The expected reward is estimated by the value network. This expected reward is used as a coefficient when updating the policy network. A higher expected reward for a given action will encourage the policy network to generate similar actions more frequently.

[0078] Step 2-1: Establish a vehicle action generator, which includes a policy network and a value network. The input to the policy network includes behavior labels c. t and vehicle status s t The output is the vehicle's acceleration action a. t For the policy network π θ A value function based on state-label-action is adopted. Evaluation of current behavior label c t and vehicle status c t Next, execute action a t The expected reward is estimated by the value network. This expected reward is used as a coefficient when updating the policy network. A higher expected reward for a given action will encourage the policy network to generate similar actions more frequently.

[0079] Value Network V φ A deep neural network structure is employed to estimate the function Q. Its training objective is achieved through a value loss function L. value The implementation is shown in the following formula.

[0080]

[0081] In the formula L valueIt is a value loss function. Represents the mathematical expectation. G(c) represents the state-action value function. t ,s t ,a t ) represents the γ discount return, as shown in the following formula.

[0082]

[0083] In the formula G(c t ,s t ,a t R(c) represents the γ-discounted return. t ,s t ,a t Let π be the reward function, γ be the discount factor, and π be the π / 4. θ (c t+1 ,s t+1 ,a t+1 ) is the policy function. This is the state-action value function.

[0084] Step 2-2: Create a vehicle behavior label selector, selector C α It is a conditional probability network, with the current state s as input. t and historical action a t―1 Output behavior label c t The distribution of the selector. The core objective of the selector is to select pseudo-behavior labels suitable for the current scene and input these pseudo-behavior labels into the generator to guide the generator's action generation. The loss function and gradient update of the selector with parameter α are shown in the following equation.

[0085]

[0086] Steps 2-3: Establish a vehicle trajectory discriminator. The discriminator maps state-label-action pairs to scores to distinguish between expert trajectories and trajectories generated by driving strategies. In the update step, the discriminator D... ψ The generated state-label-action pairs and the state-label-action pairs from expert trajectories will be scored separately. The discriminator's goal is to score the input from policy π. θ The state-label-action pair outputs a low score, with input from expert trajectory π. E The state-label-action pair outputs a high score, and the loss function and gradient update of the discriminator with parameter ψ are shown in the following equation.

[0087]

[0088] In the formula L discrim The cross-entropy loss of the discriminator, This represents the expected value of the expert data distribution. D represents the expectation D on the data distribution of the agent. ψ (s,a,c) represents the probability that the data (s,a,c) belongs to expert data, ω represents the weighting coefficient, and log(1―D) ψ (s,a,c)) represents the negative example loss of the generated data, logD ψ (s,a,c) represents the positive example loss.

[0089]

[0090] In the formula N is the gradient of the discriminator loss. e The sample size of the expert data. Let N represent the number of samples generated, and T represent the number of samples generated. j This represents the length of the j-th trajectory. ω represents the generated sample, and ω represents the weight coefficient.

[0091] Steps 2-3: Incorporate gated recurrent units (GRUs) into the Triple-GAIL generator. By using GRUs to capture temporal dependencies and reduce the cumulative error of discrete data, the displacement deviation problem caused by acceleration integration can be alleviated.

[0092] GRU consists of two parts: update gate and reset gate. It uses hidden state to pass information. The working process of GRU is as follows.

[0093] Reset Door:

[0094] R k+1 =σ(X) k+1 W xr +h k W hr +b r )

[0095] In the formula R k+1 Let X be the reset gate vector at time step k+1. k+1 h is the input feature vector at time step k+1. k Let W be the hidden state vector at time step k. x For the input weight matrix, W h Let b be the hidden state weight matrix. r σ is the bias vector for resetting the gate, and σ is the Sigmoid activation function.

[0096] Update Gate:

[0097] Z k+1 =σ(X) k+1 Wxz +h k W hz +b z )

[0098] In the formula Z k+1 Let X be the update gate vector at time step k+1. k+1 h is the input feature vector at time step k+1. k Let W be the hidden state vector at time step k. x For the input weight matrix, W h Let b be the hidden state weight matrix. z To update the bias vector of the gate, σ is the Sigmoid activation function.

[0099] Candidate hidden states:

[0100]

[0101] In the formula Let X be the candidate hidden state vector at time step k+1. k+1 Let R be the input feature vector at time step k+1. k+1 To reset the gate vector, h k Let W be the hidden state vector at time step k. xh For the input weight matrix, W hh Let b be the hidden state weight matrix. h Let be the bias vector of the candidate states, and tanh be the hyperbolic tangent activation function.

[0102] Hidden state:

[0103]

[0104] In the formula h k+1 Z is the final hidden state vector at time step k+1. k+1 h is the update gate vector at time step k+1. k Let K be the hidden state vector at time step k. Let be the candidate hidden state vector at time step k+1.

[0105] In the formula, assume the input is an X with batch size n and sequence length d. k+1 ∈R n×d Assuming the number of hidden units is h, then W xr W xz W xh ∈R d×h and W hr W hz W hh ∈R h×h It is the weight parameter, b r ,bz ,b h ∈R 1×h It is a bias parameter.

[0106] Step S4: Using a sliding time window method, extract 500 straight-ahead trajectories, 500 left-lane-changing trajectories, and 500 right-lane-changing trajectories from the real driving dataset. Divide the offset trajectories into time series of length k as training data for the GAIL-GRU driving strategy model, and train the GAIL-GRU driving strategy model.

[0107] The training process of the GAIL-GRU driving policy model is as follows: First, the environment is reset using expert trajectories with behavioral labels, each expert trajectory having a fixed real behavioral label. Next, the agent runs a policy based on these behavioral labels, generating and collecting trajectories. The generated trajectories are fed into a selector to generate pseudo-behavioral labels. The trajectories generated by the generator and selector serve as pseudo-data, while the expert trajectories serve as real data. The discriminator scores both types of data and generates a reward, which serves as optimization feedback for the selector and generator. The discriminator network is updated via gradient ascent, the selector network via gradient descent, and the policy network and value network are updated using Proximal Policy Optimization (PPO).

[0108] When the discriminator can no longer effectively distinguish between "expert trajectories" and "generated trajectories" (i.e., the discriminator output approaches 0.5), it indicates that the generated policy has approached expert behavior. At this point, the change in the policy gradient tends to stabilize, and adversarial training has converged. The training of the GAIL-GRU driving policy model is complete, and the trained GAIL-GRU driving policy model is obtained.

[0109] In traffic scenarios, vehicles exhibit various behaviors such as acceleration, deceleration, cruising, and lane changing. These behaviors are controlled by the vehicle's lateral and longitudinal accelerations; therefore, the vehicle's behavior can be considered as determined by the initial state s0 and the vehicle's longitudinal acceleration a. x lateral acceleration a y The continuous space consists of the vehicle's motion time t. By dividing the continuous time into fixed time intervals Δt, the continuous time is discretized. The vehicle's state is updated at each time step, and the vehicle trajectory is generated through continuous time-step updates. The calculation formula is as follows:

[0110]

[0111] In the formula, k is the time step and Δt is the step size.

[0112] This invention incorporates a gated loop unit (GRU) into the generator of Triple-GAIL, which alleviates the displacement deviation problem caused by acceleration integration by capturing temporal dependencies and reducing the cumulative error of discrete data.

[0113] The vehicle's directional acceleration is updated using GRU based on the vehicle's historical state information. x ,a y The working process of the policy network after adding GRU is as follows: First, the vehicle state is initialized to s0, the behavior label is initialized to c0, and the hidden state h0 is set to a zero tensor; second, the policy network inputs the current state s t Current behavior tag c t Compared to the hidden state h at the previous moment t―1 Output the updated hidden state h t Action a t With GRU output features o t Then, the environment reacts according to action a. t Update the state of the next time step. t+1 ;

[0114] Finally, repeat steps two and three for T time steps to output the complete vehicle trajectory τ. c =(s0,a0,s1,a1,…s T ,a T ).

[0115] Step S5: Deploy the environmental vehicle controlled by the driving strategy model and the autonomous vehicle controlled by the autonomous driving model in a static environment.

[0116] Follow these steps to build the simulation test scenario. Figure 5 A schematic diagram of a simulation scene construction method based on GAIL-GRU vehicle trajectory generation provided in an embodiment of the present invention includes the following processing steps:

[0117] Step 1: Start the simulation, select the control strategy for each group of vehicles. The environmental vehicles in the scenario are controlled by the GAIL-GRU driving strategy model, and the test vehicles are controlled by the autonomous driving algorithm under test.

[0118] Step 2: Sampling of initial scene states: The simulation environment randomly samples a set of initial scene states of vehicles from the constructed expert trajectories.

[0119] Step 3: Feature extraction and strategy input: The simulation environment extracts a set of features for each vehicle from the expert trajectory, as shown in Table 1, and inputs them into the GAIL-GRU driving strategy model and the autonomous driving algorithm.

[0120] Step 4: Strategy Execution: The selector of the GAIL-GRU driving strategy outputs behavior labels, the generator generates lateral and longitudinal accelerations based on the state-label pair, and the autonomous driving algorithm generates lateral and longitudinal accelerations based on the state.

[0121] Step 5: Strategy Output and State Update: The driving strategy model and autonomous driving algorithm output the longitudinal and lateral acceleration values ​​of the vehicle as the vehicle's actions, and the simulation environment updates the position and speed of all vehicles in the environment.

[0122] Repeat the process: Repeat steps three and four at a frequency of 25Hz until a collision occurs or 250 time steps are reached, establishing a simulation scenario containing a driving process of up to 10 seconds. This simulation scenario can test the performance of autonomous driving algorithms.

[0123] In summary, the anthropomorphic autonomous driving simulation test scenario construction method provided by the embodiments of the present invention has the corresponding functional modules and beneficial effects of the execution method.

[0124] The method of this invention has the following beneficial effects: By constructing simulation test scenarios with highly human-like and complex interactive characteristics, it can effectively induce and expose the decision-making defects of autonomous driving algorithms in extreme or high-risk traffic situations, improving the pertinence and depth of the test; the platform supports multi-agent behavior modeling, which can simulate the dynamic reactions of real traffic participants and enhance the realism of the simulation environment; at the same time, it has a high risk scenario coverage capability, and can generate typical dangerous working conditions such as emergency braking, cutting in, and lane changing, meeting the testing needs of high-level autonomous driving systems for extreme situations; in addition, the platform supports the integrated application of strategy training and evaluation, and can be used as a training environment for imitation learning and reinforcement learning, forming a complete testing process from strategy optimization to closed-loop verification, with good versatility, scalability and engineering application value.

[0125] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0126] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0127] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0128] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for constructing an anthropomorphic autonomous driving simulation test scenario, characterized in that, include: All vehicles in the traffic scenario are divided into two types of intelligent agents: test vehicles and environment vehicles. The traffic scenario is modeled as a Markov decision process. The test vehicles are vehicles controlled by the autonomous driving algorithm, and the environment vehicles are vehicles controlled by the GAIL-GRU driving strategy model. We use Markov decision processes to extract driving trajectories from human driving datasets and then use these driving trajectories to generate expert trajectory datasets. Construct a GAIL-GRU driving strategy model, and train the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model. The environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model are deployed in a static environment. The interaction between the environmental vehicle and the test vehicle is realized using the trained GAIL-GRU driving strategy model to construct a simulation test scenario.

2. The method according to claim 1, characterized in that, The method of classifying all vehicles in a traffic scenario into two categories of intelligent agents—test vehicles and environment vehicles—and modeling the traffic scenario as a Markov decision process includes: All vehicles in the traffic scenario are divided into two types of intelligent agents: test vehicles and environment vehicles. Test vehicles are vehicles controlled by autonomous driving algorithms, while environment vehicles are vehicles controlled by the GAIL-GRU driving strategy model. In a traffic scenario, each vehicle aims to maximize the reward during the driving process, namely driving efficiency and safety. This invention models the traffic scenario as a Markov decision process, which is represented by tuples (S,A1,A2,π1,π2,T,R1,R2). State set: S represents the set of states of all vehicles in the current traffic scenario, including vehicle speed and position; Action set: A1 represents the set of actions of the test vehicle in the traffic scene, and A2 represents the set of environmental vehicle states in the traffic scene. The lateral and longitudinal acceleration of the vehicle are used as the vehicle's actions. Policy set: π1 represents the policy set of the test vehicle, π2 represents the policy set of the environment vehicles, and the policy function determines the action selected by the vehicle in the current state; Transition probability function: T represents the state transition model that calculates the next state based on the current state and action, and determines the probability of transitioning to the next state given the current state and action; Reward functions: R1 and R2 represent the reward functions of the test vehicle and the environment vehicle, respectively. R1(s,a) represents the immediate reward obtained by the test vehicle after it performs action a from state s.

3. The method according to claim 2, characterized in that, The method of extracting driving trajectories from human driving datasets using Markov decision processes and generating expert trajectory datasets from these driving trajectories includes: This study utilizes Markov decision processes to extract driving trajectories from human driving datasets. Tuples (S, A1, A2, π1, π2, T, R1, R2) are used to describe the interaction process in traffic scenarios. Vehicle driving trajectories and multi-dimensional features are extracted. Multiple agents take actions A1 and A2 in state S and make decisions π1 and π2 according to the policy. They then transition to the next state through the transition function T and obtain rewards R1 and R2. Based on vehicle behavior, driving trajectories are divided into straight-ahead, left lane change, and right lane change, and labels are assigned to construct an expert trajectory dataset. The state characteristics of each vehicle are defined as shown in Table 1: Table 1 4. The method according to claim 3, characterized in that, The construction of the GAIL-GRU driving strategy model, which involves training the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model, includes: Construct a GAIL-GRU driving strategy model. This Triple-GAIL driving strategy model includes a generator, a selector, and a discriminator. The generator contains a policy network π. θ Value Network V φ The selector C is parameterized by α. α Vehicle pseudo-behavior labels are generated based on the scene state and then input into the generator π, which is parameterized by θ. θ In the process, the generator generates vehicle actions based on the scene state and pseudo-behavior labels, and finally, the discriminator D, parameterized by ψ, generates the actions. ψ To determine whether the state-label-action pair originates from an expert trajectory, the selector and generator work together against the discriminator, forming a game process that minimizes α and θ and maximizes ψ. The calculation formula is as follows: Policy Network π θ The input data includes behavior labels c t and vehicle status s t The output data is the vehicle's acceleration action a. t For the policy network π θ A value function based on state-label-action is adopted. (c t ,s t ,a t )Evaluate the current behavior label c t and vehicle status s t Next, execute action a t The expected return, which is estimated by the value network; A vehicle trajectory discriminator is established. This discriminator maps state-label-action pairs to scores to distinguish between expert trajectories and trajectories generated by driving strategies. In the update step, the discriminator D... ψ The discriminator scores the generated state-label-action pairs and the state-label-action pairs from the expert trajectory, with the goal of scoring the input from policy π. θ The state-label-action pair outputs a low score, with input from expert trajectory π. E The state-label-action pair outputs a high score, and the loss function and gradient update of the discriminator with parameter ψ are shown in the following equation: In the formula L discrim The cross-entropy loss of the discriminator, This represents the expected value of the expert data distribution. D represents the expectation D on the data distribution of the agent. ψ (s,a,c) represents the probability that the data (s,a,c) belongs to expert data, ω represents the weighting coefficient, and log(1―D) ψ (s,a,c)) represents the negative example loss of the generated data, logD ψ (s,a,c) represents the positive example loss: In the formula N is the gradient of the discriminator loss. e The sample size of the expert data. Let N represent the number of samples generated, and T represent the number of samples generated. j This represents the length of the j-th trajectory. ω represents the generated sample, and ω represents the weight coefficient.

5. The method according to claim 4, characterized in that, The process of training the GAIL-GRU driving strategy model using the expert trajectory dataset to obtain the trained GAIL-GRU driving strategy model includes: The sliding time window method was used to extract 500 straight-line trajectories, 500 left lane-change trajectories, and 500 right lane-change trajectories from the expert trajectory dataset. The offset trajectories were divided into time series of length k as training data for the GAIL-GRU driving strategy model. The GAIL-GRU driving strategy model was trained using the training data. The environment is reset using time-series data with behavioral labels. Each time-series data has a fixed real behavioral label. The agent runs a policy based on the real behavioral label, generates and collects trajectories. The generated trajectories are fed into the selector to generate pseudo-behavioral labels. The trajectories generated by the generator and selector are used as pseudo data, and the expert trajectories are used as real data. The discriminator scores the two types of data and generates a reward as optimization feedback for the selector and generator. The discriminator network is updated through gradient ascent, the selector network is updated through gradient descent, and the policy network and value network are updated using the proximal policy optimization algorithm. When the discriminator can no longer effectively distinguish between expert trajectories and generated trajectories, that is, when the discriminator output approaches 0.5, the adversarial training has converged, the training of the GAIL-GRU driving strategy model ends, and the trained GAIL-GRU driving strategy model is obtained.

6. The method according to claim 5, characterized in that, The process of deploying the environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model in a static environment, and using the trained GAIL-GRU driving strategy model to realize the interaction between the environmental vehicle and the test vehicle to construct a simulation test scenario includes: The environmental vehicle controlled by the trained GAIL-GRU driving strategy model and the test vehicle controlled by the autonomous driving model are deployed in a static environment. The simulation is started, and the control strategy for each group of vehicles is selected. The environmental vehicle in the scenario is controlled by the GAIL-GRU driving strategy model, and the test vehicle is controlled by the autonomous driving algorithm under test. The initial scene state of a group of vehicles is randomly sampled from the constructed expert trajectory as the simulation environment. The simulation environment extracts a set of state features for each vehicle from the expert trajectory dataset as shown in Table 1, and inputs the state features into the GAIL-GRU driving strategy model and the autonomous driving algorithm. The selector of the GAIL-GRU driving strategy model outputs behavior labels, the generator generates lateral and longitudinal accelerations based on the state-label pair, and the autonomous driving algorithm generates lateral and longitudinal accelerations based on the state. The GAIL-GRU driving strategy model and autonomous driving algorithm output the longitudinal and lateral acceleration values ​​of the vehicle as the vehicle's actions, and the simulation environment updates the position and velocity of all vehicles in the environment. The above processing procedure is repeated at a frequency of 25Hz until a collision occurs or 250 time steps are reached, thus establishing a simulation test scenario containing a driving process of up to 10 seconds. The performance of the autonomous vehicle is then tested using the simulation test scenario.

Images