A humanized virtual test scene multi-background vehicle cooperative decision-making method

Abstract

The present application provides a kind of anthropomorphic virtual test scene multi-background car cooperative decision-making method, method includes the following steps: based on graph structure, the dynamic traffic graph of multi-vehicle is built, the generation of antithetical imitation learning framework of parameter sharing is built, the node feature after optimization of graph attention network is input to reinforcement learning network to generate serialized control instruction, the state-action pair of expert data is compared with the control instruction generated using the discriminator of shared scoring parameter, to iteratively optimize control instruction generator;Introduce latent factor variable to represent driving style, combine explicit rule penalty and implicit adversarial reward, enhance strategy anthropomorphism;Through multiple iterations, highly anthropomorphic multi-background car sequence cooperative control instruction is generated.According to the present application, latent factor variable and reward reinforcement imitation learning are used to iteratively optimize generative adversarial imitation learning, to generate decisions with diversity, compliance and anthropomorphism, thereby improving the reliability of autonomous driving simulation test from multiple perspectives.

Description

Technical Field

[0001] This invention relates to the field of autonomous vehicle testing technology, and more specifically to a human-like virtual testing scenario multi-background vehicle collaborative decision-making method. Background Technology

[0002] In autonomous driving simulation testing, the realism of background vehicle behavior is a core element in constructing highly credible virtual scenarios, directly determining the effectiveness of the tested algorithm in complex traffic situations. Currently, collaborative decision-making among background vehicles mainly relies on rule-based pre-set models and data-driven reinforcement learning methods. Rule-based pre-set models simulate vehicle driving behavior through mathematical formulas and fixed thresholds, achieving simple lane keeping and steady-state following, but their essence is a static response mechanism based on deterministic logic, leading to highly predictable and mechanically repetitive behavior. Data-driven reinforcement learning methods optimize decisions through a large amount of real driving data, but their training process is severely limited by the high-dimensional state space modeling of multi-vehicle interaction scenarios and requires precisely labeled driving behavior to distinguish individual intentions, making the model susceptible to data noise interference and unable to cover the complex interaction patterns in real road conditions. In general, existing methods struggle to balance the generation efficiency and realism requirements of virtual test scenarios. Therefore, this paper proposes a human-like collaborative decision-making method for multiple background vehicles in virtual test scenarios to address these issues. Summary of the Invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of existing methods in balancing the generation efficiency and realism requirements of virtual test scenarios, thereby providing a human-like virtual test scenario multi-background vehicle collaborative decision-making method.

[0004] To address the aforementioned problems, this invention provides a human-like virtual testing scenario multi-background vehicle collaborative decision-making method, comprising the following steps:

[0005] S1: Construct a dynamic traffic map for multiple vehicles based on a graph structure;

[0006] S2: Build a parameter-sharing generative adversarial imitation learning framework. Input the node features optimized by the graph attention network into the reinforcement learning network to generate sequential control instructions. Use a discriminator with shared scoring parameters to compare the generated control instructions with the state-action pairs of expert data to iteratively optimize the control instruction generator.

[0007] S3: Introduce latent factor variables to represent driving style, and combine explicit rule penalties with implicit adversarial rewards to enhance the anthropomorphism of the strategy;

[0008] S4: Generate highly anthropomorphic multi-background vehicle sequence collaborative control instructions through multiple rounds of iteration.

[0009] Preferably, the dynamic traffic map for multiple vehicles constructed based on the graph structure is as follows:

[0010] The original traffic data was cleaned using an extended Kalman filter to generate a structured expert dataset; node features were extracted and driving styles were clustered based on headway; and a dynamic traffic map was constructed by calculating edge weights through interaction relationships using vehicles as nodes.

[0011] Preferably, the dynamic traffic map of the multiple vehicles is represented as follows:

[0012] G t =(N t E t )

[0013]

[0014] Among them, G t Represents spatial interaction characteristics, N t E represents the set of features of all vehicle nodes at the current moment, including each vehicle's longitudinal position X, speed V, lane class L, forward lane distance H, and backward lane distance T; t This represents the set of interaction relationships between multiple vehicles at the current moment.

[0015] Preferably, in S2, the generative adversarial imitation learning framework includes: a generator and a discriminator;

[0016] The generator employs a graph attention network with shared parameters, captures the dependencies between vehicle nodes through a hierarchical attention mechanism, and then generates multi-background vehicle collaborative decisions through a reinforcement learning network.

[0017] Preferably, the discriminator employs a loss function with shared scoring parameters to evaluate the degree of matching between the generated decision and expert data, and iteratively optimizes the generator through adversarial training.

[0018] Preferably, the discriminator uses a loss function with shared scoring parameters: In the discriminator, the state-action pairs generated by the generator are input into the loss function with shared scoring parameters to evaluate the rationality of the multi-vehicle cooperative decision generated by the reinforcement learning network. The loss function is as follows:

[0019]

[0020] Where M is the number of background vehicles, and D ψ It is a discriminator, A t It is the background vehicle decision generated by the reinforcement learning network, (G t A t ) is a state-action pair extracted from an expert dataset. These are state-action pairs generated by the generator;

[0021] When L D If the value is below the set threshold, it is considered that the serialized control commands generated by the generator have reached a driving level comparable to that of the expert dataset. At this point, the final generated serialized control commands are output to drive the background vehicle in the simulation environment.

[0022] Preferably, the latent factor variables characterize driving style as follows:

[0023] Driving styles are extracted from the expert dataset and defined as a hybrid driving style set. Latent variables are matched with the optimal driving style from the hybrid driving style set according to the prior distribution. Then, the latent variables are concatenated with vehicle graph nodes and embedded into the state features of the background vehicle to generate background vehicle control commands with different driving styles.

[0024] Preferably, in S3, the explicit rule penalty is:

[0025] Rewards are based on imitation learning to enhance the compliance of decision-making. By measuring the similarity between the trajectory generated by the current strategy and the trajectory demonstrated by experts, a reward value is given when the loss function value shows a downward trend during the iterative process, indicating that the current training process is in a positive optimization process; otherwise, a penalty value is given.

[0026] Preferably, in S3, the implicit adversarial reward is: guiding the strategy to learn safer driving behaviors by punishing non-compliant behaviors;

[0027] The non-compliant behaviors include: collisions, lane departures, and hard braking.

[0028] Preferably, in step S4, the highly anthropomorphic discriminator generated through multiple iterations is updated as follows:

[0029]

[0030] in, The control commands are generated based on the splicing nodes, c represents the vehicle driving style in the expert dataset, and R is the trajectory similarity reward function. These are the penalty values ​​for three types of non-compliant behaviors.

[0031] The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method provided by this invention has the following beneficial effects:

[0032] This invention realizes a human-like collaborative decision-making method for multiple background vehicles in autonomous driving virtual simulation testing; it utilizes graph structures to achieve more accurate modeling of complex interaction relationships between multiple vehicles, and uses generative adversarial learning with a fusion parameter sharing mechanism to achieve rapid training of multi-background vehicle collaboration; it uses latent factor variables and reward-enhanced imitation learning to iteratively optimize the control command generator, generating decisions that are diverse, compliant, and human-like, thereby improving the reliability of autonomous driving simulation testing in multiple ways. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the dynamic traffic map for multiple vehicles based on a graph structure according to the present invention;

[0034] Figure 2 This is a schematic diagram of the multi-background vehicle anthropomorphic collaborative decision-making framework of the present invention;

[0035] Figure 3 This is a schematic diagram comparing the RMSE of the present invention with that of other models.

[0036] Figure 4 This is a schematic diagram comparing the RMSE of lane departure with other models of this invention;

[0037] Figure 5 This is a schematic diagram comparing the RMSE of the speed of this invention with other models. Detailed Implementation

[0038] like Figure 1-5 As shown, the present invention provides a human-like virtual test scenario multi-background vehicle collaborative decision-making method, which includes the following steps:

[0039] S1: Construct a dynamic traffic map for multiple vehicles based on a graph structure;

[0040] S2: Build a parameter-sharing generative adversarial imitation learning framework. Input the node features optimized by the graph attention network into the reinforcement learning network to generate sequential control instructions. Use a discriminator with shared scoring parameters to compare the generated control instructions with the state-action pairs of expert data to iteratively optimize the control instruction generator.

[0041] S3: Introduce latent factor variables to represent driving style, and combine explicit rule penalties with implicit adversarial rewards to enhance the anthropomorphism of the strategy;

[0042] S4: Generate highly anthropomorphic multi-background vehicle sequence collaborative control instructions through multiple rounds of iteration.

[0043] Specifically, this application achieves highly human-like multi-background vehicle collaborative decision-making by integrating real driving data with generative adversarial imitation learning (GAIL). First, the original traffic dataset is cleaned using methods such as extended Kalman filtering to form an expert dataset. Node features including vehicle speed and lanes are extracted from the expert dataset, and a driving style extraction method based on headway is introduced to cluster the driving styles of vehicles in the dataset. Then, a graph structure is used, with each vehicle as a node, and edge weights are calculated to form a multi-vehicle dynamic traffic map. A parameter-sharing generative adversarial imitation learning framework is built. First, a parameter-sharing graph attention network is used to enhance the node features in the dynamic traffic map. Then, the node features optimized by the graph attention network are input into a reinforcement learning network to generate sequential control commands. The control commands and node states are combined to form state-action pairs. Finally, a discriminator with shared scoring parameters is used to compare the generated state-action pairs with those in the expert dataset, iteratively optimizing the control command generator. Simultaneously, to obtain more realistic anthropomorphic decisions, anthropomorphism can be represented by driver behavior in the expert dataset; the closer the behavior, the higher the anthropomorphism. Furthermore, the introduction of driving style makes the resulting driving decisions more anthropomorphic, while explicit rule penalties prevent collisions and boundary violations. This invention utilizes latent factor variables to represent driving style to enhance policy learning, and uses reward imitation learning to fuse explicit rule penalties (collision, boundary violations, emergency braking) with implicit adversarial rewards. Finally, the latent factor variables, reward imitation learning, and generative adversarial imitation learning are combined and iteratively optimized to form highly anthropomorphic multi-background vehicle sequential control commands.

[0044] Specifically, this application implements a multi-background vehicle anthropomorphic collaborative decision-making method in autonomous driving virtual simulation testing; it utilizes graph structures to achieve more accurate modeling of complex interaction relationships between multiple vehicles, and uses generative adversarial learning with a fusion parameter sharing mechanism to achieve rapid training of multi-background vehicle collaboration; it uses latent factor variables and reward-enhanced imitation learning to iteratively optimize the control command generator, generating decisions that are diverse, compliant, and anthropomorphic, thereby improving the reliability of autonomous driving simulation testing in multiple ways.

[0045] This application uses the root mean square error (RMSE) to measure the error between the true trajectory in the dataset and the trajectory generated using this method. Specifically, as follows: Figure 3-5 As shown, by comparing the RMSE between the position, speed, and lane departure of the vehicle driven by the original human and the vehicle after changing the decision-making method, and by comparing the method of the present invention with baseline models such as the static Gaussian model (SG) and the behavioral clonal model (BC), from... Figure 3It is evident that the RMSE of the static Gaussian model and the behavioral cloning model increases significantly with time, while the model of this invention consistently maintains the lowest RMSE. This indicates that the model of this invention has a smaller error in predicting vehicle positions and is superior to the other two models. Figure 4 The results show that as time increases, the RMSE of both the static Gaussian model and the behavioral clone model exceeds 1.2m, while the model of this invention remains stable at around 0.7m. This indicates that the model of this invention has better control over lane line deviation in lane keeping tasks, can more accurately maintain the vehicle's position within the lane, and reduce deviation errors. Figure 5 The results show that the static Gaussian model has the highest RMSE (3.5 m / s when the variable is 8), followed by the behavioral cloning model (2.5 m / s), and the model of this invention has the lowest RMSE (approximately 1.8 m / s). This indicates that the model of this invention is closer to the true value in speed prediction or control, with smaller errors and more accurate speed control.

[0046] Specifically, this application models the complex interaction relationships between multiple vehicles using a graph structure, effectively capturing the spatial interaction features between vehicles. Compared with traditional rule-based preset models, the graph structure can dynamically adjust the interaction weights between vehicles, thus more realistically reflecting the complex interaction patterns in real traffic scenarios. Furthermore, by incorporating generative adversarial learning with a parameter-sharing mechanism, this invention can significantly shorten the training time for collaborative decision-making among multiple vehicles. The parameter-sharing mechanism enables the model to quickly expand from single-agent learning to multi-agent learning, thereby significantly improving training efficiency while ensuring the realism of simulation testing.

[0047] Specifically, this application generates driving behaviors that are diverse, compliant, and human-like by decoupling latent factors and using reward-enhanced imitation learning. Latent factor variables are matched with driving styles, such as aggressive and conservative, from expert datasets and embedded into the decision model of the background vehicle, thereby generating background vehicle action states with different driving styles. The threshold for driving style is not fixed; the driving style extraction method based on headway distance categorizes driving styles into three types (aggressive, normal, and conservative) according to the clustering of lane headway distances in the data, which in this invention (0s~1s, 1s~2s, 2s~infinity) correspond to aggressive, normal, and conservative, respectively. Reward-enhanced imitation learning further optimizes the generated driving behavior by introducing explicit rule penalties (such as collision, boundary crossing, and emergency braking) and implicit adversarial rewards, making it closer to real driving scenarios.

[0048] Specifically, in the construction of a dynamic traffic map for multiple vehicles, the traffic dataset collected in the real scene is first preprocessed to obtain an expert dataset. Based on this, a graph structure is used to represent the interaction relationship between multiple vehicles, thereby obtaining the spatial interaction features between vehicles.

[0049] Specifically, the construction of a multi-vehicle dynamic traffic map based on a graph structure involves: cleaning the original traffic data using an extended Kalman filter to generate a structured expert dataset; extracting node features such as vehicle speed and lanes, and clustering driving styles based on headway; using vehicles as nodes, calculating edge weights through interaction relationships to construct the dynamic traffic map. This process involves using an extended Kalman filter to clean the original traffic data, generating a structured expert dataset, extracting node features, clustering driving styles based on headway, and then using vehicles as nodes to calculate edge weights through interaction relationships to construct the dynamic traffic map. This application first uses an extended Kalman filter to eliminate noise in the original data, and then uses a traffic rule-based data cleaning method to remove non-compliant data, forming an expert dataset. It introduces a driving style extraction method based on headway, extracting different driving styles from the expert dataset as influencing factors for anthropomorphic decision-making. Then, it extracts node features such as vehicle speed, vehicle position, and vehicle heading angle from the expert dataset, and uses a graph structure to calculate edge weights between nodes, representing the interaction relationships between vehicle nodes, thus constructing a multi-vehicle dynamic traffic map as follows:

[0050] G t =(N t E t )

[0051]

[0052] Among them, G t Represents spatial interaction characteristics, N t E represents the set of features of all vehicle nodes at the current moment, including each vehicle's longitudinal position X, speed V, lane class L, forward lane distance H, and backward lane distance T; t This represents the set of interaction relationships between multiple vehicles at the current moment.

[0053] Specifically, such as Figure 2 As shown, the generative adversarial imitation learning framework in this application includes: a generator and a discriminator; the generator employs a graph attention network with shared parameters, captures the dependencies between vehicle nodes through a hierarchical attention mechanism, and then generates multi-background vehicle collaborative decisions through a reinforcement learning network; the discriminator employs a loss function with shared scoring parameters to evaluate the matching degree between the generated decisions and expert data, and iteratively optimizes the generator through adversarial training; wherein, this invention first bases on spatial interaction features G t =(N t E t The joint action set A, consisting of all vehicle driving actions (acceleration α, steering angle θ, etc.) in the expert dataset and the data itself, is formed by... t Constructing state-action pairs (G) in generative adversarial imitation learning t A t ).

[0054] Specifically, unlike the common application of generative adversarial imitation learning in multi-vehicle collaborative decision-making, the background vehicles in the virtual test scenario only serve as part of the interaction information. This eliminates the need for massive training time or datasets, but rather requires faster training speeds to more efficiently build the virtual test scenario. This application employs a parameter sharing mechanism, enabling the network model to quickly expand from single-agent learning to multi-agent learning. Specifically, in the generator, the local observations of agent i (including its own state, neighbor states, and lane information) are input into a shared graph attention network to quickly obtain the interaction relationships between background vehicles. The multi-vehicle interaction relationships are shown below:

[0055]

[0056] in, σ is the output of the graph attention network at layer l for vehicle i; σ is the activation function. It is a degree matrix, representing the degree of each vertex in the graph; It is an adjacency matrix, representing the relationships between nodes; W represents the original node features. ( l) represents the learning parameters shared by layer l.

[0057] Specifically, the node features processed by the graph attention network are input into the reinforcement learning network to generate control decisions (acceleration α and heading angle θ) for the background vehicle, as shown below:

[0058]

[0059] in, It is the output of the graph attention network at layer l of vehicle i. It is a reinforcement learning generative network. This indicates the vehicle's decision-making actions after sampling.

[0060] Specifically, to increase the robustness of decision-making, this application assigns the action state output by the action prediction network a Gaussian distribution and samples the specific action from the generated probability distribution. The sampled action Applied to the vehicle dynamics model, the vehicle's state is updated as follows:

[0061]

[0062] Among them, G i,t+1 Here, f represents the state of background car i at time t+1, and f is the vehicle dynamic model. It is the set of states of all background vehicles.

[0063] Specifically, in the discriminator, the state-action pairs generated by the generator are input into a loss function with shared scoring parameters to evaluate the rationality of the reinforcement learning network's multi-vehicle cooperative decision-making. The loss function is shown below:

[0064]

[0065] Where M is the number of background cars, D ψ It is a discriminator, (G t A t ) is a state-action pair extracted from an expert dataset. These are state-action pairs generated by the generator.

[0066] Among them, when L D If the value is below the set threshold, it is considered that the serialized control commands generated by the generator have reached a driving level comparable to that of the expert dataset. At this point, the final generated serialized control commands are output to drive the background vehicle in the simulation environment.

[0067] Specifically, in existing autonomous driving simulation tests, the interaction between the tested vehicle and the background vehicle differs significantly from real-world interaction. This is primarily because the complexity of the background vehicle's decision-making, i.e., its anthropomorphic decision-making that aligns with real driver behavior, is not adequately considered when constructing the virtual test scenario. In contrast, this application sets a potential factor variable z to represent the driving style obtained in S1, and then modifies this variable with the current node state G. t By splicing together these elements to form an enhanced state, a collaborative strategy is guided to generate more human-like driving behavior. First, this application extracts driving styles from the expert dataset, defining them as a mixed driving style set. Latent variables are then matched with the optimal driving style from the mixed driving style set based on their prior distribution. These variables are then fused with vehicle graph nodes and embedded into the state features of the background vehicle, thereby generating control commands for background vehicles with different driving styles, as shown below:

[0068] G0~ρ0

[0069] z~p(z)

[0070] π~p(π|z)

[0071]

[0072] Where G0 is the initial state of the background vehicle, z is a discrete latent variable used to select a specific driving style, p(z) is the prior distribution of the latent variable, and π is the driving style selected from the mixed driving style set.

[0073] This is the state of the spliced ​​nodes. A is the output of a graph attention network based on splicing nodes.t+1 It is a next-state control instruction generated based on the enhanced node characteristics.

[0074] Specifically, in order to provide knowledge optimization strategies for the transportation field, this application introduces reward-based imitation learning to enhance the compliance of decision-making. The reward mechanism of this invention consists of two parts. The first part is a reward based on imitation learning (explicit rule penalty), which is used to measure the similarity between the trajectory generated by the current strategy and the trajectory demonstrated by the expert. When the loss function value shows a downward trend during the iterative process, it indicates that the current training process is a positive optimization process, and a reward value is given; otherwise, a penalty value is given.

[0075] The second part is domain knowledge-based rewards (implicit adversarial rewards), which guide the strategy to learn safer driving behaviors by punishing non-compliant behaviors;

[0076] This application penalizes three types of non-compliant behavior: collision, calculated by determining the minimum distance d between the vehicle and other vehicles on the road. c When the distance is less than a certain threshold, a penalty is applied; road deviation is determined by calculating the distance d between the vehicle and the road edge. road When d road When the acceleration is less than a certain threshold, a penalty is applied; hard braking is applied when the vehicle's acceleration 'a' is less than a certain threshold.

[0077] Specifically, the highly anthropomorphic discriminant, generated through multiple iterations, is updated as follows:

[0078]

[0079] in, The control commands are generated based on the splicing nodes, c represents the vehicle driving style in the expert dataset, and R is the trajectory similarity reward function. These are the penalty values ​​for three types of non-compliant behaviors.

[0080] By introducing driving style and reward-based imitation learning, this invention ultimately generates multi-background vehicle serial control instructions that satisfy anthropomorphism.

[0081] In some implementations, the NGSIM dataset can be used as the expert dataset in this application. First, the original trajectory data is smoothed using an extended Kalman filter, and non-compliant behaviors are eliminated using hard filtering based on traffic rules. Then, the vehicle position and speed are transformed to the Frenet coordinate system, and features such as the relative coordinates of the lane centerline, longitudinal speed, and lane category are extracted. Using a driving style extraction method based on headway, different driving styles are extracted from the expert dataset. In this embodiment, the extracted driving styles are normal, aggressive, and conservative. Then, a dynamic traffic map is constructed using a graph structure. The node features include the vehicle's longitudinal position, speed, lane category, and distance to the vehicle in front and behind. The edge weights are calculated based on the relative distance between vehicles. Based on the dynamic traffic map, a generative adversarial learning generator is constructed, using a 3-layer graph annotation. The intentional network, with each layer having a feature dimension of 128, aggregates neighbor vehicle information to obtain a feature-enhanced dynamic traffic map. This enhanced dynamic traffic map is then input into a reinforcement learning network to obtain the background vehicle's action control commands, including Gaussian distribution parameters for acceleration and steering angle. After sampling, the state is updated using a vehicle dynamics model. The generated state-action pairs are compared with those obtained from the expert dataset, including comparisons of driving styles. Penalties are also applied for collisions, lane departures, and hard braking. A collision penalty of P is applied when the background vehicle collides with another vehicle; a lane departure penalty of 3 / 4P is applied when the difference between the background vehicle's geometric center and the lane edge is less than 0.2m; and a penalty is applied when the background vehicle's acceleration is less than -4m / s². 2 If the background vehicle is considered to have engaged in hard braking, a hard braking penalty of 1 / 2P is imposed. The above reward and penalty functions are combined, the adversarial loss function is calculated, and the generator is iteratively optimized. Finally, when the loss function value is approximately 0, the generator outputs highly anthropomorphic multi-background vehicle serialized control commands.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention. The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the protection scope of the present invention.

Claims

1. A humanized virtual test scene multi-background vehicle cooperative decision-making method, characterized in that, Includes the following steps: S1: Construct a dynamic traffic map for multiple vehicles based on a graph structure; S2: Build a parameter-sharing generative adversarial imitation learning framework. Input the node features optimized by the graph attention network into the reinforcement learning network to generate sequential control instructions. Use a discriminator with shared scoring parameters to compare the generated control instructions with the state-action pairs of expert data to iteratively optimize the control instruction generator. S3: Introduce latent factor variables to represent driving style, and combine explicit rule penalties with implicit adversarial rewards to enhance the anthropomorphism of the strategy; S4: Generate highly anthropomorphic multi-background vehicle sequence collaborative control commands through multiple rounds of iteration; The dynamic traffic map for multiple vehicles constructed based on the graph structure is as follows: The original traffic data was cleaned using an extended Kalman filter to generate a structured expert dataset; node features were extracted and driving styles were clustered based on headway; and a dynamic traffic map was constructed using vehicles as nodes and edge weights calculated through interaction relationships. The latent factor variables characterize driving style as follows: Driving styles are extracted from the expert dataset and defined as a hybrid driving style set. Latent variables are matched with the best driving style from the hybrid driving style set according to the prior distribution. Then, the latent variables are concatenated with vehicle graph nodes and embedded into the state features of the background vehicle to generate background vehicle control commands with different driving styles. In S3, the explicit rule penalty is: Rewards are based on imitation learning to enhance the compliance of decision-making. By measuring the similarity between the trajectory generated by the current strategy and the trajectory demonstrated by experts, when the loss function value shows a downward trend during the iterative process, it indicates that the current training process is in a positive optimization process, and a reward value is given; otherwise, a penalty value is given. In S3, the implicit adversarial reward is: to guide the strategy to learn safer driving behaviors by punishing non-compliant behaviors; The non-compliant behaviors include: collisions, lane departures, and hard braking.

2. The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method according to claim 1, characterized in that: The dynamic traffic map of the multiple vehicles is represented as follows: ； ； ； wherein, represents spatial interaction features, a set of all vehicle node features at the current time, including the longitudinal position , speed , lane category , lane forward distance , and lane backward distance of each vehicle; represents a set of interaction relationships between multiple vehicles at the current time.

3. The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method according to claim 1, characterized in that: In S2, the generative adversarial imitation learning framework includes a generator and a discriminator; The generator employs a graph attention network with shared parameters, captures the dependencies between vehicle nodes through a hierarchical attention mechanism, and then generates multi-background vehicle collaborative decisions through a reinforcement learning network.

4. The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method according to claim 3, characterized in that: The discriminator employs a loss function with shared scoring parameters to evaluate the degree of matching between the generated decision and expert data, and iteratively optimizes the generator through adversarial training.

5. The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method according to claim 4, characterized in that: The discriminator uses a loss function with shared scoring parameters: In the discriminator, the state-action pairs generated by the generator are input into the loss function with shared scoring parameters to evaluate the rationality of the multi-vehicle cooperative decision generated by the reinforcement learning network. The loss function is as follows: ； wherein is the number of background cars, is the discriminator, is the background car decision generated by the reinforcement learning network, is the state-action pairs extracted from the expert dataset, is the state-action pairs generated by the generator; When If the threshold is set low, then the generated sequence of control commands is considered to have reached a driving level comparable to the expert dataset, and the final generated sequence of control commands is output to drive the background vehicles in the simulation environment.

6. The anthropomorphic virtual test scenario multi-background vehicle collaborative decision-making method according to claim 1, characterized in that: In step S4, the highly anthropomorphic discriminator generated through multiple iterations is updated as follows: ； in, It is a control command generated based on splicing nodes. It is the vehicle driving style in the expert dataset. It is a trajectory similarity reward function. These are the penalty values ​​for three types of non-compliant behaviors.

Images