Personified virtual test scene multi-background vehicle collaborative decision-making method
By using graph structure and generative adversarial learning methods, we optimize multi-background vehicle collaborative decision-making in autonomous driving virtual test scenarios, solving the problem of balancing virtual test scenario generation efficiency and realism, and achieving more efficient anthropomorphic vehicle behavior simulation.
Patent Information
- Application Number
- CN202510678323.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-26
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2045-05-26
AI Technical Summary
Existing technologies struggle to balance the efficiency and realism of virtual test scenario generation in autonomous driving simulation testing, making it difficult to achieve anthropomorphic decision-making regarding the behavior of background vehicles.
A graph-based dynamic traffic graph construction method is adopted, which combines generative adversarial learning and latent factor variables. Sequential control instructions are generated through graph attention networks and reinforcement learning. Explicit rule penalties and implicit adversarial rewards are introduced to optimize collaborative decision-making among vehicles in multiple backgrounds.
It achieves more accurate multi-vehicle interaction relationship modeling, improves the reliability and efficiency of autonomous driving simulation testing, and generates driving behavior decisions that are diverse, compliant, and human-like.
Smart Images

Figure CN120805390A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of automatic driving vehicle testing, in particular to a humanized virtual test scene multi-background vehicle cooperative decision-making method. BACKGROUND
[0002] In the automatic driving simulation test, the authenticity of the background vehicle behavior is the core element of building a high credibility virtual scene, which directly determines whether the tested algorithm is effective in complex traffic scenarios. At present, the cooperative decision-making of the background vehicle mainly relies on the preset model based on rules and the reinforcement learning method based on data driving; the preset model based on rules simulates the driving behavior of the vehicle through mathematical formulas and fixed thresholds, although it can realize simple lane keeping and steady following, but its essence is a static response mechanism based on deterministic logic, which leads to high predictability and mechanical repetition of behavior; the core of the reinforcement learning method based on data driving is to optimize the decision through a large amount of real driving data, but its training process is severely limited by the high-dimensional state space modeling of multi-vehicle interaction scenes, and it needs to rely on accurately labeled driving behavior labels to distinguish individual intentions, which leads to the model being easily disturbed by data noise and being difficult to cover complex interaction modes in real road conditions; in general, the existing methods are difficult to balance the generation efficiency and authenticity requirements of the virtual test scene. Therefore, a humanized virtual test scene multi-background vehicle cooperative decision-making method is invented to improve the above problems. SUMMARY
[0003] Therefore, the technical problem to be solved by the present application is to overcome the defects in the prior art that the existing methods are difficult to balance the generation efficiency and authenticity requirements of the virtual test scene, thereby providing a humanized virtual test scene multi-background vehicle cooperative decision-making method.
[0004] In order to solve the above problems, the present application provides a humanized virtual test scene multi-background vehicle cooperative decision-making method, which comprises the following steps:
[0005] S1: constructing a dynamic traffic graph of multiple vehicles based on a graph structure;
[0006] S2: building a generative adversarial imitation learning framework sharing parameters, inputting the node features optimized by the graph attention network into the reinforcement learning network to generate serialized control instructions, and using the discriminator sharing the scoring parameters to compare the generated control instructions with the state-action pairs of the expert data to iteratively optimize the control instruction generator;
[0007] S3: introducing latent factor variables to represent driving styles, combining explicit rule penalties and implicit adversarial rewards to enhance the humanization of the strategy;
[0008] S4: generating highly humanized multi-background vehicle sequence cooperative control instructions through multiple iterations.
[0009] Preferably, the multi-vehicle dynamic traffic graph based on the graph structure is:
[0010] The original traffic data is cleaned using an extended Kalman filter to generate a structured expert dataset; node features are extracted, and driving styles are clustered based on headway; the vehicle is taken as a node, the edge weight is calculated through interaction, and a dynamic traffic graph is constructed.
[0011] Preferably, the multi-vehicle dynamic traffic graph is represented as:
[0012] G t =(N t ,E t )
[0013]
[0014] wherein G t represents spatial interaction features, N t represents a set of all vehicle node features at the current time, including the longitudinal position X, speed V, lane category L, lane forward distance H, and lane backward distance T of each vehicle; E t represents a set of interaction relationships between multiple vehicles at the current time.
[0015] Preferably, in S2, the generative adversarial imitation learning framework comprises a generator and a discriminator.
[0016] The generator adopts a graph attention network with shared parameters, captures the dependency relationship between vehicle nodes through a hierarchical attention mechanism, and generates multi-background vehicle cooperative decision-making through a reinforcement learning network.
[0017] Preferably, the discriminator adopts a loss function with shared scoring parameters, which is used to evaluate the matching degree of the generated decision with the expert data, and iteratively optimizes the generator through adversarial training.
[0018] Preferably, the loss function with shared scoring parameters adopted by the discriminator is: in the discriminator, the state-action pair generated by the generator is input into the loss function with shared scoring parameters to evaluate the rationality of the multi-vehicle cooperative decision generated by the reinforcement learning network, and the loss function is as follows:
[0019]
[0020] wherein M is the number of background vehicles, D ψ is the discriminator, A t is the background vehicle decision generated by the reinforcement learning network, (G t ,A t ) is a state-action pair extracted from the expert dataset, is a state-action pair generated by the generator;
[0021] When L D When the set threshold is lower than the set threshold, it is considered that the sequence control instruction generated by the generator at this time reaches the driving level comparable to the expert data set, and the finally generated sequence control instruction is output at this time to drive the background vehicle in the simulation environment.
[0022] Preferably, the potential factor variable represents the driving style as:
[0023] The driving style definition extracted from the expert data set is defined as a mixed driving style set, and the latent variable is matched with the best driving style from the mixed driving style set according to the prior distribution, and then the latent variable is spliced with the vehicle graph node, and the variable is embedded into the state feature of the background vehicle, thereby generating the background vehicle control instruction with different driving styles.
[0024] Preferably, in S3, the explicit rule penalty is:
[0025] Based on the reward of imitative learning, the compliance of the decision is enhanced, and the similarity between the trajectory generated by the current strategy and the expert demonstration trajectory is measured, and when the loss function value iterative process shows a downward trend, it means that the current training process is a positive optimization process, and a reward value is given; otherwise, a penalty value is given.
[0026] Preferably, in S3, the implicit adversarial reward is:
[0027] The non-compliant behavior includes collision, road deviation and hard braking.
[0028] Preferably, in S4, the multi-round iteration generates a highly personalized discriminator update:
[0029]
[0030] Wherein, is the control instruction generated based on the spliced node, c is the vehicle driving style in the expert data set, R is the trajectory similarity reward function, is the penalty value of the three non-compliant behaviors.
[0031] The humanized virtual test scene multi-background vehicle cooperative decision method provided by the application has the following beneficial effects:
[0032] The application realizes a multi-background vehicle personification cooperative decision-making method in automatic driving virtual simulation testing; a graph structure is used to realize more accurate modeling of complex interaction relationships between multiple vehicles, a generative adversarial imitation learning with a fusion parameter sharing mechanism is used to realize rapid training of multi-background vehicle cooperation; a latent factor variable and reward enhanced imitation learning iterative optimization control instruction generator are used to generate diversified, compliant and personified decisions, thereby improving the reliability of automatic driving simulation testing in multiple directions. BRIEF DESCRIPTION OF DRAWINGS
[0033] Figure 1 A dynamic traffic graph of multiple vehicles is constructed based on a graph structure of the application;
[0034] Figure 2 A multi-background vehicle personification cooperative decision-making framework of the application is shown;
[0035] Figure 3 A comparison diagram of RMSE of the application and other models is shown;
[0036] Figure 4 A comparison diagram of RMSE of the application and other models is shown;
[0037] Figure 5 A comparison diagram of RMSE of the application and other models is shown. DETAILED DESCRIPTION
[0038] As shown in Figures 1-5 The application provides a personified multi-background vehicle cooperative decision-making method for a virtual test scene, which includes the following steps:
[0039] S1: constructing a dynamic traffic graph of multiple vehicles based on a graph structure;
[0040] S2: building a generative adversarial imitation learning framework with parameter sharing, inputting the node features optimized by the graph attention network into the reinforcement learning network to generate serialized control instructions, and using the discriminator with shared scoring parameters to compare the generated control instructions with the state-action pairs of expert data to iteratively optimize the control instruction generator;
[0041] S3: introducing a latent factor variable to represent driving style, combining explicit rule punishment and implicit adversarial reward to enhance strategy personification;
[0042] S4: generating highly personified multi-background vehicle sequence cooperative control instructions through multiple iterations.
[0043] Specifically, the application realizes highly humanized multi-background vehicle cooperative decision-making by fusing real driving data and generative adversarial imitation learning (GAIL). First, the original traffic data set is cleaned by using extended Kalman filtering and other methods to form an expert data set. Node features such as vehicle speed and lane are extracted from the expert data set, and a driving style extraction method based on headway is introduced to cluster and divide the driving styles of vehicles in the data set. Then, a graph structure is used, with a single vehicle as a node, to calculate the edge weight and form a multi-vehicle dynamic traffic graph. A parameter-shared generative adversarial imitation learning framework is built. First, the parameter-shared graph attention network is used to strengthen the node features in the dynamic traffic graph. Then, the node features optimized by the graph attention network are input into the reinforcement learning network to generate serialized control instructions. The control instructions and node states are combined to form state-action pairs. Finally, the discriminator with shared scoring parameters is used to compare the generated state-action pairs with the state-action pairs in the expert data set, and iteratively optimize the control instruction generator. At the same time, in order to obtain more realistic humanized decision-making, humanization can be represented on the one hand as the driver behavior in the expert data set. The closer it is, the higher the humanization is. On the other hand, because the driving style is introduced, the driving decisions formed are more humanized, and explicit rule penalties are relied on to avoid collisions, boundary crossings, and other behaviors. The application uses latent factor variables to represent driving styles to enhance strategy learning, and uses reward imitation learning to combine explicit rule penalties (collision, boundary crossing, and sudden braking) with implicit adversarial rewards. Finally, the latent factor variables, reward imitation learning, and generative adversarial imitation learning are combined for iterative optimization to form highly humanized multi-background vehicle serialized control instructions.
[0044] Specifically, the application realizes a multi-background vehicle humanized cooperative decision-making method in automatic driving virtual simulation testing. A graph structure is used to realize more accurate modeling of complex interaction relationships between multiple vehicles. A generative adversarial imitation learning method with a parameter sharing mechanism is used to realize rapid training of multi-background vehicle cooperation. Latent factor variables and reward-enhanced imitation learning are used to iteratively optimize the control instruction generator to generate decisions with diversity, compliance, and humanization, thereby improving the reliability of automatic driving simulation testing from multiple perspectives.
[0045] The application uses root mean square error (RMSE) to measure the error between the real trajectories in the data set and the trajectories generated by the method. Specifically, as shown in Figures 3-5 RMSE between the positions, speeds, and lane offsets of the original human-driven vehicles and the vehicles after replacing the decision-making method is calculated by comparing the RMSE between the positions, speeds, and lane offsets of the original human-driven vehicles and the vehicles after replacing the decision-making method. By comparing the application method with the baseline models static Gaussian model (SG) and behavior cloning model (BC), it is found that Figure 3It can be seen from the figure that the RMSE of the static Gaussian model and the behavior cloning model increases significantly with time, while the model of the application always keeps the lowest, which shows that the application model has smaller error in predicting the position of the vehicle and is superior to the other two models. Figure 4 It can be seen from the figure that the RMSE of the static Gaussian model and the behavior cloning model exceeds 1.2 m after time increases, while the model of the application is stable at about 0.7 m, which shows that the application model is more optimal in controlling the lane line deviation in the lane keeping task, and can more accurately maintain the position of the vehicle in the lane and reduce the deviation error. Figure 5 It can be seen from the figure that the static Gaussian model has the highest RMSE (such as 3.5 m / s when the variable is 8), the behavior cloning model is second (2.5 m / s), and the application model is the lowest (about 1.8 m / s), which shows that the application model is closer to the true value in speed prediction or control, has smaller error, and has more accurate speed control.
[0046] Specifically, the application can effectively capture the spatial interaction features between vehicles by modeling the complex interaction relationship between multiple vehicles through a graph structure. Compared with the traditional rule-based preset model, the graph structure can dynamically adjust the interaction weight between vehicles, thereby more truly reflecting the complex interaction mode in the real traffic scene. In addition, by fusing the parameter sharing mechanism of generative adversarial imitation learning, the application can significantly shorten the training time of multi-background vehicle cooperative decision-making. The parameter sharing mechanism enables the model to quickly expand from single-agent learning to multi-agent learning, thereby significantly improving the training efficiency while ensuring the authenticity of the simulation test.
[0047] Specifically, the application generates driving behaviors with diversity, compliance and anthropomorphism through latent factor decoupling and reward enhanced imitation learning. The latent factor variable matches the driving style from the expert data set, such as aggressive type, conservative type, etc., and embeds it into the decision-making model of the background vehicle, thereby generating the action state of the background vehicle with different driving styles. The threshold of the driving style is not fixed, and the driving style extraction method based on the headway divides the driving style into three categories (aggressive type, ordinary type and conservative type) according to the clustering of the lane time interval in the data set. In the application, (0s~1s, 1s~2s, 2s~infinity) correspond to aggressive type, ordinary type and conservative type respectively. The reward enhanced imitation learning further optimizes the generated driving behavior by introducing explicit rule punishment (such as collision, boundary crossing, sudden braking) and implicit adversarial reward, so that the generated driving behavior is closer to the real driving scene.
[0048] Specifically, in the construction of the dynamic traffic graph of multiple vehicles, the traffic data set collected in the real scene is preprocessed to obtain an expert data set, and then the interaction relationship between multiple vehicles is represented by a graph structure, thereby obtaining the spatial interaction features between vehicles.
[0049] Specifically, the dynamic traffic graph of multiple vehicles is constructed based on the graph structure: the original traffic data is cleaned using extended Kalman filtering to generate a structured expert data set; node features such as vehicle speed and lane are extracted, and driving styles are clustered based on headway; the vehicle is taken as a node, the edge weight is calculated through the interaction relationship, and the dynamic traffic graph is constructed. In this process, the original traffic data is cleaned using extended Kalman filtering to generate a structured expert data set, then the node features are extracted, the driving style is clustered based on the headway, and the vehicle is taken as a node, the edge weight is calculated through the interaction relationship, and the dynamic traffic graph is constructed. First, the extended Kalman filter is used to eliminate the noise of the original data, and the data cleaning method based on traffic rules is used to eliminate the irregular data to form an expert data set. The driving style extraction method based on headway is introduced to extract different driving styles from the expert data set as the influencing factors of personification decision. Then, the node features such as vehicle speed, vehicle position and vehicle heading angle are extracted from the expert data set, the edge weight between nodes is calculated using the graph structure to represent the interaction relationship between vehicle nodes, and the multi-vehicle dynamic traffic graph is constructed as:
[0050] G t =(N t ,E t )
[0051]
[0052] Wherein, G t represents the spatial interaction feature, N t represents the set of all vehicle node features at the current time, including the longitudinal position X, speed V, lane category L, lane forward distance H and lane backward distance T of each vehicle; E t represents the interaction relationship set between multiple vehicles at the current time.
[0053] Specifically, as shown in Figure 2 , the generative adversarial imitation learning framework in the application includes a generator and a discriminator; the generator adopts a graph attention network with shared parameters, captures the dependency relationship between vehicle nodes through a hierarchical attention mechanism, and generates multi-background vehicle cooperative decision through a reinforcement learning network; the discriminator adopts a loss function with shared scoring parameters, which is used to evaluate the matching degree of the generated decision and the expert data, and the generator is iteratively optimized through adversarial training; wherein, the application first constructs the state-action pair (G t ,A t ) in the generative adversarial imitation learning based on the spatial interaction feature G t =(N t ,E t ) and the joint action set A t formed by all vehicle driving actions (acceleration a, steering angle θ, etc.) in the expert data set.
[0054] Specifically, unlike the application of generative adversarial imitation learning in multi-vehicle cooperative decision-making, the background vehicles are only part of the interaction information in the virtual test scene, and do not require a huge training time or data set, but require a faster training speed, thereby more efficiently building a virtual test scene; In the present application, a parameter sharing mechanism is used to enable the network model to quickly expand from single-agent learning to multi-agent learning; In the generator, the local observation of the agent i (including the state of itself, the state of the neighbor and the lane information) is input into the shared graph attention network, thereby quickly obtaining the interaction relationship between the background vehicles, and the multi-vehicle interaction relationship is as follows:
[0055]
[0056] wherein, is the graph attention network output of the lth layer of the vehicle i; and sigma is an activation function; is a degree matrix, representing the degree of each vertex in the graph; is an adjacency matrix, representing the connection relationship between nodes; represents the original node feature, and W ( l) represents the learning parameters shared by the lth layer.
[0057] Specifically, the node features processed by the graph attention network are input into the reinforcement learning network to generate the control decisions (acceleration alpha and heading angle theta) of the background vehicles, as follows:
[0058]
[0059] wherein, is the graph attention network output of the lth layer of the vehicle i, is a reinforcement learning generation network, represents the sampled vehicle decision action.
[0060] Specifically, in order to increase the robustness of the decision, the action state output by the action prediction network is a Gaussian distribution, and a specific action is sampled from the generated probability distribution The sampled action is applied to the vehicle dynamic model to update the state of the vehicle as follows:
[0061]
[0062] wherein, G i,t+1 is the state of the background vehicle i at time t+1, f is a vehicle dynamic model, is the state set of all background vehicles.
[0063] Specifically, in the discriminator, the state-action pair generated by the generator is input into the loss function shared by the scoring parameter to evaluate the rationality of the multi-vehicle cooperative decision generated by the reinforcement learning network, and the loss function is as follows:
[0064]
[0065] Wherein, M is the number of background vehicles, D ψ is a discriminator, (G t , A t ) is a state-action pair extracted from an expert dataset, is a state-action pair generated by the generator.
[0066] Wherein, when L D is lower than a set threshold, it is considered that the serialized control instruction generated by the generator at this time reaches the driving level comparable to the expert dataset, and the finally generated serialized control instruction is output at this time to drive the background vehicle in the simulation environment.
[0067] Specifically, in the prior art, the interaction between the measured vehicle and the background vehicle in the automatic driving simulation test is greatly different from the interaction in reality, and the main reason is that the decision complexity of the background vehicle is not fully considered when constructing the virtual test scene, that is, the personification decision conforming to the real driver behavior; In the present application, the potential factor variable z is set to represent the driving style obtained in S1, and the variable is spliced with the current node state G t to form an enhanced state, guiding the cooperative strategy to generate more personified driving behavior; wherein, the present application first defines the driving style extracted from the expert dataset as a mixed driving style set, and the potential variable matches the best driving style from the mixed driving style set according to the prior distribution, and then the variable is fused with the vehicle graph node, and the variable is embedded into the state features of the background vehicle, so as to generate the background vehicle control instruction with different driving styles, which is represented as:
[0068] G0~ρ0
[0069] z~p(z)
[0070] π~p(π|z)
[0071]
[0072] Wherein, G0 is the initial state of the background vehicle, z is a discrete potential variable for selecting a specific driving style, p(z) is the prior distribution of the potential variable, π is a driving style selected from the mixed driving style set,
[0073] is the spliced node state, is the graph attention network output based on the spliced node, and At+1 is a next state control instruction generated according to the enhanced node features.
[0074] Specifically, in order to provide traffic field related knowledge optimization strategy learning, the application introduces reward imitation learning to enhance the compliance of decision-making; the reward mechanism of the application consists of two parts, the first part is the reward based on imitation learning (explicit rule punishment), which is used to measure the similarity between the trajectory generated by the current strategy and the expert demonstration trajectory, when the loss function value iterative process shows a downward trend, it means that the current training process is a positive optimization process, and a reward value is given; otherwise, a penalty value is given.
[0075] The second part is the reward based on domain knowledge (implicit adversarial reward), which guides the policy learning to safer driving behavior by punishing non-compliant behavior.
[0076] Among them, the application punishes three kinds of non-compliant behaviors: collision, by calculating the minimum distance d c between the ego vehicle and other vehicles on the road, when the distance is less than a certain threshold, a penalty is applied; road deviation, by calculating the distance d road between the ego vehicle and the road edge, when d road is less than a certain threshold, a penalty is applied; hard braking, when the acceleration a of the vehicle is less than a certain threshold, a penalty is applied.
[0077] Specifically, the multi-round iteration generates a highly personalized discriminator update as follows:
[0078]
[0079] wherein, is a control instruction generated based on the splicing node, c is the driving style of the vehicle in the expert data set, R is the trajectory similarity reward function, is the penalty value of the three non-compliant behaviors.
[0080] Among them, through the introduction of driving style and reward imitation learning, the application finally generates a multi-background vehicle sequence control instruction that meets the personalization.
[0081] In some embodiments, the NGSIM dataset can be used in this application as an expert dataset, first smoothing the original trajectory data by using extended Kalman filtering, and using hard filtering based on traffic rules to eliminate irregular behaviors; Then convert the vehicle position and speed to the Frenet coordinate system, extract features such as lane center line relative coordinates, longitudinal speed and lane type; Use the driving style extraction method based on the headway to extract different driving styles from the expert dataset, and the driving styles extracted in this embodiment are ordinary, aggressive and conservative; Then use the graph structure to construct a dynamic traffic graph, the node features include longitudinal position, speed, lane type and front and rear vehicle distance, and the edge weight is calculated based on the relative distance between vehicles; On the basis of the dynamic traffic graph, the generator of the generative adversarial imitation learning is constructed, a 3-layer graph attention network is adopted, the node feature dimension of each layer is 128, and the feature enhanced dynamic traffic graph is obtained by aggregating neighbor vehicle information; Then input the feature enhanced dynamic traffic graph into the reinforcement learning network to obtain the action control instruction of the background vehicle, including the Gaussian distribution parameters of acceleration and steering angle, and then sample and update the state through the vehicle dynamics model; Compare the generated state-action pair with the state-action pair obtained from the expert dataset, and also need to compare the differences between the driving styles during the comparison process, and also need to punish the current decision if there is a collision, road deviation and hard braking behavior; When the background vehicle collides, a P collision penalty is given, when the distance difference between the geometric center of the background vehicle and the lane edge is less than 0.2m, a 3 / 4P lane deviation penalty is given, and when the acceleration of the background vehicle is less than -4m / s 2 , it is considered that the background vehicle has hard braking, and a 1 / 2P hard braking penalty is given; Fuse the above reward and punishment functions to comprehensively calculate the adversarial loss function, and iteratively optimize the generator; Finally, when the loss function value is approximately 0, the generator outputs highly personalized multi-background vehicle sequential control instructions.
[0082] The above only describes the preferred embodiments of the present application and should not be used to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. The above only describes the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be regarded as the protection scope of the present application.
Claims
1. A multi-background vehicle collaborative decision-making method for an anthropomorphic virtual test scenario, characterized by: The following steps are involved: S1: Construct a dynamic traffic graph of multiple vehicles based on graph structure; S2: Build a parameter-sharing generative adversarial imitation learning framework. Input the node features optimized by the graph attention network into the reinforcement learning network to generate serialized control instructions. Use a discriminator with shared scoring parameters to compare the generated control instructions with the state-action pairs of expert data to iteratively optimize the control instruction generator. S3: Introducing latent factor variables to characterize driving style, combining explicit rule penalties and implicit adversarial rewards to enhance strategy anthropomorphism; S4: Generate highly anthropomorphic multi-background vehicle sequence collaborative control instructions through multiple rounds of iteration.
2. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1 is characterized by: The multi-vehicle dynamic traffic graph constructed based on the graph structure is: The extended Kalman filter is used to clean the raw traffic data and generate a structured expert dataset. Node features are extracted and driving styles are clustered based on headway. With vehicles as nodes, edge weights are calculated through interaction relationships to construct a dynamic traffic map.
3. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1 is characterized by: The dynamic traffic graph of multiple vehicles is represented as follows: G t =(N t ,E t ) Among them, G t represents the spatial interaction feature, N t Represents the set of all vehicle node features at the current moment, including each vehicle's longitudinal position X, speed V, lane category L, lane forward distance H, and lane backward distance T; E t Represents the set of interaction relationships between multiple vehicles at the current moment.
4. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1 is characterized by: In S2, the generative adversarial imitation learning framework includes: a generator and a discriminator; The generator adopts a graph attention network with shared parameters, captures the dependencies between vehicle nodes through a hierarchical attention mechanism, and then generates multi-background vehicle collaborative decision-making through a reinforcement learning network.
5. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 4 is characterized by: The discriminator uses a loss function with shared scoring parameters to evaluate the degree of match between generated decisions and expert data, and iteratively optimizes the generator through adversarial training.
6. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 5 is characterized by: The discriminator uses the loss function of the shared scoring parameter as follows: In the discriminator, the state-action pairs generated by the generator are input into the loss function of the shared scoring parameter to evaluate the rationality of the reinforcement learning network in generating multi-vehicle collaborative decision-making. The loss function is as follows: Where M is the number of background cars, D ψ is the discriminator, A t is the background car decision generated by the reinforcement learning network, (G t ,A t ) are state-action pairs extracted from the expert dataset, is the state-action pair generated by the generator; When L D When it is lower than the set threshold, it is considered that the serialized control instructions generated by the generator have reached a driving level equivalent to that of the expert dataset. At this time, the final generated serialized control instructions are output to drive the background car in the simulation environment.
7. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1 is characterized by: The potential factor variables characterizing driving style are: The driving styles extracted from the expert dataset are defined as a mixed driving style set. The latent variables are matched with the best driving style from the mixed driving style set according to the prior distribution. The latent variables are then concatenated with the vehicle graph nodes and embedded into the state features of the background vehicles, thereby generating control instructions for background vehicles with different driving styles.
8. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1 is characterized by: In S3, the explicit rule penalty is: Rewards are given based on imitation learning to enhance decision compliance. By measuring the similarity between the trajectory generated by the current strategy and the trajectory demonstrated by the expert, when the iterative process of the loss function value shows a downward trend, it indicates that the current training process is a positive optimization process, and a reward value is given; Otherwise, a penalty value is given.
9. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 8, characterized in that: In S3, the implicit adversarial reward is: guiding the strategy to learn safer driving behavior by punishing non-compliant behavior; The non-compliant behaviors include collision, road departure and hard braking.
10. The method for collaborative decision-making in a multi-background vehicle environment in an anthropomorphic virtual test scenario according to claim 1, characterized in that: In S4, the discriminator after multiple rounds of iterations to generate highly anthropomorphic features is updated as follows: in, is the control command generated based on the splicing node, c is the vehicle driving style in the expert dataset, R is the trajectory similarity reward function, are the penalty values for the three non-compliant behaviors.
Citation Information
Patent Citations
Vehicle lane changing detection method based on time-space interaction graph attention network
CN116383685A
Automatic driving behavior model generation method and system
CN118410875A
Internet of vehicles environment intelligent penetration test method and system based on generative adversarial imitation learning
CN118944912A
Multi-modal feature extraction network enhanced negotiation decision-making method based on space-time directed graph
CN119357627A
Robust decision-making method for interconnected autonomous vehicles in mixed traffic flow based on deep reinforcement learning
CN119723519A
Cited By
Method and system for generating man-machine cooperation conflict test case
CN120994569A
Interactive simulation method and system oriented to automatic driving test and centered on tested vehicle
CN121365008A
Test-oriented interactive simulation method and system for autonomous driving test
CN121365008B
