Vehicle behavior decision-making method and system for heterogeneous traffic environment

Abstract

The application discloses a vehicle behavior decision-making method and system in a heterogeneous traffic environment, comprising: obtaining trajectory data of a self vehicle and other vehicles in a historical interaction process; determining a driving personality representation of the other vehicles according to the trajectory data, which reflects the driving behavior preference of the other vehicles in long-term interaction; determining a driving motivation representation of the other vehicles in a current traffic situation according to the driving personality representation and short-term interaction data in the current round; and determining a current driving control strategy of the self vehicle according to current perception observation of the self vehicle and the driving motivation representation, which is used for controlling longitudinal acceleration and lateral steering action of the self vehicle. The application can improve the inference accuracy of the driving intention of the other vehicles by the self vehicle in a heterogeneous traffic environment, and improve driving safety and traffic efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for vehicle behavior decision-making in heterogeneous traffic environments. Background Technology

[0002] In autonomous driving scenarios, vehicles need to continuously interact with other vehicles in heterogeneous traffic environments such as highways, exhibiting different driving styles, risk preferences, and decision-making habits. For example, aggressive drivers tend to perform frequent lane changes and close following maneuvers, while conservative drivers maintain a larger following distance and lower speed. Accurately inferring the behavioral patterns of other vehicles and making reasonable driving decisions during continuous interaction constitutes a core challenge in autonomous driving technology. To address this, existing autonomous driving technologies have proposed various opponent modeling methods, aiming to enable vehicles to infer hidden attributes such as the intentions and strategy types of other vehicles based on interaction history and observable data, thereby reducing environmental uncertainty and enhancing decision-making capabilities.

[0003] In the process of developing this invention, the inventors discovered that existing methods have at least the following drawbacks: First, existing methods typically assume that the set of adversary strategies used in the training phase is fixed and learn the adversary model based on this fixed set. However, in actual autonomous driving scenarios, the styles and strategies of other drivers are diverse and inexhaustible, making it difficult to fully cover them with a limited training strategy set. When the autonomous vehicle encounters an unknown driving style not present in the training set during the testing phase, existing methods will face out-of-distribution (OOD) behavioral patterns, leading to difficulties in effective model generalization and severe degradation or even failure of decision-making performance. Second, most existing methods only perform responsive modeling of other vehicles at the behavioral or action level, such as predicting their actions at the next moment. They lack the ability to structurally model and distinguish the intrinsic stable factors (such as driving personality) driving the decision-making behavior of other vehicles and the short-term decision-making driving forces induced by the situation (such as driving motivation in the current local traffic situation), resulting in a lack of transferable decision-making basis when facing unknown driving styles. Third, in multi-vehicle interaction environments, changes in the driving strategies of other vehicles can lead to non-stationary environmental characteristics, causing drastic fluctuations in the value estimation of the vehicle itself. Existing methods mostly adopt risk-neutral expected return assessment methods, which fail to effectively characterize and cope with the uncertainty brought about by changes in competitors, resulting in unstable training processes and further restricting the reliable deployment of autonomous driving systems in real open environments.

[0004] Therefore, how to extract stable and transferable behavioral representations of other vehicles from a limited interaction history without exhaustively enumerating all adversary strategies in advance, and how to cope with environmental non-stationarity and decision uncertainty caused by unknown driving styles, so as to enable the vehicle to maintain stable, safe and efficient driving decision-making capabilities in heterogeneous traffic environments, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention provides a vehicle behavior decision-making method and system for heterogeneous traffic environments, in order to solve the technical problems of poor generalization ability and unstable decision-making in existing adversary modeling methods when facing unknown driving styles in heterogeneous traffic environments.

[0006] To address the aforementioned technical problems, this application provides a vehicle behavior decision-making method for heterogeneous traffic environments, wherein the heterogeneous traffic environment includes other vehicles with different driving styles, and the method includes: Acquire interaction trajectory data generated during historical interactions between the self-intelligent agent corresponding to the vehicle and the adversary intelligent agents corresponding to other vehicles. Based on the interaction trajectory data, the driving personality representation of the adversary agent is determined. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the adversary agent in cross-round interactions. Based on the driving personality representation and the short-term interaction data of the adversary agent in the current round, the current driving motivation representation of the adversary agent is determined. The current driving motivation representation is used to represent the short-term decision-making drive of the adversary agent in the current local traffic situation. Based on the current perception observations of the autonomous agent and the current driving motivation representation, the current driving control strategy of the autonomous vehicle is determined. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the autonomous vehicle so that the autonomous vehicle maintains driving safety and traffic efficiency during interaction with other vehicles.

[0007] To address the aforementioned technical problems, this application also provides a vehicle behavior decision-making system for heterogeneous traffic environments, wherein the heterogeneous traffic environment includes other vehicles with different driving styles, and the system includes: The data acquisition module is used to acquire interaction trajectory data generated by the self-intelligent agent corresponding to the vehicle and the adversary intelligent agents corresponding to other vehicles during historical interactions; The driving personality inference module is used to determine the driving personality representation of the opponent agent based on the interaction trajectory data. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the opponent agent in cross-round interactions. The driving motivation inference module is used to determine the current driving motivation representation of the opponent agent based on the driving personality representation and the short-term interaction data of the opponent agent in the current round. The current driving motivation representation is used to represent the short-term decision-making drive of the opponent agent in the current local traffic situation. The driving control strategy decision module is used to determine the current driving control strategy of the vehicle based on the current perception observation of the self-intelligent agent and the current driving motivation representation. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the vehicle so that the vehicle maintains driving safety and traffic efficiency during interaction with other vehicles.

[0008] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described vehicle behavior decision-making method for heterogeneous traffic environments.

[0009] The vehicle behavior decision-making method and system provided in this invention for heterogeneous traffic environments acquires interaction data of other vehicles from historical interaction trajectories. The autonomous vehicle can extract stable driving personality representations of these vehicles from long-term behavioral patterns, and then combine these representations with the current short-term action sequence to infer the vehicle's immediate driving intention in the current traffic situation. This allows the autonomous vehicle to understand the opponent's behavior as a comprehensive result of two complementary dimensions: long-term style and short-term motivation. The autonomous vehicle not only sees the opponent's current behavior but also understands its motivation for performing that behavior. Based on this, the inferred driving motivation is integrated as additional conditional information into the autonomous vehicle's own strategy generation process. This ensures that the final output of longitudinal acceleration and lateral steering control commands is not only based on the autonomous vehicle's current perception and observation but also takes into account forward-looking judgments of other vehicles' behavioral trends. This enables more accurate behavior prediction and more rational autonomous decision-making when interacting with other vehicles in heterogeneous traffic environments. By transforming the problem of inferring unknown driving styles into a structured cognitive problem of observable interaction sequences, the autonomous vehicle's driving safety and traffic efficiency in heterogeneous traffic environments are improved. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the vehicle behavior decision-making method for heterogeneous traffic environments according to this application; Figure 3 This is a schematic diagram of a highway heterogeneous traffic environment, one embodiment of the vehicle behavior decision-making method for heterogeneous traffic environments in this application; Figure 4a This is a schematic diagram of the Highway environment to which this application can be applied; Figure 4b This is a schematic diagram of the Pommerman environment in which this application can be applied; Figure 4c This is a schematic diagram of the OverCooked environment to which this application can be applied; Figure 5 This is a schematic diagram of a structural embodiment of a vehicle behavior decision-making system for heterogeneous traffic environments according to this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Please see Figure 1 ,like Figure 1As shown, the system architecture 100 may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0016] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc.

[0017] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to desktop computers, tablet computers, and laptop computers.

[0018] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0019] It should be noted that the vehicle behavior decision-making method for heterogeneous traffic environments provided in this application embodiment is executed by a server, and correspondingly, the vehicle behavior decision-making system for heterogeneous traffic environments is set in the server.

[0020] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. Specifically, the first terminal device 101, the second terminal device 102, and the third terminal device 103 in this embodiment can correspond to an actual application system in production.

[0021] Please see Figure 2 , Figure 2 This invention illustrates a vehicle behavior decision-making method for heterogeneous traffic environments, which include other vehicles with different driving styles. The method is applied to... Figure 1 Taking the server-side as an example, the details are as follows: S201: Obtain the interaction trajectory data generated by the self-intelligent agent corresponding to the vehicle and the adversary intelligent agents corresponding to other vehicles during the historical interaction process.

[0022] This embodiment provides a schematic diagram of a heterogeneous traffic environment on a highway, such as... Figure 3 As shown. Figure 3An exemplary heterogeneous traffic environment scenario is illustrated, in which multiple vehicles with different driving styles travel on the same highway segment. Different colored vehicle boxes represent agents with different driving styles: green boxes represent the agent corresponding to the vehicle itself, yellow boxes represent aggressive drivers, blue boxes represent neutral drivers, and white boxes represent conservative drivers. The historical trajectories of these vehicles are represented by semi-transparent phantoms and dashed lines, where the spacing between the phantoms reflects the speed differences between the vehicles; larger spacing indicates higher speed, and smaller spacing indicates lower speed. Figure 3 As shown, the ghosting distance of aggressive drivers (vehicles in yellow boxes) is usually larger, indicating that they have higher speeds, while the ghosting distance of conservative drivers (vehicles in white boxes) is relatively smaller, indicating that they have lower speeds.

[0023] exist Figure 3 In the scenario shown, the yellow vehicle on the left performs an aggressive lane-changing maneuver, quickly cutting into the lane in front of its own vehicle from the adjacent lane. This behavior is represented by a sudden lateral descent in the dashed trajectory. The yellow vehicle on the right exhibits a close following behavior, maintaining a small safe distance from the vehicle in front. Both behaviors demonstrate the typical decision-making patterns of aggressive drivers in different local traffic situations, which differ significantly from conventional driving strategies.

[0024] Furthermore, Figure 3 The image uses red shading to represent potential interaction conflict areas. These areas are located within the spatial range where the vehicle's trajectory intersects with or approaches the trajectories of surrounding vehicles, highlighting potential risks and uncertainties in complex traffic environments. Specifically, these interaction conflict areas can serve as a warning that the vehicle's current driving strategy may conflict with the trajectories of other vehicles within this area, thus providing a spatial reference for optimizing the vehicle's driving control strategy.

[0025] Figure 3 The heterogeneous traffic environment illustrated illustrates a typical challenge faced by autonomous driving systems in real-world deployment: the autonomous vehicle needs to continuously engage in a game-like interaction with human drivers who have different driving styles, risk preferences, and decision-making habits, and these behavioral differences are difficult to fully cover with a limited set of training policies. The autonomous vehicle cannot directly know the potential personality traits and current driving intentions of other drivers; it can only indirectly infer their behavioral patterns by observing their trajectories. Therefore, Figure 3 The scenario shown provides an intuitive context for the application of the vehicle behavior decision-making method provided in this embodiment. In this scenario, how to enable the vehicle to maintain stable decision-making and safe passage when facing a driver with unknown personality characteristics constitutes the core problem to be solved by the method in this embodiment.

[0026] Specifically, the interaction trajectory data is used to provide a data foundation for subsequent inference of driving personality. In this embodiment, the historical interaction process involves continuous interaction between the self-agent and the adversary-agent in a multi-round environment, with each round corresponding to a complete driving task cycle, such as the complete process of driving from the highway entrance to the exit.

[0027] The interaction trajectory data includes the observation and action information of the adversary agent at each time step; the observation information includes the adversary agent's own position, speed, acceleration and other motion states, as well as the surrounding vehicle information and road environment information it perceives; the action information includes the longitudinal control actions and / or lateral control actions performed by the adversary agent, such as acceleration, deceleration, lane changing and other operations.

[0028] Furthermore, in order to ensure that the interaction trajectory data can cover a sufficiently diverse range of driving behavior patterns, this embodiment pre-constructs a population of adversary strategies containing multiple adversary strategies, each adversary strategy in the population having a different driving behavior style.

[0029] Specifically, a population-based training algorithm is used to optimize the opponent's strategy population, and style diversity constraints are introduced during the optimization process to encourage different strategies in the population to exhibit distinguishable driving styles at the behavioral trajectory level.

[0030] The style diversity constraint is achieved by maximizing the mutual information between trajectory history and policy index, so that vehicle trajectories corresponding to different policy indices are more easily distinguishable in a statistical sense.

[0031] Optionally, the lower bound of mutual information is approximated by introducing a variational distribution, and the information gain at each time step is used as a style reward to guide policy optimization during population training.

[0032] It should be noted that during the population training process, a maximum entropy mechanism is also introduced to constrain the policy sampling distribution, so as to avoid the training process being overly concentrated on a few policies and to ensure the full participation of population members in the training and sampling process.

[0033] Based on this, a portion of the strategies from the trained opponent strategy population is selected to form the training set, and another portion of the strategies is selected to form the test set. The strategies in the test set can be all strategies that did not appear during the training phase, or they can be a mixture of strategies that appeared in the training set and strategies that did not appear in the training set in different proportions.

[0034] As a preferred implementation, for each opponent policy in the training set, the opponent policy is fixed, and a reinforcement learning algorithm (e.g., Proximal Policy Optimization, PPO) is used to train the ego agent to compete against the opponent policy until convergence, obtaining the optimal response policy corresponding to the opponent policy. Then, opponent policies are continuously sampled from the training set, and the agent interacts with them using the corresponding optimal response policy to generate interaction trajectory data for pre-training the personality inference module.

[0035] It should be understood that, considering that the driving behavior preferences of adversary agents typically need to be stably revealed over a longer timescale, rather than being determined solely by single-step actions, this embodiment constructs cross-round contextual data (denoted as ) from historical interaction trajectories. This serves as the organizational form for interactive trajectory data. Among them, k Indicates the first k Each opponent's strategy This is used to provide a basis for the self-aware agent to identify the behavioral style of the opponent's strategy.

[0036] Specifically, in construction At that time, first maintain an experience cache to store the experience points of the first experience point. k The historical trajectory generated by the strategic interactions of each opponent is then processed; multiple complete trajectories are then uniformly sampled from this experience buffer; next, continuous segments are extracted from each trajectory and sequentially spliced ​​together to form a... By employing a continuous segment approach, the local temporal coherence of the opponent's driving behavior can be preserved; by sampling from multiple trajectories, the behavioral changes exhibited by the same opponent's strategy in different driving cycles due to randomness or strategy adaptation can be covered.

[0037] Optionally, context data The form is ,in l For the index of the context sequence, L For the length of the context sequence, and They respectively represent the relationship between the first and the second. k Observations and action samples collected during the interaction of strategies between opponents.

[0038] Furthermore, during the testing phase, to evaluate the vehicle's decision-making performance when faced with unknown and time-varying driving styles, this embodiment samples the opponent's strategy from the test set and keeps it unchanged for a preset number of rounds. It is then replaced with another strategy resampled from the test set, thus making the opponent's strategy non-stationary in the time dimension. Throughout the testing process, the autonomous agent cannot obtain any prior information about the opponent's strategy and can only rely on historical trajectory data generated during its interactions with the opponent to make inferences and decisions.

[0039] Through the above method, the interaction trajectory data obtained in this embodiment has the characteristics of wide coverage and high style differentiation, which provides reliable data support for subsequent data-driven inference of driving personality and prediction of motivation, and is conducive to improving the vehicle's generalization ability when facing unknown and diverse driving styles.

[0040] S202: Based on the interaction trajectory data, determine the driving personality representation of the adversary agent. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the adversary agent in cross-round interactions.

[0041] Specifically, this embodiment, from the perspective of cognitive psychology, views the driving behavior of an adversary agent as the result of the interaction between its internal stable personality preferences and external traffic situations. The driving personality representation is used to capture the relatively stable behavioral tendencies of the adversary agent over a long timescale, such as a tendency towards aggressive driving (frequent lane changes, shorter following distance) or conservative driving (maintaining a larger following distance, fewer lane changes), thereby providing a structured prior basis for the vehicle to understand the adversary's behavioral patterns.

[0042] In one specific optional implementation, determining the driving personality representation of the adversary agent based on interaction trajectory data includes: The interactive trajectory data is encoded using a trajectory encoder to obtain the context embedding; By using a personality gating network to process the context embedding, we obtain mixed weights corresponding to multiple learnable personality prototypes. A driving personality representation is obtained by weighting and combining multiple learnable personality archetypes and their corresponding mixed weights.

[0043] In one specific optional implementation, a driving personality representation is obtained by weighting and combining multiple learnable personality archetypes and their corresponding mixed weights, including: Each personality prototype vector is multiplied by its corresponding mixed weight and then summed. The sum is used as a representation of driving personality.

[0044] In one specific optional implementation, a personality gating network is used to process the context embedding to obtain mixed weights corresponding to multiple learnable personality prototypes, including: A personality gating network is used to perform non-linear mapping on the context embedding to obtain the initial scores corresponding to each personality archetype; Normalized index calculations are performed based on each initial score, and the results are used as the mixed weights corresponding to each personality archetype.

[0045] In one specific optional implementation, the method for determining the driving personality representation of the adversary agent based on the interaction trajectory data is as follows: First, the interaction trajectory data is encoded using a trajectory encoder to obtain context embeddings. This trajectory encoder is used to convert the raw interaction trajectory data (including observation and action sequences) into fixed-dimensional feature vectors to capture pattern information related to the opponent's long-term driving style.

[0046] Preferably, this embodiment employs a causal Transformer architecture to implement the trajectory encoder. A masked self-attention mechanism ensures that the encoding process satisfies temporal causality, meaning the encoding result at the current moment depends only on historical information and is unaffected by future information. Furthermore, observations and actions in the interactive trajectory have different data dimensions and physical meanings. This embodiment maps observations and actions to a unified latent embedding space and organizes them into a sequence according to their temporal order in the trajectory, alternating between them. This sequence is then input into the causal Transformer for encoding processing to obtain the context embedding.

[0047] Then, a personality gating network is used to process the context embedding, obtaining mixed weights corresponding to multiple learnable personality prototypes. Each personality prototype is a set of learnable vectors, and each prototype vector corresponds to an abstract driving style base class, such as aggressive, average, or conservative driving style tendencies. Unlike rigidly classifying driving styles into discrete categories, this embodiment treats these multiple personality prototypes as basis vectors in a driving style space. By assigning continuous mixed weights to each personality prototype, driving personality representation can be expressed as a continuous combination of multiple basic styles. The personality gating network adaptively determines the mixed weights of each personality prototype based on the context embedding, and its output is normalized to ensure that the sum of all weights is 1.

[0048] Specifically, the personality gating network performs nonlinear mapping on the context embedding to obtain the initial score corresponding to each personality prototype; then, it performs normalized exponent calculation based on each initial score, and uses the calculation result as the mixed weight corresponding to each personality prototype.

[0049] Next, a weighted combination of multiple personality archetypes and their corresponding mixed weights is performed to obtain the driving personality representation. This is achieved by multiplying each personality archetype vector by its corresponding mixed weight and summing the results; the sum is then used as the driving personality representation. In this way, the driving personality of any opponent can be represented as a weighted combination of multiple personality archetypes.

[0050] For ease of understanding, taking a highway scenario as an example, if a vehicle frequently exhibits aggressive lane-changing and close-following behavior in historical interactions, the personality gating network will assign higher weights to the personality archetype representing aggressive driving style, resulting in a driving personality representation that is biased towards aggressive driving in the feature space. If a vehicle maintains a large following distance and has a low frequency of lane changes in historical interactions, the corresponding mixed weights will be biased towards conservative driving. For driving styles that fall between the two, they are continuously characterized by the combined weights of multiple archetypes.

[0051] In one specific optional implementation, the method further includes a pre-training phase, which includes: Based on the driver's personality profile and current observations of the adversary agent, predict the distribution of the adversary agent's actions; Based on the difference between the action distribution and the actual actions of the adversary agent, a supervised loss is constructed; Based on the characteristic differences among multiple learnable personality archetypes, a diversity regularization term is constructed; Based on the supervised loss and diversity regularization term, the trajectory encoder and personalized gating network are jointly optimized.

[0052] In one specific alternative implementation, a diversity regularization term is constructed based on the characteristic differences among multiple learnable personality archetypes, including: A kernel matrix is ​​constructed based on the feature similarity between various personality archetypes; Calculate the determinant of the kernel matrix and use the logarithm of the determinant as the diversity regularization term.

[0053] Furthermore, in the pre-training phase, this embodiment employs supervised learning to train the trajectory encoder and the personalized gating network, enabling them to effectively extract driving personality representations from interactive trajectory data.

[0054] Specifically, the pre-training phase includes the following processes: Based on the driving personality representation and the current observations of the opposing agent, the distribution of the opposing agent's actions is predicted. Specifically, this embodiment introduces an action decoder, which takes the current observations of the opposing agent and the determined driving personality representation as input, and outputs the probability distribution of each action taken by the opposing agent. This action decoder can be regarded as a conditional action prediction model. The underlying modeling logic is as follows: under the same traffic observation conditions, agents with different driving personalities tend to take different driving actions. For example, when facing a slow vehicle ahead, an aggressive driver is more likely to choose to change lanes to overtake, while a conservative driver is more likely to slow down and follow.

[0055] Then, a supervised loss is constructed based on the difference between the predicted action distribution and the actual actions performed by the adversary agent in the actual interaction. Specifically, this supervised loss is used to measure the degree of deviation between the prediction result and the actual observation. This embodiment uses negative log-likelihood loss as the supervised loss, that is, minimizing the negative log probability of the predicted action distribution on the actual actions.

[0056] Meanwhile, to maintain the diversity and complementarity of multiple personality prototypes in the feature space and avoid degradation of different prototypes during training (i.e., multiple prototypes tending to the same feature representation), this embodiment constructs a diversity regularization term based on the feature differences between multiple personality prototypes. Specifically, a kernel matrix is ​​constructed based on the feature similarity between each personality prototype, the determinant value of the kernel matrix is ​​calculated, and the logarithm of the determinant value is used as the diversity regularization term. The theoretical basis of this regularization term comes from the Determinantal Point Process (DPP): the larger the determinant value of the kernel matrix, the stronger the orthogonality and the greater the feature differences between the prototype vectors. By maximizing this regularization term, each personality prototype can be encouraged to cover complementary driving style spaces, thereby improving the ability to express complex or mixed driving styles.

[0057] Ultimately, the overall training objective of the pre-training phase is the joint optimization of the supervised loss and the diversity regularization term. Specifically, the final optimization objective is to subtract the product of the diversity regularization term and the preset tradeoff coefficient from the supervised loss, that is, to simultaneously minimize the action prediction error and maximize the feature differences between personality prototypes. Through the above pre-training, the trajectory encoder and the personality gating network can extract driving personality representations with discriminative and generalizable capabilities from the interaction trajectory.

[0058] It should be noted that the above-mentioned action decoder is only used for parameter optimization of the trajectory encoder and personalized gating network during the pre-training stage. In the actual inference process after the pre-training is completed, the action decoder is no longer used.

[0059] Through the above methods, this embodiment can extract stable and transferable driving personality representations of adversary agents from limited historical interaction trajectories. Compared to rigidly classifying driving styles, the continuously mixed personality prototype representation can more accurately depict the gradual and mixed nature of driving styles in real traffic environments, which is beneficial for improving the vehicle's generalization ability when facing unknown driving styles not encountered during training.

[0060] S203: Based on the driving personality representation and the short-term interaction data of the adversary agent in the current round, determine the current driving motivation representation of the adversary agent. The current driving motivation representation is used to represent the short-term decision-making drive of the adversary agent in the current local traffic situation.

[0061] In step S203, based on the driving personality representation determined in step S202 and the short-term interaction data of the adversary agent in the current round, the current driving motivation representation of the adversary agent is determined. This current driving motivation representation is used to represent the short-term decision-making drive of the adversary agent in the current local traffic situation. For example, in a specific traffic scenario, the adversary agent's main intention at the current moment is to accelerate to overtake the vehicle in front, slow down to give way, or maintain the current lane.

[0062] Specifically, since driving personality representations reflect long-term stable characteristics across rounds, their temporal granularity is relatively coarse, making it difficult to capture behavioral changes of adversary agents in specific local traffic situations. For example, the same conservative driver may exhibit a brief period of aggressive evasive action when faced with a vehicle suddenly cutting in front, while the same aggressive driver may exhibit conservative following behavior in congested traffic due to a lack of lane-changing space. Therefore, this embodiment, based on driving personality representations, further introduces current driving motivation representations to characterize the short-term decision-making tendencies of adversary agents driven by local traffic situations at the current moment.

[0063] In one specific optional implementation, the current driving motivation representation of the adversary agent is determined based on the driving personality representation and short-term interaction data of the adversary agent in the current round, including: Obtain the action sequence of the opponent's agent within a preset historical time window in the current round, and use the action sequence as short-term interaction data; The action sequence is processed using a variational encoder, and the distribution of latent variables corresponding to the current driving motivation representation is determined by using driving personality representation as a conditional constraint. Samples are taken from the latent variable distribution, and the sampling results are used as a representation of the current driving motivation.

[0064] In one specific optional implementation, the current driving motivation representation of the adversary agent is determined based on the driving personality representation and short-term interaction data of the adversary agent in the current round, specifically including the following process: First, the action sequence of the opponent agent within a preset historical time window in the current round is obtained, and this action sequence is used as short-term interaction data. The preset historical time window refers to a preset number of consecutive time steps before the current decision moment. The window size can be set according to the actual application scenario and computing resources, for example, it can be set to the action sequence of the most recent 5 time steps.

[0065] Then, the above action sequence is processed using a variational encoder, and the latent variable distribution corresponding to the current driving motivation representation is determined using driving personality representation as a conditional constraint. This latent variable distribution is used to characterize the uncertainty of the adversary agent's current driving motivation under the combined influence of the currently observed short-term behavioral sequence and long-term personality preferences.

[0066] Specifically, this embodiment uses a Conditional Variational Autoencoder (CVAE) to implement the variational encoder described above. A Conditional Variational Autoencoder is a generative model that learns the posterior distribution of latent variables with input conditions as constraints and generates diverse outputs through sampling.

[0067] In this embodiment, the driving personality representation of the adversary agent is used as a global condition, and the short-term action sequence in the current round is used as input to learn a variational distribution to approximate the true posterior distribution. ,in, Indicates time t The current driving motivation representation of the adversary agent. This represents the sequence of actions within a preset historical time window. It represents the individual characteristics of a driver.

[0068] Next, samples are taken from the learned latent variable distribution, and the sampling results are used as a representation of the current driving motivation. Since the latent variable distribution reflects multiple possible driving motivations under the current conditions, a specific motivation representation vector can be obtained through sampling, which is used in the subsequent decision-making process.

[0069] In one specific alternative implementation, the variational encoder is trained as follows: Obtain the future observation sequence of the adversary agent within a future time window; Determine the attention weight corresponding to each future observation in the future observation sequence; The future observation sequence is weighted and aggregated according to each attention weight to obtain the target observation signal used to train the variational encoder. The target observation signal is reconstructed based on the driving personality and current driving motivation characteristics. Based on the difference between the reconstruction results and the target observation signal, a reconstruction loss is constructed; Based on the difference between the latent variable distribution and the preset prior distribution, a regularization loss is constructed; The parameters of the variational encoder are optimized based on the reconstruction loss and regularization loss.

[0070] In one specific optional implementation, determining the attention weight corresponding to each future observation in the future observation sequence includes: Encode the action sequence into a context representation; Each future observation in the future observation sequence is encoded into a corresponding future observation feature; Based on the degree of matching between the context representation and the features of each future observation, the attention weights corresponding to each future observation are determined.

[0071] Furthermore, the variational encoder described above is optimized for parameters through the following training process: Obtain the sequence of future observations of the adversary agent within a future time window. Future observations refer to the states that the adversary agent may be in several time steps after the current moment, such as the position, velocity, and other observation information of the adversary agent in the next 5 time steps.

[0072] The attention weights for each future observation in the future observation sequence are determined. These attention weights measure the importance of each future observation in characterizing the current driving motivation. Different future observations contain varying amounts of information; for example, if a future observation reflects the adversary agent's current urgent intentions, it should be assigned a higher weight. Specifically, the historical action sequence is first encoded as a contextual representation, and each future observation in the future observation sequence is encoded as a corresponding future observation feature. Then, based on the degree of matching between the contextual representation and each future observation feature, the attention weights for each future observation are determined. A higher degree of matching indicates a stronger consistency between the future observation and the dynamic evolution reflected in the current behavioral trajectory.

[0073] Furthermore, the future observation sequences are weighted and aggregated according to each attention weight to obtain the target observation signal used to train the variational encoder. This target observation signal is a weighted fusion representation used to provide a reconstruction target for the variational encoder during training.

[0074] It should be noted that the construction of the above target observation signal is not a simple averaging of future observations, but rather adopts a value-driven target observation generation mechanism.

[0075] Specifically, each future observation in the future observation sequence is input into the policy network of the self-agent to obtain candidate actions corresponding to each future observation; each candidate action is input into a distributed critic network, which is used to model the future reward distribution of the action to obtain the reward distribution corresponding to each candidate action; based on each reward distribution and risk sensitivity parameter, the utility score corresponding to each future observation is determined; based on each utility score, each future observation is weighted and fused, and the fusion result is used as the target observation signal.

[0076] Among them, the risk sensitivity parameter is used to control the vehicle's risk preference, and it is adaptively determined based on the driving personality characteristics.

[0077] Specifically, the driving personality representation is input into a pre-defined mapping network to obtain an intermediate variable. This intermediate variable is then subjected to amplitude limiting using a hyperbolic tangent function. The amplitude limiting result is multiplied by a pre-defined upper limit of the risk parameter range, and the product is used as a risk-sensitive parameter. Through this method, when the adversary agent's driving personality representation is biased towards aggressiveness, the risk-sensitive parameter is adjusted accordingly, causing the utility score to respond to this risk preference, thereby affecting the aggregation result of the target observation signal.

[0078] After obtaining the target observation signal, the signal is reconstructed based on the driving personality representation and the current driving motivation representation. Specifically, the decoder attempts to reconstruct the target observation signal using the sampled current driving motivation and driving personality representations as conditions. Then, a reconstruction loss is constructed based on the difference between the reconstructed signal and the target observation signal. A regularization loss is constructed based on the difference between the latent variable distribution and the preset prior distribution; this regularization loss is used to constrain the latent variable distribution from deviating too far from the prior distribution. Finally, the parameters of the variational encoder are optimized based on the reconstruction loss and the regularization loss.

[0079] Preferably, the training objective of the variational encoder is to maximize the conditional variational lower bound (ELBO), which is in the form of: The first term on the right is the reconstruction loss, which measures the accuracy of the decoder in reconstructing the target observation signal; the second term is the regularization loss, which constrains the distribution of latent variables. Compared with the preset prior distribution KL divergence between them This is a hyperparameter used to balance the importance of reconstruction loss and regularization loss. The preset prior distribution is a standard Gaussian distribution p(m) = N(0, I).

[0080] Furthermore, assuming that the decoder output follows a multivariate Gaussian distribution with constant diagonal covariance, the objective of maximizing ELBO can be equivalently transformed into minimizing the following loss function: in, and σ These are the mean and variance parameters of the latent variable distribution, respectively. The dimension of the latent variable.

[0081] Through the above methods, this embodiment can infer the current driving motivation of an adversary agent from its short-term interactive behavior. Compared to reactive modeling at the action level only, generative modeling using conditional variational autoencoders can better express the multimodal motivations (such as overtaking, following, and yielding) that the adversary agent may have in the current situation. The value-driven target observation generation mechanism aligns the training signal with the final decision reward, making the learned driving motivation representation more conducive to subsequent strategy decisions, thereby improving the vehicle's adaptability to unknown and non-stationary driving behaviors.

[0082] S204: Based on the current perception observations and current driving motivation representation of the self-intelligent agent, determine the current driving control strategy of the self-vehicle. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the self-vehicle so that the self-vehicle maintains driving safety and traffic efficiency when interacting with other vehicles.

[0083] In S204, based on the current perception observations of the autonomous agent and the current driving motivation representation determined in S203, the current driving control strategy of the vehicle is determined. This current driving control strategy is used to control the longitudinal acceleration and / or lateral steering actions of the vehicle, thereby enabling the vehicle to maintain driving safety and traffic efficiency during interactions with other vehicles.

[0084] It should be noted that the lateral steering action described in this embodiment refers to the steering operation of the vehicle in the lateral direction, and the specific control quantity is the steering wheel angle or the front wheel angle. In practice, this lateral steering action can be achieved by controlling the vehicle's steering system to adjust the vehicle's driving direction laterally.

[0085] In one specific optional implementation, the current driving control strategy of the vehicle is determined based on the current perceptual observations and current driving motivation representations of the self-aware agent, including: By inputting the current perception observations and current driving motivation representations into the policy network, the probability distribution of each candidate control action taken by the vehicle under the current traffic conditions is obtained. Sampling is performed based on a probability distribution, and the sampling results are used as the current driving control strategy.

[0086] The current perception and observation includes information on the vehicle's own motion status, as well as information on surrounding vehicles and the road environment, obtained by the vehicle through onboard sensors (such as cameras, millimeter-wave radar, lidar, etc.).

[0087] It should be noted that during the strategy learning phase, driving personality representations are no longer used as explicit inputs to the policy network. This is because the current driving motivation representations have incorporated long-term personality biases and short-term interaction information during the inference process, and can serve as compact and highly relevant conditional variables for decision-making.

[0088] In one specific alternative implementation, the policy network is optimized in the following way: The probability ratio of the agent's current policy to the probability of taking the same action under the same conditions, given the previous policy. Obtain the current time-instance estimate of the dominance function; The first optimization term is determined by multiplying the probability ratio by the estimated value of the advantage function. The probability ratios are pruned, and the second optimization term is determined based on the product of the pruned probability ratios and the estimated value of the dominance function. The smaller value between the first and second optimization terms is selected and added to the regularization term of the policy entropy, which is then used as the policy optimization objective. Update the parameters of the policy network according to the policy optimization objective.

[0089] Specifically, the policy network described above is optimized in the following way: The probability ratio of the agent's current policy to the probability of taking the same action under the same state is obtained, i.e. .

[0090] Simultaneously obtain the current time-to-time estimate of the dominance function. The advantage function measures the superiority of the current action relative to the average level. Then, the first optimization term is determined by multiplying the probability ratio by the estimated advantage function value. The probability ratios are pruned and restricted to the interval [1-ϵ, 1+ϵ]. The second optimization term is determined by multiplying the pruned probability ratios by the estimated advantage function. .

[0091] The smaller value between the first and second optimization terms is selected and added to the regularization term of the policy entropy as the policy optimization objective. Finally, the parameters of the policy network are updated based on this objective. This optimization method is the pruning objective function form of Proximal Policy Optimization (PPO): in, This is the pruning factor, used to control the magnitude of policy updates and avoid drastic policy changes; Entropy regularization weights; This represents policy entropy, used to encourage policies to maintain a certain degree of exploratory nature.

[0092] In one specific optional implementation, the method further includes: Based on the self-aware agent's current perceptual observations, current control actions, and current driving motivation representations, a distributed critic network is used to determine the self-aware agent's reward distribution. The valuation result is determined based on the distribution of returns; The current driving control strategy is optimized based on the value assessment results.

[0093] In one specific alternative implementation, the distribution of rewards for the self-agent is determined using a distributed critic network, including: A predetermined number of quantile values ​​are output through a distributed network of critics, with each quantile value corresponding to a predetermined number of quantile levels; A predetermined number of quantile values ​​are used together as an approximate representation of the return distribution.

[0094] In one specific alternative implementation, multiple quantile levels are distributed at equal intervals within the [0,1] interval.

[0095] In one specific alternative implementation, the distributed critic network is trained as follows: Obtain the difference between the target quantile and the predicted quantile; The quantile Huber loss function is determined based on each quantile level and whether the difference is greater than zero. The parameters of the distributed critic network are updated with the goal of minimizing the quantile Huber loss function.

[0096] Furthermore, to provide a more granular evaluation of the strategy, this embodiment also includes a process of optimizing the strategy using a distributed network of critics.

[0097] Specifically, based on the current perception observations, current control actions, and current driving motivation representations of the self-aware agent, a distributed critic network is used to determine the reward distribution of the self-aware agent. Then, based on this reward distribution, a value assessment result is determined, and the current driving control strategy is optimized based on this value assessment result.

[0098] It should be noted that the current control action described in this embodiment refers to a specific control quantity generated and executed by the self-aware agent at the current decision-making moment, such as the current longitudinal acceleration value or lateral control value. This current control action corresponds to the action distribution output by the policy network.

[0099] Unlike traditional commentator networks that only estimate expected returns, distributed commentator networks model the complete return distribution. Specifically, a distributed commentator network is implemented as a deep neural network that outputs a predetermined number of quantile values, each corresponding to a predetermined number of quantile levels. These quantile values ​​together constitute an approximate representation of the return distribution.

[0100] Among them, multiple quantile levels are equally spaced within the interval [0,1], and their specific values ​​are... , This represents the number of quantile values.

[0101] The aforementioned distributed commentator network is trained as follows: The difference between the target quantile and the predicted quantile is obtained; based on each quantile level and whether the difference is greater than zero, the quantile Huber loss function is determined; and the parameters of the distributed commentator network are updated with the goal of minimizing this quantile Huber loss function. Specifically, the quantile Huber loss takes the following form: in, , , The Huber loss is used, and 1μ≤0 is the indicator function. This is a preset threshold.

[0102] In one specific alternative implementation, determining the value assessment result based on the return distribution includes: Risk sensitivity parameters are determined based on driving personality traits; The return distribution is distorted based on risk-sensitive parameters, and the expected value obtained after the distortion transformation is used as the value assessment result.

[0103] In one specific optional implementation, the return distribution is distorted according to a risk-sensitive parameter, including: The distortion weights corresponding to each quantile level are determined based on the risk sensitivity parameter. The larger the value of the risk sensitivity parameter, the larger the distortion weights corresponding to the tails of the return distribution. The quantile values ​​are weighted and summed according to each distortion weight, and the weighted sum is used as the value assessment result after distortion transformation.

[0104] In one specific optional implementation, the interaction trajectory data includes the historical observations and action sequences of the adversary agent in multiple driving cycle rounds, and the short-term interaction data includes the action sequence of the adversary agent in the most recent preset number of time steps in the current round.

[0105] Furthermore, when determining the value assessment result based on the aforementioned return distribution, this embodiment introduces a risk-sensitive mechanism. Specifically, firstly, a risk-sensitive parameter λt is determined based on driving personality characteristics, and then the return distribution is distorted according to this risk-sensitive parameter. The expected value obtained after the distortion transformation is used as the value assessment result.

[0106] Among them, the risk sensitivity parameter λt is adaptively determined based on the driving personality profile, and its specific form is as follows: in, ( ) is a two-layer multilayer perceptron (MLP). As a representation of driving personality, This refers to the hyperparameters used to constrain the range of risk parameters. Through this method, different driving personalities of adversaries will lead to different values ​​for risk-sensitive parameters, allowing the vehicle to adaptively adjust its assessment of the reward distribution based on the uncertainty of the adversary's behavior.

[0107] When distorting the return distribution based on a risk-sensitive parameter, the distortion weights corresponding to each quantile level are first determined according to the risk-sensitive parameter. The larger the value of the risk-sensitive parameter, the greater the distortion weight at the tail of the return distribution. Then, the quantile values ​​are weighted and summed according to each distortion weight, and the weighted sum is used as the value assessment result after the distortion transformation. The specific form is as follows: in, , Indicates the distortion function at the quantile level The first derivative at that point. Through this distortion transformation, the vehicle can dynamically adjust its risk preference for value assessment based on the opponent's driving style. For example, when the opponent's personality is more aggressive, the vehicle can adopt a more conservative decision-making attitude accordingly.

[0108] It should be noted that, in this embodiment, the interaction trajectory data includes the historical observations and action sequences of the adversary agent across multiple driving cycles, used for extracting driving personality representations; the short-term interaction data includes the action sequences of the adversary agent within the most recent preset number of time steps in the current cycle, used for inferring current driving motivation representations. These two types of data complement each other in terms of time scale: the former covers long-term interaction information across cycles, while the latter focuses on local behavioral changes within the current cycle.

[0109] In this embodiment, the inferred current driving motivation representation is used as a conditional input to the policy network, enabling the vehicle's decisions to rely not only on current perceptual observations but also on judgments of other vehicles' current driving intentions. Simultaneously, the distributed critic network, through reward distribution modeling and risk-sensitive distortion transformation, provides a more stable and robust value assessment signal for policy learning. The synergistic effect of these two mechanisms enhances the vehicle's decision-making stability and driving safety in heterogeneous traffic environments facing unknown driving styles.

[0110] As an optional implementation, this embodiment provides schematic diagrams of three multi-agent benchmark testing environments. Figure 4a The diagram illustrates a highway scenario within a highway environment. This environment is a multi-agent autonomous driving simulation environment with continuous state and action spaces, and can be considered a heterogeneous social environment. Figure 4a In the scenario depicted, multiple vehicles travel on the same multi-lane road. The autonomous vehicle's intelligent agent and the opposing intelligent agents of other vehicles each make decisions based on independent driving strategies. The driving styles of each vehicle differ, including aggressive, moderate, and conservative types. These style differences are reflected in speed selection, following distance, and lane-changing decisions, thus creating implicit conflicts regarding speed and lane resource utilization. The autonomous vehicle needs to continuously interact with other vehicles to maintain traffic efficiency while ensuring its own driving safety.

[0111] Figure 4b This illustrates a Free For All (FFA) scenario within a Pommerman environment. This environment is a competitive multi-agent environment with a discrete action space and partial observability. Figure 4b In the scenario depicted, multiple agents place bombs on a two-dimensional grid map to destroy obstacles and eliminate other agents, while avoiding being affected by the blast waves. Each agent can only obtain local observation information within a limited field of view around it, and cannot access global map information. The main characteristic of this environment is its long-term decision-making nature; the explosion effect after an agent places a bomb has a delay, causing the action to lag behind the decision-making time. Simultaneously, the actions of agents are difficult to accurately predict under partially observable conditions. These factors collectively determine the final outcome of the game. This scenario is used to verify the adaptability of the method provided in this embodiment to competitive adversarial scenarios facing unknown opponent strategies.

[0112] Figure 4c An asymmetric layout scenario is illustrated in the OverCooked environment. This environment is a cooperative multi-agent environment with partial observability. Figure 4cIn the scenario shown, the autonomous agent corresponding to the vehicle needs to collaborate with a teammate agent to complete the preparation and delivery of as many dishes as possible within a shared kitchen layout. Completing a dish typically requires executing multiple sub-tasks sequentially, such as ingredient collection, cooking, and serving. Inefficient or mismatched behavior from any agent can significantly reduce the overall team performance. An asymmetric layout refers to the uneven distribution of work areas (such as ingredient areas, cooking areas, and serving areas) within the kitchen, requiring efficient coordination and cooperation between agents to complete complex collaborative tasks. This scenario is used to verify the collaborative generalization ability of the method provided in this embodiment when facing unknown teammate strategies in a cooperative setting.

[0113] The three environments mentioned above cover competitive scenarios (Pommerman-FFA), cooperative scenarios (OverCooked), and heterogeneous social interaction scenarios (Highway) that combine cooperative and competitive elements. They can comprehensively evaluate the generalization performance and decision stability of the vehicle behavior decision-making method provided in this embodiment under diverse interaction types from different perspectives.

[0114] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0115] Figure 5 This diagram illustrates the principle block diagram of a vehicle behavior decision-making system for heterogeneous traffic environments, corresponding one-to-one with the vehicle behavior decision-making method for heterogeneous traffic environments described in the above embodiments. For example... Figure 5 As shown, the functional modules of this vehicle behavior decision-making system for heterogeneous traffic environments are described in detail below: Data acquisition module 51 is used to acquire interaction trajectory data generated by the self-intelligent agent corresponding to the vehicle and the opponent intelligent agent corresponding to other vehicles during historical interactions; The driving personality inference module 52 is used to determine the driving personality representation of the opponent agent based on the interaction trajectory data. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the opponent agent in cross-round interaction. The driving motivation inference module 53 is used to determine the current driving motivation representation of the opponent agent based on the driving personality representation and the short-term interaction data of the opponent agent in the current round. The current driving motivation representation is used to represent the short-term decision-making drive of the opponent agent in the current local traffic situation. The driving control strategy decision module 54 is used to determine the current driving control strategy of the vehicle based on the current perception observation and current driving motivation representation of the self-intelligent agent. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the vehicle so that the vehicle can maintain driving safety and traffic efficiency when interacting with other vehicles.

[0116] Specific limitations regarding vehicle behavior decision-making systems for heterogeneous traffic environments can be found in the limitations of vehicle behavior decision-making methods for heterogeneous traffic environments described above, and will not be repeated here. The modules in the aforementioned vehicle behavior decision-making system for heterogeneous traffic environments can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0117] This application also provides another embodiment, namely, a computer-readable storage medium storing an interface display program that can be executed by at least one processor to cause the at least one processor to perform the steps of the vehicle behavior decision-making method for heterogeneous traffic environments as described above.

[0118] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A vehicle behavior decision-making method for heterogeneous traffic environments, characterized in that, The heterogeneous traffic environment includes other vehicles with different driving styles, and the method includes: Acquire interaction trajectory data generated during historical interactions between the self-intelligent agent corresponding to the vehicle and the adversary intelligent agents corresponding to other vehicles. Based on the interaction trajectory data, the driving personality representation of the adversary agent is determined. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the adversary agent in cross-round interactions. Based on the driving personality representation and the short-term interaction data of the adversary agent in the current round, the current driving motivation representation of the adversary agent is determined. The current driving motivation representation is used to represent the short-term decision-making drive of the adversary agent in the current local traffic situation. Based on the current perception observations of the autonomous agent and the current driving motivation representation, the current driving control strategy of the autonomous vehicle is determined. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the autonomous vehicle so that the autonomous vehicle maintains driving safety and traffic efficiency during interaction with other vehicles.

2. The method according to claim 1, characterized in that, Determining the driving personality profile of the adversary agent based on the interaction trajectory data includes: The interactive trajectory data is encoded using a trajectory encoder to obtain the context embedding; The context embedding is processed using a personality gating network to obtain mixed weights corresponding to multiple learnable personality prototypes; The driving personality representation is obtained by weighting and combining the multiple learnable personality archetypes and their corresponding hybrid weights.

3. The method according to claim 2, characterized in that, The driving personality representation is obtained by weighting and combining the multiple learnable personality archetypes and their corresponding hybrid weights, including: Each personality prototype vector is multiplied by its corresponding hybrid weight and then summed. The summation result is used as the driving personality representation.

4. The method according to claim 2, characterized in that, The context embedding is processed using a personality gating network to obtain mixed weights corresponding to multiple learnable personality archetypes, including: The context embedding is nonlinearly mapped using the personality gating network to obtain the initial scores corresponding to each personality archetype. The normalized index is calculated based on the initial scores, and the result is used as the mixed weight corresponding to each personality archetype.

5. The method according to claim 1, characterized in that, The step of determining the current driving motivation representation of the adversary agent based on the driving personality representation and the short-term interaction data of the adversary agent in the current round includes: Obtain the action sequence of the opponent agent within a preset historical time window in the current round, and use the action sequence as the short-term interaction data; The action sequence is processed using a variational encoder, and the distribution of latent variables corresponding to the current driving motivation representation is determined using the driving personality representation as a conditional constraint. Samples are taken from the latent variable distribution, and the sampling results are used as a representation of the current driving motivation.

6. The method according to claim 5, characterized in that, The variational encoder is trained in the following manner: Obtain the future observation sequence of the adversary agent within a future time window; Determine the attention weight corresponding to each future observation in the future observation sequence; The future observation sequence is weighted and aggregated according to each of the attention weights to obtain the target observation signal used to train the variational encoder; The target observation signal is reconstructed based on the driving personality representation and the current driving motivation representation; Based on the difference between the reconstruction result and the target observation signal, a reconstruction loss is constructed; Based on the difference between the latent variable distribution and the preset prior distribution, a regularization loss is constructed; The parameters of the variational encoder are optimized based on the reconstruction loss and the regularization loss.

7. The method according to claim 1, characterized in that, The method further includes: Based on the current perceptual observations, current control actions, and current driving motivation representations of the self-aware agent, the reward distribution of the self-aware agent is determined using a distributed critic network; The valuation result is determined based on the aforementioned return distribution; The current driving control strategy is optimized based on the value assessment results.

8. The method according to claim 7, characterized in that, The method of determining the reward distribution of the self-aware agent using a distributed critic network includes: The distributed critic network outputs a preset number of quantile values, each of which corresponds to a preset number of quantile levels. The preset number of quantile values ​​are used together as an approximate representation of the return distribution.

9. The method according to claim 7, characterized in that, The determination of the valuation result based on the return distribution includes: Risk sensitivity parameters are determined based on the aforementioned driving personality characteristics; The return distribution is distorted according to the risk sensitivity parameter, and the expected value obtained after the distortion transformation is used as the value assessment result.

10. A vehicle behavior decision-making system for heterogeneous traffic environments, characterized in that, The heterogeneous traffic environment includes other vehicles with different driving styles, and the system includes: The data acquisition module is used to acquire interaction trajectory data generated by the self-intelligent agent corresponding to the vehicle and the adversary intelligent agents corresponding to other vehicles during historical interactions; The driving personality inference module is used to determine the driving personality representation of the opponent agent based on the interaction trajectory data. The driving personality representation is used to represent the long-term stable driving behavior preference characteristics of the opponent agent in cross-round interactions. The driving motivation inference module is used to determine the current driving motivation representation of the opponent agent based on the driving personality representation and the short-term interaction data of the opponent agent in the current round. The current driving motivation representation is used to represent the short-term decision-making drive of the opponent agent in the current local traffic situation. The driving control strategy decision module is used to determine the current driving control strategy of the vehicle based on the current perception observation of the self-intelligent agent and the current driving motivation representation. The current driving control strategy is used to control the longitudinal acceleration and / or lateral steering of the vehicle so that the vehicle maintains driving safety and traffic efficiency during interaction with other vehicles.

Images