A mixed traffic interaction behavior trajectory prediction method, device and storage medium

Abstract

This invention discloses a method, device, and storage medium for predicting the trajectory of mixed traffic interaction behavior. The method includes: acquiring historical trajectory data and initial state information of multiple agents; inputting the historical trajectory data into a maximum entropy inverse reinforcement learning model to construct a maximum entropy behavior generation probability model based on the interaction behavior of multiple agents; employing a MA-AIRL framework with embedded dynamic reward generation for behavior backpropagation modeling and extracting implicit intentions; simultaneously utilizing Nash Q-learning based on Nash equilibrium to learn the game relationship between agents and generate a cooperative strategy; finally, inputting the strategy into a simulator to generate a predicted trajectory; comparing the predicted results with the actual trajectories; and verifying the accuracy of the prediction model based on specified indicators. This invention can significantly improve the accuracy and stability of multi-agent interaction trajectory prediction and effectively enhance the reliability of interaction behavior trajectory prediction results in mixed traffic environments.

Description

Technical Field

[0001] This invention relates to the field of traffic trajectory data prediction technology, specifically to a method, device, and storage medium for predicting traffic interaction behavior trajectories based on multi-agent inverse reinforcement learning and game theory, applicable to application scenarios such as autonomous driving, traffic simulation, and traffic behavior modeling and decision support in human-vehicle hybrid environments. Background Technology

[0002] With the rapid development of autonomous driving systems and intelligent transportation infrastructure, accurately predicting the future behavior of traffic participants has become a key technology for enhancing traffic safety and decision-making reliability. Among them, trajectory prediction, as an important process for understanding the interaction mechanism of multi-agent behavior, has received widespread attention.

[0003] Existing trajectory prediction methods still have significant shortcomings when modeling complex mixed traffic environments, especially in the difficulty of simultaneously characterizing the randomness, uncertainty, dynamic game and policy coupling relationships among multiple agents in the behavioral decision-making process.

[0004] In existing technologies, while Maximum Entropy IRL can reasonably model the uncertainty and distribution structure behind expert behavior, it lacks the ability to characterize the dependence of multi-agent interaction strategies. On the other hand, traditional methods based on Nash equilibrium or game-theoretic reinforcement learning can describe the strategy competition and cooperation mechanism between agents, but they are difficult to generate trajectories that conform to the characteristics of real behavior distribution, resulting in prediction results deviating from actual traffic patterns.

[0005] To address this, this application proposes a hybrid traffic interaction behavior trajectory prediction method that integrates multi-agent adversarial reinforcement learning (MA-AIRL) and Nash Q-learning for hybrid trajectory prediction. It also constructs a multi-agent trajectory prediction model based on a dynamic reward generation mechanism, which can adjust the reward gradient in real time according to the interaction scenario and learn the optimal strategy. This significantly improves the accuracy, stability, and cross-scenario generalization ability of trajectory prediction in complex mixed traffic environments, thereby solving the technical problems of the above-mentioned existing technologies in multi-agent interaction modeling and real trajectory generation. Summary of the Invention

[0006] The main objective of this invention is to provide a multi-agent hybrid traffic interaction behavior trajectory prediction method that integrates multi-agent adversarial reinforcement learning (MA-AIRL) and NashQ-learning, and introduces a dynamic reward mechanism. This method can adaptively adjust the reward function according to the real-time interaction status of traffic participants, enabling the model to maintain stable learning capabilities even in high-interaction and high-uncertainty scenarios. This improves the accuracy, stability, and realism of trajectory prediction and behavior generation, thereby addressing the technical problems of insufficient accuracy, inadequate policy modeling, and weak cross-scenario generalization ability in multi-agent interaction behavior trajectory prediction in mixed traffic environments, as proposed in the background art.

[0007] The present invention solves the above-mentioned technical problems by adopting the following technical solutions:

[0008] A hybrid traffic interaction behavior trajectory prediction method, based on MA-AIRL and Nash Q-learning, and incorporating a dynamic reward mechanism, is a multi-agent trajectory prediction method that includes the following steps:

[0009] S1. Acquire historical trajectory data and initial state information of multiple intelligent agents, including pedestrians and vehicles, in a mixed traffic scenario;

[0010] S2. Input historical trajectory data into the maximum entropy inverse reinforcement learning model, construct a maximum entropy behavior generation probability model based on multi-agent interaction behavior, and initialize the basic reward structure to provide the initial policy learning distribution;

[0011] S3. Based on maximum entropy reward modeling, an adversarial inverse reinforcement learning structure is introduced. The reward function is further optimized through the adversarial training mechanism between the discriminator output and the policy network behavior trajectory. An adaptive reward calibration mechanism is also incorporated so that the reward value can be dynamically adjusted with the interaction intensity, thereby improving the stability of reward learning and the accuracy of trajectory generation.

[0012] S4. In the process of modeling the interaction behavior of multiple agents, the Nash Q-learning algorithm is used to update the action value function of multiple agents, and the joint optimal policy distribution is solved based on the approximate Nash equilibrium principle to obtain the policy model;

[0013] S5. Input the trained policy model and the initial state into the behavior simulator, and generate a future trajectory prediction sequence through policy sampling;

[0014] S6. Compare the predicted results with the actual trajectory, and verify the accuracy of the prediction model based on specified indicators such as trajectory offset, acceleration deviation, and angular velocity offset.

[0015] Preferably, the formula for constructing the maximum entropy inverse reinforcement learning model in step S2 is:

[0016]

[0017] in, For the reward parameter is Time, trajectory The probability, For the reward function, For trajectory The cumulative reward represents the non-optimal behavior in the 1st month. each state Next action The total of the rewards The total number of states. Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility;

[0018] Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1, and its calculation formula is as follows:

[0019]

[0020] Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. Defined and implemented using a three-layer neural network, its parameter updates strictly follow the stochastic gradient ascent method. The updated calculation formula is as follows:

[0021]

[0022] in, It is a collection of historical trajectories. It is the learning rate parameter. These are the parameters of the reward function. The partial derivative of .

[0023] Preferably, for any group of intelligent agents, the specific operation flow of step S3 includes:

[0024]

[0025] in, Represented as the mathematical expectation of the historical trajectory, This is expressed as the mathematical expectation of the predicted trajectory. The total step size, For expert strategy distribution, Step size For the state Next action Discriminator calculation, existence discriminator calculation As shown below:

[0026]

[0027] in, Step size For the state Next action The reward function, Step size The action distribution predicted by the policy network is determined by the sampling weights. To calculate the effective ;

[0028] Sampling weight It is obtained using the discriminator function, and its calculation is as follows:

[0029]

[0030] The learning rate of the policy network;

[0031] Finally, by improving the reward parameters Adaptive rewards that enhance interaction intensity.

[0032] Preferably, the specific adaptive operation process for enhancing the interaction intensity through improved reward parameters in step S3 includes:

[0033] According to the status Constructing a dynamic weighting function for traffic interaction intensity Furthermore, the gradient term of the expert trajectory is weighted proportionally during the reward parameter update, allowing high-risk interaction segments to receive a larger update magnitude. The update formula with reward parameters is:

[0034]

[0035] Determined based on features including relative distance, velocity difference, and interaction angle, its calculation formula is as follows:

[0036]

[0037] in, Represents the relative distance between vehicles and pedestrians. This represents the speed difference between pedestrians and vehicles. This is the distance adjustment coefficient. This is the speed adjustment coefficient.

[0038] Preferably, the specific operation process of step S4 includes:

[0039] Modeling multi-agent interactions using a Nash equilibrium strategy yields the following formula:

[0040]

[0041] in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, For a collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q-function of an agent, Defined in state Below, the reward parameters The influence of local game relationships among multiple agents;

[0042] Based on current reward parameters and local game Solve for the Nash equilibrium mixed strategy ,in Nash equilibrium strategy is the state of an agent in the next moment after it performs an action. There exists that satisfy the following conditions:

[0043]

[0044] in, This is represented as assuming other intelligent agents They all follow the Nash strategy, and The average total reward that an agent can obtain when following candidate strategies. For the optimal action combination of other agents under Nash equilibrium, For the reward function parameters Below, from the state Initial expected cumulative return, As a discount factor, For the Nash equilibrium strategy of other agents, Representing other intelligent agents, For intelligent agents Candidate strategies, For intelligent agents Candidate actions, For reward function parameters The intelligent agent Consider the optimal actions of other agents combined with immediate rewards in the state. Immediate rewards for executing candidate actions;

[0045] The Nash equilibrium solution is achieved through the following formula:

[0046]

[0047] From state Initial expected cumulative return ,in The calculation is as follows:

[0048]

[0049] in, Indicates the state Now, assume that all agents take the optimal equilibrium action achieved through game theory. Calculate the expectation of these action combinations;

[0050] Update the Q-value function using the Bellman formula. ,have:

[0051]

[0052] in, This represents the weight parameters that control the update intensity.

[0053] Preferably, the specific prediction process for predicting the sequence using a simulator in step S5 includes:

[0054] Input the initial state of the agent ;

[0055] At each time step Sampling action Update the state via the physics engine. until a complete predicted trajectory is generated. , Total duration;

[0056] The pedestrian locations have been updated as follows:

[0057]

[0058]

[0059] The pedestrian speed update is as follows:

[0060]

[0061] The pedestrian angular velocity is updated as follows:

[0062]

[0063] in, express The lateral position of pedestrians at all times express The vertical position of pedestrians at all times express The speed of pedestrians at all times express Pedestrian heading angle at all times express The angular velocity of the pedestrian at any given moment Indicates time difference; Indicates pedestrian acceleration;

[0064] The vehicle locations have been updated as follows:

[0065]

[0066]

[0067] The vehicle speed updates are as follows:

[0068]

[0069] The vehicle angular velocity is updated as follows:

[0070]

[0071] in, express The lateral position of the vehicle at any given time. express The longitudinal position of the vehicle at any given time. express The speed of the vehicle at any time express Vehicle heading angle at any time express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any given moment.

[0072] Preferably, the verification process in step S6 includes:

[0073] The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula:

[0074]

[0075] The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula:

[0076]

[0077] in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment;

[0078] The root mean square error is calculated to reflect the overall magnitude of the error, as follows:

[0079]

[0080] in, for Real-time acceleration for Predict acceleration at all times.

[0081] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0082] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0083] As can be seen from the above technical solution, the present invention provides a method for predicting the trajectory of hybrid traffic interaction behavior. Compared with the prior art, the present invention has the following advantages:

[0084] 1. This invention introduces a dynamic reward function generation mechanism based on scene interaction intensity, which can adaptively adjust the reward gradient according to the interaction characteristics such as distance, speed difference, direction change rate and potential conflict risk between pedestrians and vehicles, thereby enhancing the model's policy learning ability in high-interaction scenarios. This solves the problem that existing trajectory prediction methods cannot adapt to changes in dynamic interaction intensity, resulting in weak policy learning and large trajectory prediction errors.

[0085] 2. This invention constructs a multi-agent historical trajectory dataset, employs the MA-AIRL framework with embedded dynamic reward generation for behavioral reverse modeling and extracts implicit intentions, and combines Nash Q-learning based on Nash equilibrium to learn the game relationship between agents and generate cooperative strategies. Finally, the strategies are input into a simulator to generate predicted trajectories, which can significantly improve the accuracy and stability of multi-agent interaction trajectory prediction and effectively enhance the reliability of interaction behavior trajectory prediction results in mixed traffic environments.

[0086] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0087] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0088] Figure 1 This is a schematic diagram of the overall trajectory prediction process of the present invention;

[0089] Figure 2 This is a comparison chart of pedestrian trajectory prediction in mixed traffic according to the present invention;

[0090] Figure 3 This is a comparison chart of vehicle trajectory prediction in mixed traffic according to the present invention;

[0091] Figure 4 This invention provides a comparison of vehicle and pedestrian acceleration predictions in mixed traffic. Detailed Implementation

[0092] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] For details in the embodiments, please refer to Figures 1 to 4 .

[0094] like Figure 1As shown in the embodiments of the present invention, a hybrid traffic interaction behavior trajectory prediction method is mainly aimed at scenarios where the interaction intensity in hybrid traffic changes drastically over time. It integrates multi-agent adversarial reinforcement learning (MA-AIRL) and Nash Q-learning, and introduces a dynamic reward mechanism. This multi-agent trajectory prediction method can adaptively adjust the reward function according to the real-time interaction state of traffic participants, enabling the model to maintain stable learning capabilities even in high-interaction, high-uncertainty scenarios. This improves the accuracy, stability, and realism of trajectory prediction and behavior generation. Specifically, it includes the following steps:

[0095] Step L1: Collect mixed traffic trajectory data, and acquire historical trajectory data of multiple intelligent agents, including pedestrians and vehicles, in a mixed traffic scenario. And its initial state information.

[0096] Specifically, this involves acquiring trajectory records of traffic participants, including pedestrians and vehicles, and using 500 historical trajectories of pedestrian and vehicle interaction events in CQUT-PVI5 as a historical dataset. The trajectory data comes from real-world urban traffic interaction scenarios, and the data state vector has a total of 18 dimensions (including 5 dimensions of pedestrian movement state, 5 dimensions of vehicle movement state, and 8 dimensions of interaction features).

[0097] Furthermore, based on the interaction event number and time synchronization information, the trajectories of multiple agents under the same traffic event are paired to form interacting trajectory segments. The merged trajectory data undergoes cleaning and preprocessing, including: interpolating missing position or velocity data to complete it, and removing noisy trajectory points. Finally, the multi-source heterogeneous data is uniformly organized into state-action-reward structured samples based on time series format, and divided into training and test sets at a ratio of 0.7 and 0.3, respectively, for use as input and validation for the inverse reinforcement learning model training.

[0098] At this point, each trajectory represents a sequence of states and actions. , Representing historical trajectory, among which This is the current state. The action (composed of agent acceleration and steering angle) is shown in Table 1 below:

[0099] Table 1: Dataset of Vehicle and Pedestrian Interaction Events

[0100]

[0101] Step L2: In terms of model architecture design, a deep neural network structure is adopted. The input layer receives an 18-dimensional state vector, which passes through a first fully connected layer with 256 neurons. The Tanh activation function is used and layer normalization is implemented (eps=1e-5). Here, the output layer is divided into a mean head and a log-standard deviation head, which output 2-dimensional action parameters respectively. The initial bias of the mean head is set to 0.0, and the initial bias of the log-standard deviation head is set to -0.5. At the same time, the reward network adopts a 3-layer fully connected structure (256→128→2). The weights of the final layer are initialized with a normal distribution N(0,0.01). Finally, the maximum entropy inverse reinforcement learning model used in this embodiment can be obtained.

[0102] Step L3: Input the aforementioned historical trajectory data into the maximum entropy inverse reinforcement learning model to construct a maximum entropy behavior generation probability model based on multi-agent interaction behavior, and initialize the basic reward structure to provide the initial policy learning distribution. At this point, each trajectory represents a sequence of states and actions. , It represents the historical trajectory and describes the specific steps that multiple agents perform in the environment.

[0103] The formula for constructing the maximum entropy inverse reinforcement learning model is as follows:

[0104]

[0105] in, For the reward parameter is Time, trajectory The probability, For the reward function, For trajectory The cumulative reward represents the non-optimal behavior in the 1st month. each state Next action The total of the rewards The total number of states. Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility;

[0106] Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1, and its calculation formula is as follows:

[0107]

[0108] Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. Defined and implemented using a three-layer neural network, its parameter updates strictly follow the stochastic gradient ascent method. The updated calculation formula is as follows:

[0109]

[0110] in, It is a collection of historical trajectories. It is the learning rate parameter. These are the parameters of the reward function. The partial derivative of .

[0111] Step L4: Based on maximum entropy reward modeling, an adversarial inverse reinforcement learning structure is introduced. Specifically, the historical trajectory and policy trajectory are input into the discriminator in the adversarial inverse reinforcement learning structure. The reward function is further optimized through the adversarial training mechanism between the discriminator output and the policy network behavior trajectory. Specifically, the discriminant function is trained to obtain the discrimination result used to distinguish expert prediction behavior, and an adaptive reward calibration mechanism is incorporated to jointly optimize the reward representation of state-action pairs in the policy network, so that the reward value can be dynamically adjusted with the interaction intensity, thereby improving the stability of reward learning and trajectory generation accuracy. Afterwards, the generator samples the data distribution, and these data distributions should be input into the discriminator.

[0112] Conversely, the discriminator is now responsible for classifying whether the data distribution comes from expert demonstrations or predictions. The discriminator and generator are trained simultaneously until the discriminator can no longer distinguish between the generator's output and the actual data. The goal of this method is to maximize the probabilities given by the discriminator function, as shown in the following equation for any group of agents:

[0113]

[0114] in, Represented as the mathematical expectation of the historical trajectory, This is expressed as the mathematical expectation of the predicted trajectory. The total step size, For expert strategy distribution, Step size For the state Next action Discriminator calculation, existence discriminator calculation As shown below:

[0115]

[0116] in, Step size For the state Next action The reward function, Step size The action distribution predicted by the policy network is determined by the sampling weights. To calculate the effective ;

[0117] Sampling weight It is obtained using the discriminator function, and its calculation is as follows:

[0118]

[0119] The learning rate for the policy network is η; specifically, in this example, the parameter η is set to 3e-4.

[0120] To enhance the learning ability of the reward function in key interaction scenarios, an adaptive reward calibration mechanism based on interaction intensity can be further introduced. This mechanism dynamically adjusts the weighting coefficients of the reward gradient based on features such as the relative distance, speed difference, and interaction angle between pedestrians and vehicles. This allows high-risk interaction segments to receive greater weight in the reward function update, thereby improving the representation accuracy and stability of reward learning. In other words, it improves the reward parameters. Adaptive rewards that enhance interaction intensity.

[0121] The specific adaptive operation process of adaptive rewards, which enhances interaction intensity by improving reward parameters, includes:

[0122] According to the status Constructing a dynamic weighting function for traffic interaction intensity Furthermore, the gradient term of the expert trajectory is weighted proportionally during the reward parameter update, allowing high-risk interaction segments to receive a larger update magnitude. The update formula with reward parameters is:

[0123]

[0124] It is determined based on features including relative distance, velocity difference, and interaction angle. The calculation formula is as follows:

[0125]

[0126] in, Represents the relative distance between vehicles and pedestrians. This represents the speed difference between pedestrians and vehicles. This is the distance adjustment coefficient. This is the speed adjustment coefficient.

[0127] Here, the reward parameters are adjusted using relative distance and speed difference. Its main objective is to highlight high-conflict, high-risk traffic moments during the reward update phase, enabling the model to learn key interactive behaviors in mixed traffic more accurately.

[0128] In this embodiment, the learning rate β=0.01, the batch size is 32, the iteration is 1000, and the Adam optimizer (β1=0.9, β2=0.999) is used to control the update step size of the reward parameters.

[0129] When the interaction intensity increases Increase the size to enhance keyframe reward learning;

[0130] When interaction decreases Reduce the reward to avoid overfitting to redundant background frames.

[0131] Step L5: In the process of modeling the multi-agent interaction behavior, the policy optimization objective of each agent is defined as maximizing its own reward when the policies of other agents are fixed. The action value function of the multi-agent is updated using the Nash Q-learning algorithm, and the joint optimal policy distribution is solved based on the approximate Nash equilibrium principle (the policies of multiple agents are jointly trained based on the approximate Nash Q-learning method), and finally the policy model is obtained.

[0132] The specific operational procedures at this time include:

[0133] This method employs a Nash equilibrium strategy to model multi-agent interactions. Each agent i updates its policy by maximizing the expected reward, and the system is jointly trained using an approximate Nash Q-learning algorithm. Nash Q-learning primarily addresses the immediate game relationships between agents, with Q-value updates directly dependent on the reward function output, without constraining the offset between policy distributions. This approach is more suitable for scenarios with relatively stable traffic structures where policy changes are mainly driven by the intensity of immediate interactions.

[0134] First, initialize the Q-value function. and reward parameters For each iteration: in the current state Downsampling combined action Perform actions and receive instant rewards. and the next state Constructing local games At this point, the corresponding calculation formula is:

[0135]

[0136] in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, For a collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q-function of an agent, Defined in state Below, the reward parameters The influence of local game relationships among multiple agents;

[0137] Based on current reward parameters and local game Solve for the Nash equilibrium mixed strategy ,in Nash equilibrium strategy is the state of an agent in the next moment after it performs an action. There exists that satisfy the following conditions:

[0138]

[0139] in, This is represented as assuming other intelligent agents They all follow the Nash strategy, and The average total reward that an agent can obtain when following candidate strategies. For the optimal action combination of other agents under Nash equilibrium, For the reward function parameters Below, from the state Initial expected cumulative return, As a discount factor, For the Nash equilibrium strategy of other agents, Representing other intelligent agents, For intelligent agents Candidate strategies, For intelligent agents Candidate actions, For reward function parameters The intelligent agent Consider the optimal actions of other agents combined with immediate rewards in the state. Immediate rewards for executing candidate actions.

[0140] In this example, a virtual game algorithm is used in the code, iterating for 5000 rounds until convergence, sampling 50 action candidates in each round, with a discount factor. .

[0141] Reward parameters Below, from the state Initial expected cumulative return ,in The calculation is as follows:

[0142]

[0143] in, Indicates the state Now, assume that all agents take the optimal equilibrium action achieved through game theory. Calculate the expectation of these action combinations;

[0144] Update the Q-value function using the Bellman formula. ,have:

[0145]

[0146] in, This represents the weight parameters that control the update intensity.

[0147] It should be noted that here The 'a' in the middle is used as a whole symbol to emphasize... The value is a global evaluation of "all combinations of actions by agents in the game," which is relevant during the game-solving phase. To solve for Nash equilibrium, it is necessary to explicitly distinguish the agents. one's own actions Actions with competitors (Omitted here) Because in the specific strategy update steps Considered as a known constant in the current iteration, and for the update iteration phase: in the Bellman update formula It is emphasized that the update logic includes feedback on the predictions of actions of other agents.

[0148] In summary, this method introduces a dynamic reward function generation mechanism based on scene interaction intensity, and incorporates an interaction intensity weight factor during the reward function update phase. It can adaptively adjust the reward gradient based on the interaction characteristics (traffic elements) such as the distance between pedestrians and vehicles, speed difference, rate of change of direction (interaction angle), and potential conflict risks. This can enhance the model's policy learning ability in high-interaction scenarios and solve the problem that existing trajectory prediction methods cannot adapt to dynamic interaction intensity changes, resulting in weak policy learning and large trajectory prediction errors.

[0149] It should be noted that this method focuses on improving the model's ability to identify and fit traffic segments with strong interactions, rather than constraining the stability of the policy distribution.

[0150] Step L6: Input the trained policy model and the initial state into the behavior simulator, and generate a future trajectory prediction sequence through policy sampling.

[0151] The specific prediction process for predicting sequences using a simulator includes:

[0152] Input the initial state of the agent ;

[0153] At each time step Sampling action Update the state via the physics engine. until a complete predicted trajectory is generated. , Total duration;

[0154] The pedestrian locations have been updated as follows:

[0155]

[0156]

[0157] The pedestrian speed update is as follows:

[0158]

[0159] The pedestrian angular velocity is updated as follows:

[0160]

[0161] in, express The lateral position of pedestrians at all times express The vertical position of pedestrians at all times express The speed of pedestrians at all times express Pedestrian heading angle at all times express The angular velocity of the pedestrian at any given moment Indicates time difference; Indicates pedestrian acceleration;

[0162] The vehicle locations have been updated as follows:

[0163]

[0164]

[0165] The vehicle speed updates are as follows:

[0166]

[0167] The vehicle angular velocity is updated as follows:

[0168]

[0169] in, express The lateral position of the vehicle at any given time. express The longitudinal position of the vehicle at any given time. express The speed of the vehicle at any time express Vehicle heading angle at any time express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any given moment.

[0170] In this example, The maximum angular velocity of the pedestrian is π rad / s (180° / s), and the turning angular rate of the vehicle is ±π / 2 rad / s (90° / s); the trajectory prediction of the agent is as follows: Figure 2 , 3 As shown, the acceleration prediction results are as follows: Figure 4 As shown.

[0171] It should be noted that the above method mainly adopts a single-path strategy output, the trajectory is entirely driven by the learning strategy, and it assumes that the prediction environment and the training environment have the same traffic structure and behavior pattern. It does not explicitly consider the impact of traffic uncertainty on the safety of strategy execution, and the calculation process mainly revolves around the real-time adjustment of the reward function. Since the trajectory is generated entirely by a single learning strategy, its formula system assumes that the prediction environment and the training environment have the same traffic structure and behavior pattern. Therefore, the adaptability to environmental uncertainty mainly depends on the learning effect of the reward function stage.

[0172] Step L7: Compare the predicted results with the actual trajectory, and evaluate and verify the accuracy of the prediction model based on specified indicators such as trajectory offset, acceleration deviation, and angular velocity offset.

[0173] The verification process at this time includes:

[0174] The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula:

[0175]

[0176] The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula:

[0177]

[0178] in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment;

[0179] The root mean square error (RMSE) is the square root of the average of the squares of the errors between the predicted and actual values. It reflects the overall magnitude of the error and is more sensitive to larger errors. The root mean square error for acceleration and yaw angle is calculated as follows:

[0180]

[0181] in, for Real-time acceleration for Predict acceleration at all times.

[0182] In this example, H=50. The training dataset was used to train a hybrid traffic trajectory prediction system combining MA-AIRL and Nash equilibrium. The final trajectory comparison results are shown in Table 2.

[0183] Table 2: Comparison and Analysis of Actual Trajectory and Predicted Trajectory

[0184]

[0185] Among the pedestrian trajectory indicators: the average displacement error (ADE) is 0.37m, the final displacement error (FDE) is 0.30m, and the root mean square error (RMSE) is... The mean square error is 0.76; vehicle trajectory indicators: average displacement error ADE 0.20m, final displacement error FDE 0.18m, root mean square error It is 0.54.

[0186] In summary, this method constructs a multi-agent historical trajectory dataset, employs the MA-AIRL framework with embedded dynamic reward generation for behavioral reverse modeling and extracts implicit intentions, and utilizes Nash Q-learning based on Nash equilibrium to learn the game relationship between agents and generate cooperative policies. Finally, the policies are input into a simulator to generate predicted trajectories. The experimental results in Table 2 above show that this method can significantly improve the accuracy and stability of multi-agent interaction trajectory prediction and effectively enhance the reliability of interaction behavior trajectory prediction results in mixed traffic environments.

[0187] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0188] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0189] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the hybrid traffic interaction behavior trajectory prediction methods described in the above embodiments.

[0190] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above method.

[0191] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0192] Memory, used to store computer programs;

[0193] When the processor executes the program stored in the memory, it implements the above-mentioned method for predicting the trajectory of hybrid traffic interaction behavior.

[0194] The communication bus mentioned in the above-mentioned electronic devices can be a standard bus for interconnecting peripheral components or an extended industrial standard structure bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0195] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0196] The memory may include random access memory or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0197] The processors mentioned above can be general-purpose processors, including central processing units, network processors, etc.; they can also be digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0198] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0199] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0200] Furthermore, it should be noted that if any directional indication (such as up, down, left, right, front, back, etc.) is involved in the embodiments of the present invention, the directional indication is only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0201] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, in the embodiments of this invention, "multiple" refers to two or more. Moreover, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

Claims

1. A method for predicting the trajectory of hybrid traffic interaction behavior, characterized in that, include: S1. Obtain historical trajectory data and initial state information of multiple intelligent agents in a mixed traffic scenario; S2. Input historical trajectory data into the maximum entropy inverse reinforcement learning model, construct a maximum entropy behavior generation probability model based on multi-agent interaction behavior, and initialize the basic reward structure; S3. The modeling introduces an adversarial inverse reinforcement learning structure, optimizes the reward function through a discriminator, and incorporates an adaptive reward calibration mechanism; S4. During the modeling process, the Nash Q-learning algorithm is used to update the action value function of the multi-agent, and the joint optimal policy distribution is solved based on the approximate Nash equilibrium principle to obtain the policy model; S5. Input the policy model and the initial state into the behavior simulator, and generate a future trajectory prediction sequence through policy sampling; S6. Compare the predicted results with the actual trajectory, and verify the accuracy of the prediction model based on the specified indicators; The specific operation process of step S4 includes: Modeling multi-agent interactions using a Nash equilibrium strategy yields the following formula: in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, For a collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q-function of an agent, Defined in state Below, the reward parameters The influence of local game relationships among multiple agents; Based on current reward parameters and local game Solve for the Nash equilibrium mixed strategy ,in Nash equilibrium strategy is the state of an agent in the next moment after it performs an action. There exists that satisfy the following conditions: in, Step size, This is represented as assuming other intelligent agents They all follow the Nash strategy, and The average total reward that an agent can obtain when following candidate strategies. For other intelligent agents Optimal action combination under Nash equilibrium For the reward function parameters Below, from the state Initial expected cumulative return, As a discount factor, For other intelligent agents Nash equilibrium strategy For intelligent agents Candidate strategies, For intelligent agents Candidate actions, For reward function parameters The intelligent agent Consider the optimal actions of other agents combined with immediate rewards in the state. Immediate rewards for executing candidate actions; The Nash equilibrium solution is achieved through the following formula: From state Initial expected cumulative return ,in The calculation is as follows: in, Indicates the state Now, assume that all agents take the optimal equilibrium action achieved through game theory. Calculate the expectation of these action combinations; Update the Q-value function using the Bellman formula. ,have: in, This represents the weight parameters that control the update intensity.

2. The method for predicting the trajectory of hybrid traffic interaction behavior as described in claim 1, characterized in that, The formula for constructing the maximum entropy inverse reinforcement learning model in step S2 is as follows: in, For the reward parameter is Time, trajectory The probability, For the reward function, For trajectory The cumulative reward represents the non-optimal behavior in the 1st month. each state Next action The total of the rewards The total number of states. Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility; Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1, and its calculation formula is as follows: Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. Define and update using stochastic gradient ascent. The updated calculation formula is as follows: in, It is a collection of historical trajectories. It is the learning rate parameter. These are the parameters of the reward function. The partial derivative of .

3. The method for predicting the trajectory of hybrid traffic interaction behavior as described in claim 2, characterized in that, For any group of intelligent agents, the specific operation flow of step S3 includes: in, Represented as the mathematical expectation of the historical trajectory, This is expressed as the mathematical expectation of the predicted trajectory. The total step size, For expert strategy distribution, Step size For the state Next action Discriminator calculation, existence discriminator calculation As shown below: in, Step size For the state Next action The reward function, Step size The action distribution predicted by the policy network is determined by the sampling weights. To calculate the effective ; Sampling weight It is obtained using the discriminator function, and its calculation is as follows: The learning rate of the policy network; Finally, by improving the reward parameters Adaptive rewards that enhance interaction intensity.

4. The method for predicting the trajectory of hybrid traffic interaction behavior as described in claim 3, characterized in that, The specific adaptive operation process for enhancing the interaction intensity through improved reward parameters in step S3 includes: According to the status Constructing a dynamic weighting function for traffic interaction intensity Furthermore, the gradient term of the expert trajectory is weighted proportionally during the reward parameter update, allowing high-risk interaction segments to receive a larger update magnitude. The update formula with reward parameters is: Determined based on features including relative distance, velocity difference, and interaction angle, its calculation formula is as follows: in, Represents the relative distance between vehicles and pedestrians. This represents the speed difference between pedestrians and vehicles. This is the distance adjustment coefficient. This is the speed adjustment coefficient.

5. The method for predicting the trajectory of hybrid traffic interaction behavior as described in claim 1, characterized in that, The specific prediction process for predicting the sequence using a simulator in step S5 includes: Input the initial state of the agent ; At each time step Sampling action Update the state via the physics engine. until a complete predicted trajectory is generated. , Total duration; The pedestrian locations have been updated as follows: The pedestrian speed update is as follows: The pedestrian angular velocity is updated as follows: in, express The lateral position of pedestrians at all times express The vertical position of pedestrians at all times express The speed of pedestrians at all times express Pedestrian heading angle at all times express The angular velocity of the pedestrian at any given moment Indicates time difference; Indicates pedestrian acceleration; The vehicle locations have been updated as follows: The vehicle speed updates are as follows: The vehicle angular velocity is updated as follows: in, express The lateral position of the vehicle at any given time. express The longitudinal position of the vehicle at any given time. express The speed of the vehicle at any time express Vehicle heading angle at any time express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any given moment.

6. The method for predicting the trajectory of hybrid traffic interaction behavior as described in claim 1, characterized in that, The verification process in step S6 includes: The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula: The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula: in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment; The root mean square error is calculated to reflect the overall magnitude of the error, as follows: in, for Real-time acceleration for Predict acceleration at all times.

7. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images