Intersection right-turn vehicle and pedestrian interaction trajectory prediction method and computer device

Abstract

The application discloses a kind of intersection right-turn vehicle and pedestrian interactive trajectory prediction method and computer equipment, method includes: obtaining intersection right-turn vehicle and pedestrian interactive person-vehicle history trajectory and initial state information;Through KL regularization improvement to adversarial inverse reinforcement learning structure, realize the joint learning of reward function and strategy;Joint optimization is carried out using Nash Q-learning algorithm with KL constraint;The optimized strategy model is input into behavior simulator, and the prediction trajectory generated by initial state driving is compared with the real trajectory, and the prediction result is evaluated based on specified indicators.The application can effectively improve the performance of the model in prediction accuracy, behavior consistency and environmental generalization ability, and further improve the stability of the reward function and the strategy network, enhance the generalization ability of the model in different traffic scenarios subsequently, so that the learned strategy has better migration potential.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of traffic trajectory data prediction, in particular to a prediction method for interactive trajectories of right-turn vehicles and pedestrians at an intersection and a computer device, and is suitable for predicting and verifying interactive behavior trajectories of right-turn vehicles and pedestrians in a mixed traffic environment. BACKGROUND

[0002] With the development of multi-agent interaction modeling technology and automatic driving systems, how to accurately predict the interactive trajectories of vehicles and pedestrians in a dynamic traffic environment has become an important basis for intelligent traffic safety decision-making and planning.

[0003] The existing trajectory prediction methods still have obvious deficiencies in dealing with high-coupling behavior patterns and environmental uncertainties, mainly manifested in the difficulty in simultaneously dealing with complex spatio-temporal interaction relationships, potential game nature in behavior decision-making, and dynamic changes in strategy distribution. Among them, although the deep learning type trajectory prediction method has advantages in curve fitting capability, it generally lacks causal explanation capability in the behavior level and is prone to cross-scene performance degradation; the prediction method based on inverse reinforcement learning can utilize implicit rewards to represent behavior generation logic, but its strategy convergence and stability are still limited when dealing with multi-agent multi-intention game structure in the vehicle right-turn scene; and the strategy modeling method based on game reinforcement learning can describe the strategy dependence relationship and dynamic competition structure between agents, but lacks an effective alignment mechanism for real observation trajectories and behavior distribution, resulting in deviation of the predicted trajectory from the real traffic behavior rules.

[0004] Therefore, the application provides a prediction method for interactive trajectories of right-turn vehicles and pedestrians at an intersection, which can realize multi-agent interactive trajectory prediction by combining AIR with Nash Q-learning with KL divergence constraint to solve the above technical problems. SUMMARY

[0005] The main purpose of the application is to provide a prediction method for interactive trajectories of right-turn vehicles and pedestrians at an intersection and a computer device, which can constrain strategy deviation and improve the alignment capability of strategy distribution to real behavior patterns by introducing KL policy sampling and regularization mechanism, thereby effectively improving the performance of the model in prediction accuracy, behavior consistency and environmental generalization capability to solve the technical problems proposed in the background art.

[0006] The application solves the above technical problems by adopting the following technical solutions:

[0007] A prediction method for interactive trajectories of right-turn vehicles and pedestrians at an intersection, comprising:

[0008] S1. Obtain the historical trajectory data of a target right-turn vehicle and a specified plurality of agents including surrounding pedestrians and vehicles in a mixed traffic scene, and initial state information thereof, and construct an environment encoding state vector;

[0009] S2. Input the historical trajectory data into a maximum entropy inverse reinforcement learning model, generate a maximum entropy probability distribution model of the trajectory, and learn an optimal reward function of the historical trajectory of the right-turn vehicle and the pedestrian at the intersection, which is used to provide an initial policy learning distribution;

[0010] S3. On the basis of maximum entropy reward modeling, construct a KL-AIRL model based on the optimal reward function of the maximum entropy probability distribution model, introduce a KL regularization term in the AIRL structure, which is used to constrain the difference between the policy distribution and the expert distribution, and enhance the training stability and sensitivity to behavior differences of the model;

[0011] S4. Introduce a multi-agent KL constraint Nash Q-learning algorithm in the KL-AIRL model to jointly optimize the strategies of each agent, so as to realize stable convergence and global optimal balance of the strategies in the game interaction environment;

[0012] S5. Construct an environment adaptive strategy fusion mechanism, input the strategy model obtained by the KL-AIRL model and the initial state into a behavior simulator, sample actions according to the fusion strategy, and generate a trajectory prediction sequence of the target right-turn vehicle and the pedestrian as the final prediction result;

[0013] S6. Compare the prediction result with the real trajectory, and verify the accuracy of the KL-AIRL model based on specified indicators such as trajectory offset, acceleration deviation, and angular velocity offset.

[0014] Preferably, the environment encoding state vector in the S1 step is set as: , wherein, is an environment encoding state vector containing , and , , , respectively represent the horizontal and vertical vectors of the agent plane coordinates,

[0015] Preferably, the model construction calculation formula of the maximum entropy inverse reinforcement learning model in the S2 step comprises:

[0016]

[0017] wherein, In order to achieve the reward parameter is Time, trajectory The probability of; Indicates the state ,action and environment The reward below, Indicates the first each state and this state Corresponding actions and environment The reward below, For trajectory The cumulative reward represents the optimal behavior in each state. Next action The total of the rewards, Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility;

[0018] Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1. Its calculation formula is as follows:

[0019]

[0020] Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. By definition, its parameter updates strictly use the stochastic gradient ascent method. The updated calculation formula is as follows:

[0021]

[0022] in, It is a collection of historical trajectories. It is the learning rate parameter. It is a parameter The gradient.

[0023] Preferably, in step S3, the discriminator is used to maximize the probability through a function, and its specific operation process includes:

[0024]

[0025] in, Represented as the mathematical expectation of the historical trajectory, is expressed as the mathematical expectation of the predicted trajectory, is the total time step, is the historical policy distribution, is the step The discriminator calculates the following for the state The action is executed The discriminator calculates the following for the state As shown below:

[0026]

[0027] wherein, is the step The reward function is calculated for the state The action is executed is the feature vector of the environment classification in italic, is the weight matrix of the environment classification, is the weight coefficient of the environment classification branch, is the step The action distribution predicted by the policy network is calculated by the sampling weight The sampling weight is fused using the KL divergence, and the calculation is as shown below:

[0028]

[0029] is the policy network learning rate, is the expectation under the policy is the regularization coefficient, represents the calculation of the KL divergence value, specifically an index that measures the distribution difference between the current learning policy and the historical reference policy is the probability of the reference policy selecting the action in the state is the policy network parameter The partial derivative is calculated, and the dynamic reference policy is updated as shown below:

[0030]

[0031] wherein, is the smoothing coefficient, is the current policy.

[0032] ​​​​​Preferably, the specific optimization process of the Nash Q-learning algorithm for jointly optimizing the policies of each agent in step S4 includes:

[0033] The formula for constructing multi-agent interaction is as follows:

[0034]

[0035] in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, For a collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q-function of an agent, Defined in state Below, the reward parameters The influence of multi-agent local game relationships. For reward parameters, This is the current state. For joint action;

[0036] Based on current reward parameters and local game Solve for a regularized Nash equilibrium hybrid strategy. ,in The regularized Nash equilibrium strategy represents the state of the agent in the next moment after performing an action. Based on current reward parameters It is confirmed that the following conditions exist:

[0037]

[0038] in, This indicates that when other agents all adopt their optimal strategies... At that time, intelligent agent The expected rewards that can be obtained For intelligent agents In state and actions Q value under, The optimal strategy for other agents For intelligent agents Reference strategy, For the optimal action combination of other agents under Nash equilibrium, For intelligent agents candidate strategies;

[0039] state expected cumulative return Here, the expectation is based on the Nash equilibrium strategy where The calculation is as follows:

[0040]

[0041] where, the expectation of joint action , is the action sampled according to the Nash equilibrium mixed strategy , is the set of optimal strategies of each agent obtained by game solving under the current reward parameter , is the Q value of agent ;

[0042] After that, for the Q value function , the Bellman formula is used for updating, and the updating formula is as follows:

[0043]

[0044] where, represents a weight parameter for controlling the updating strength;

[0045] Finally, according to the likelihood function of the historical demonstration trajectory, the reward parameter is updated, and has:

[0046]

[0047] where, is the set of historical demonstration trajectories, a real expert trajectory data sample library for model training, is the trajectory probability, indicating the possibility of observing a specific historical trajectory under the current reward parameter , is the partial derivative of the parameter , is a learning rate parameter, which is a hyperparameter for controlling the step size of parameter updating;

[0048] where is updated as shown in the following equation:

[0049]

[0050] where, is the average value of the KL divergence between strategies, For environmental quantity, In the environment The following strategy.

[0051] Preferably, in step S5, a dual-path trajectory prediction mechanism is introduced during the motion sampling process of the simulator. The specific execution flow of this mechanism is as follows:

[0052] When the system starts generating the predicted trajectory, it simultaneously invokes the trained policy network. Reference Strategy And construct the final action strategy through an environment-adaptive fusion formula. The fusion weights of the two strategies are determined by the environmental uncertainty coefficient. Dynamic adjustment:

[0053] When the environment is known or the model has high confidence, Prefer to adopt The output;

[0054] When environmental noise, unobservable factors, or changes in the interaction structure exceed a specified threshold are present, the system automatically increases its focus on the reference strategy. This dependency is used to enhance behavioral stability and security;

[0055] After completing the strategy fusion, the system according to Action sampling is performed, and the sampling results are input into the physics engine or dynamics model to update the state. This process is repeated until a complete trajectory is generated. Through this process, the model can adjust its strategy selection in real time under different environments, achieving a balance between learned decision-making capabilities and historical experience rules. This significantly improves the accuracy, robustness, and safety performance of trajectory prediction, as shown in the formula:

[0056]

[0057] in, In time step The chosen action As a fusion strategy, This represents the environmental uncertainty coefficient.

[0058] Preferably, the specific operation process for the simulator to perform action sampling and finally generate the prediction result in step S5 includes:

[0059] Input the initial state of the agent ;

[0060] Sampling action at each time step h ;

[0061] Update the state via the physics engine. until a complete predicted trajectory is generated. , is the total time length;

[0062] wherein the position of the pedestrian is updated as follows:

[0063]

[0064]

[0065] wherein the speed of the pedestrian is updated as follows:

[0066]

[0067] wherein the angular speed of the pedestrian is updated as follows:

[0068]

[0069] wherein, denotes the lateral position of the pedestrian at time t, denotes the longitudinal position of the pedestrian at time t, denotes the speed of the pedestrian at time t, denotes the heading angle of the pedestrian at time t, denotes the angular speed of the pedestrian at time t, denotes the time difference; denotes the acceleration of the pedestrian;

[0070] wherein the position of the vehicle is updated as follows:

[0071]

[0072]

[0073] wherein the speed of the vehicle is updated as follows:

[0074]

[0075] wherein the angular speed of the vehicle is updated as follows:

[0076]

[0077] wherein, denotes the lateral position of the vehicle at time t, denotes the longitudinal position of the vehicle at time t, denotes the speed of the vehicle at time t, denotes the heading angle of the vehicle at time t, express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any moment This indicates ambient noise.

[0078] Preferably, the verification process in step S6 includes:

[0079] The results of policy transfer indices, including KL divergence, are used to evaluate changes in conflict rate, adaptability of avoidance behavior, and trajectory deviation after policy transfer. Policy transfer indices (such as KL divergence) can be used to analyze the learning effect and generalization ability of the policy, and are calculated using the following formula:

[0080]

[0081] in, As a strategy transferability index, For environmental quantity, In the environment The following strategy checks all formatting issues across the entire document.

[0082] Preferably, the verification process in step S6 further includes:

[0083] The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula:

[0084]

[0085] The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula:

[0086]

[0087] in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment;

[0088] The root mean square error is calculated to reflect the overall magnitude of the error, as follows:

[0089]

[0090] in, for Real-time acceleration for Predict acceleration at all times.

[0091] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0092] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0093] As can be seen from the above technical solution, the present invention provides a method for predicting the interaction trajectory of right-turning vehicles and pedestrians at intersections. Compared with the prior art, the present invention has the following advantages:

[0094] 1. This invention integrates adversarial reinforcement learning (AIRL) and Nash Q-learning in the interactive trajectory prediction process, and introduces KL policy sampling and regularization mechanisms to constrain policy bias and improve the alignment of policy distribution with real behavior patterns, thereby effectively improving the model's performance in terms of prediction accuracy, behavior consistency and environment generalization ability.

[0095] 2. This invention further improves the stability of the reward function and policy network by introducing KL divergence constraints during model training, enhances the model's generalization ability in different traffic scenarios, and enables the learned policy to have better transfer potential.

[0096] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0097] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0098] Figure 1 This is a schematic diagram of the overall trajectory prediction process of the present invention;

[0099] Figure 2 This is a schematic diagram comparing trajectory prediction in the interaction scenario of right-turning vehicles and pedestrians at intersections, as presented in this invention. Detailed Implementation

[0100] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0101] like Figure 1 As shown in the embodiment of the present invention, the method for predicting the interaction trajectory of right-turning vehicles and pedestrians at intersections addresses the traffic characteristics of strong randomness and policy uncertainty in human-vehicle interaction behavior in right-turning scenarios at intersections by introducing [a method] into the objective function of the AIRL generator. The historical reference strategy constraint, by limiting the distribution offset between the current learned strategy and the historical reference strategy, suppresses abnormal decisions caused by local traffic disturbances, perceptual noise, or sparse samples, thereby ensuring the consistency of behavior and traffic safety rationality of right-turning vehicles and pedestrians in the continuous decision-making process. Simultaneously, through a dynamic reference strategy update mechanism, the reference strategy can be gradually adjusted according to changes in traffic flow conditions, avoiding strategy rigidity and better conforming to the traffic pattern of "progressive behavior correction" in real intersections. In specific applications, the execution based on KL divergence includes the following example steps:

[0102] Step L1: Mixed Traffic Trajectory Data Acquisition: Acquire historical trajectory data and initial state information of multiple agents, including the target right-turning vehicle and surrounding pedestrians and vehicles, in a mixed traffic scenario. At this stage, 500 interaction trajectories from the CQUT-RTI dataset are used, as shown in Table 1 below:

[0103] Table 1: Dataset of Vehicle and Pedestrian Interaction Events

[0104]

[0105] The dataset in Table 1 above is derived from real-world right-turn scenarios on urban roads. The data state vector has 19 dimensions (including 5 dimensions for pedestrian movement, 5 dimensions for vehicle movement, 8 dimensions for interaction features, and environmental features). Based on the interaction event number and time synchronization information, multi-agent trajectories under the same right-turn event are paired to form interactive trajectory segments. Missing position / velocity data is filled using linear interpolation, and noisy trajectory points are removed through box plot anomaly removal and other data cleaning and preprocessing. Finally, the data is uniformly processed into time-series format state-action-reward structured samples, and divided into training and test sets in a 7:3 ratio for input and validation of the inverse reinforcement learning model.

[0106] Each trajectory represents a sequence of states and actions. , Indicates the first A historical trajectory is used, and then an environmental encoding state vector is constructed by collecting data, including various aspects such as the speed, acceleration, and straight-line distance of vehicles and pedestrians. ,in, Encode the state vector for the environment. , These represent the horizontal and vertical dimensions of the agent's planar coordinates, respectively. , , These represent the agent's velocity, acceleration, and direction of movement angle and angular velocity, respectively. It represents scene attributes such as road type / traffic density and uses one-hot encoding.

[0107] In a further embodiment, the specific vector construction operation consists of a series of state-action pairs. ,in This is the current state. Accelerate the intelligent agent.

[0108] Step L2: In terms of model architecture design, the policy network adopts a deep neural network structure. The input layer receives an 18-dimensional state vector, which passes through a first fully connected layer with 256 neurons, using the Tanh activation function and implementing layer normalization (eps=1e-5). The output layer consists of a mean head and a log-standard deviation head, each outputting 2-dimensional action parameters. The initial bias of the mean head is set to 0.0, and the initial bias of the log-standard deviation head is set to -0.5. The reward network adopts a 3-layer fully connected structure (256→128→2), and the final layer weights are initialized using a normal distribution N(0,0.01).

[0109] Step L3: Input the above historical trajectory data into the maximum entropy inverse reinforcement learning model to generate the maximum entropy probability distribution model of the trajectory, and learn the optimal reward function of the historical trajectories of right-turning vehicles and pedestrians at the intersection to provide the initial policy learning distribution.

[0110] The calculation formula for constructing the maximum entropy inverse reinforcement learning model includes:

[0111]

[0112] in, In order to achieve the reward parameter is At that time, the trajectory of history The probability, in each trajectory representing a sequence of states and actions. middle, Indicates historical trajectory; Indicates the state ,action and environment The reward below, For trajectory The cumulative reward represents the optimal behavior in each state. Next action The total of the rewards, Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility;

[0113] Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1. Its calculation formula is as follows:

[0114]

[0115] Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. Defined as a three-layer neural network, its parameters are updated strictly using the stochastic gradient ascent method. The updated calculation formula is as follows:

[0116]

[0117] in, It is a collection of historical trajectories. It is the learning rate parameter. It is a parameter The gradient;

[0118] In this embodiment, the learning rate β is set to 0.01, the batch size to 32, and the iterations to 1000. The Adam optimizer is used to control the update step size of the reward parameters.

[0119] Step L4: Based on the maximum entropy reward modeling, construct the KL-AIRL model based on the optimal reward function of the maximum entropy probability distribution model. By introducing the KL regularization term into the AIRL structure, the difference between the policy distribution and the expert distribution is constrained, thereby enhancing the training stability of the model and its sensitivity to behavioral differences.

[0120] Historical trajectories and policy-generated trajectories are respectively input into the discriminator in the adversarial inverse reinforcement learning structure. The discriminator identifies and distinguishes between expert behavior and predicted behavior, and jointly optimizes the reward function and policy network. Simultaneously, the generator samples according to the policy distribution and inputs the generated trajectory into the discriminator, realizing the collaborative learning of adversarial training and reward representation. The discriminator is used to maximize the probability through a function, and its specific operation process includes:

[0121]

[0122] in, Represented as the mathematical expectation of the historical trajectory, This is expressed as the mathematical expectation of the predicted trajectory. The total time step, For historical strategy distribution, Step size For the state Next action Discriminator calculation, existence discriminator calculation As shown below:

[0123]

[0124] in, For the state Next action The reward function, Feature vectors for environmental classification (in italics) The weight matrix for environmental classification. These are the weighting coefficients for the environmental classification branches. Step size The action distribution predicted by the policy network is determined by the sampling weights. To calculate the validity Sampling weight KL divergence fusion is used to improve cross-environment prediction capabilities, and the calculation is shown below:

[0125]

[0126] For the policy network learning rate, In strategy The expectations below The regularization coefficient is . This refers to calculating the KL divergence value, specifically measuring the current learning strategy. Historical reference strategy Indicators of distribution differences between them As a reference strategy in state Select action The probability, For policy network parameters Find the partial derivative.

[0127] In this example The parameter is set to 3e-4. The parameter is set to 0.05;

[0128] Dynamic Reference Strategy The update is as follows:

[0129]

[0130] in, For smoothing coefficients, For the current strategy, the smoothing coefficient in this example Set it to 0.95.

[0131] In step L5, the NashQ-learning algorithm with multi-agent KL constraints is introduced into the KL-AIRL model to jointly optimize the policies of each agent, so as to achieve stable convergence of policies and global optimal balance in the game interaction environment.

[0132] Further, a human-vehicle interaction model is constructed, defining how pedestrians and vehicles maximize their own rewards under the condition that other agents' policies are fixed. Co-training of multi-agent policies is achieved based on an approximate NashQ-learning method. Here, the NashQ-learning algorithm models multi-agent interactions based on a Nash equilibrium policy. Each agent i updates its policy by maximizing its expected reward, and joint training is performed using an approximate NashQ-learning algorithm. The specific optimization process for jointly optimizing the policies of each agent includes:

[0133] Q-initialization function and reward parameters ;

[0134] For each iteration: in the current state Downsampling combined action Perform actions and receive instant rewards. and the next state Constructing local games The formula for constructing multi-agent interaction is as follows:

[0135]

[0136] in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, For a collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q-function of an agent, Defined in state Below, the reward parameters The influence of multi-agent local game relationships. For reward parameters, This is the current state. For joint action;

[0137] Based on current reward parameters and local game Solve for a regularized Nash equilibrium hybrid strategy. ,in The regularized Nash equilibrium strategy represents the state of the agent in the next moment after performing an action. Based on current reward parameters It is confirmed that the following conditions exist:

[0138]

[0139] in, This indicates that when other agents all adopt their optimal strategies... At that time, intelligent agent The expected rewards that can be obtained For intelligent agents In state and actions Q value under, The optimal strategy for other agents For intelligent agents Reference strategy, For the optimal action combination of other agents under Nash equilibrium, For intelligent agents Candidate strategies;

[0140] Reward parameters Below, from the state Initial expected cumulative return The expectation here is based on the Nash equilibrium strategy. ,in The calculation is as follows:

[0141]

[0142] in, For joint operations Expectations To implement the Nash equilibrium mixing strategy The action of performing a sampling operation To the current reward parameters The following is the set of optimal strategies for each agent obtained through game theory. For intelligent agents Q value;

[0143] Then for the Q-value function Update using the Bellman formula, as follows:

[0144]

[0145] in, The weight parameters that control the update intensity;

[0146] Finally, the reward parameters are updated based on the likelihood function of the historical demonstration trajectories. ,have:

[0147]

[0148] in, This is a collection of historical demonstration trajectories, serving as a database of real expert trajectory data samples used for model training. Let be the trajectory probability, representing the current reward parameter. Below, specific historical trajectories were observed. The possibility, For parameters Find the partial derivative. Here, represents the learning rate parameter, and represents the hyperparameter for controlling the update step size.

[0149] in The update is shown in the following equation:

[0150]

[0151] in, The average KL divergence among strategies. For environmental quantity, In the environment The following strategy.

[0152] Here, the KL divergence fusion term is explicitly introduced, and its role is integrated throughout multiple stages of policy learning, game solving, and reward updating, forming a unified constraint mechanism for traffic behavior stability. During the use of KL divergence fusion sampling weights, the distribution offset between the current policy and the reference policy is constrained by KL divergence, preventing right-turning vehicles or pedestrians from making abnormal actions under local traffic disturbances, obstructions, or perceived noise conditions. This ensures the rationality of continuous decision-making behavior from a traffic perspective. In solving the regularized Nash equilibrium mixed policy, KL regularization is introduced, ensuring that the game solution not only satisfies the immediate Q-value optimum but also conforms to the historical traffic behavior distribution. This avoids extreme policies that do not conform to actual traffic experience in the yield-cutting game. Finally, when updating the average KL divergence between policies, the KL mean of the cross-environment policy distribution is fed back to the reward parameter update, enabling the reward learning process to perceive policy differences under different intersection structures and traffic density changes, achieving consistency and transferability modeling of traffic behavior across different scenarios.

[0153] In this example, the weight parameters that control the update intensity In this example, a virtual game algorithm is used in the code, iterating for 5000 rounds until convergence, sampling 50 action candidates in each round, with a discount factor. NashQ-learning employs a dynamically adjusted learning rate strategy, with an initial value of 0.1, which is gradually reduced to 0.01 through cosine annealing. The discriminator uses the Adam optimizer, with a fixed learning rate of 1e-5 (β1=0.9, β2=0.999). The policy network performs an update once every 10 discriminator updates and implements global L2 norm gradient pruning (threshold 1.0).

[0154] By introducing KL divergence constraints during model training, the stability of the reward function and policy network can be further improved, enhancing the model's generalization ability in different traffic scenarios and giving the learned policy better transfer potential.

[0155] Furthermore, it should be noted that when solving the hybrid strategy, the model not only considers the optimality of the Q-value of each agent under the current traffic state, but also constrains the consistency of its strategy distribution with the reference strategy, and feeds back the KL mean of the strategy distribution under different traffic environments to the reward parameter update process, thereby realizing the closed-loop modeling of "game decision-making - strategy stability - cross-environment consistency". This design enables the model to maintain relatively consistent yielding and avoidance behaviors that conform to traffic common sense under different intersection structures, traffic density changes or perceived noise conditions.

[0156] Step L6: Construct an environment-adaptive policy fusion mechanism. Combine the policy model trained by the KL-AIRL model with the initial state input behavior simulator, perform action sampling according to the fusion policy, and generate trajectory prediction sequences for target right-turning vehicles and pedestrians as the final prediction results.

[0157] At this point, a dual-path trajectory prediction mechanism is introduced during the motion sampling process of the simulator. The specific execution flow of this mechanism is as follows:

[0158] When the system starts generating the predicted trajectory (dual-path trajectory prediction and generation stage), it simultaneously calls the trained policy network. Reference Strategy And use the environment-adaptive policy fusion formula to integrate the policy network and reference strategy The outputs are fused according to certain weights to construct the final action strategy. The fusion weights of the two strategies are determined by the environmental uncertainty coefficient. Dynamic adjustment determines:

[0159] When the environment is known or the model has high confidence, ， Prefer to adopt The output;

[0160] When environmental noise, unobservable factors, or changes in the interaction structure exceed a specified threshold are present, the system automatically increases its focus on the reference strategy. This dependency is used to enhance behavioral stability and security;

[0161] After completing the strategy fusion, the system according to Action sampling is performed, and the sampling results are input into the physics engine or dynamics model to update the state. This process is repeated until a complete trajectory is generated. This process fully demonstrates the adaptability of the fusion strategy in different environments. Through this process, the model can adjust the strategy selection method in real time in different environments. This achieves a balance between learned decision-making capabilities and historical experience rules, allowing the system to focus on policy network predictions in known environments and refer more to historical experience in unknown environments. This significantly improves the accuracy, robustness, and safety of trajectory prediction, as shown in the formula:

[0162]

[0163] in, In time step The chosen action As a fusion strategy, For environmental uncertainty coefficients, the parameters in this example are... .

[0164] This section explicitly introduces a dual-path trajectory prediction mechanism, adaptively fusing the learning strategy and the reference strategy. This allows for dynamic adjustment of the weights of the two strategies based on the environmental uncertainty coefficient, which corresponds to the uncertainties commonly present in real intersection environments, such as sudden changes in direction by irregular pedestrians, limited visibility for right-turning vehicles, and temporary traffic interference. When the environmental conditions are reliable, the system relies more on the learning strategy to maintain efficiency; however, when uncertainty increases, the weight of the reference strategy is increased to guide pedestrians to slow down or vehicles to yield, thus proactively introducing more conservative and safer traffic behaviors during the prediction phase.

[0165] Furthermore, the specific operational process of the simulator in sampling actions and ultimately generating prediction results includes:

[0166] Input the initial state of the agent ;

[0167] Sampling action at each time step h ;

[0168] Update the state via the physics engine. until a complete predicted trajectory is generated. , Total duration;

[0169] The pedestrian locations have been updated as follows:

[0170]

[0171]

[0172] The pedestrian speed updates are as follows:

[0173]

[0174] The pedestrian angular velocity is updated as follows:

[0175]

[0176] in, express The lateral position of pedestrians at all times express The vertical position of pedestrians at all times express The speed of pedestrians at all times express Pedestrian heading angle at all times express The angular velocity of the pedestrian at any given moment Indicates time difference; Indicates pedestrian acceleration;

[0177] The vehicle locations have been updated as follows:

[0178]

[0179]

[0180] The vehicle speed updates are as follows:

[0181]

[0182] The vehicle angular velocity is updated as follows:

[0183]

[0184] in, express The lateral position of the vehicle at any given time. express The longitudinal position of the vehicle at any given time. express The speed of the vehicle at any time express Vehicle heading angle at any time express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any moment This indicates ambient noise.

[0185] In this example, The pedestrian's maximum angular velocity is π rad / s (180° / s), and the vehicle's turning angular rate is ±π / 2 rad / s (90° / s). The agent's trajectory prediction at this time is as follows: Figure 2 As shown.

[0186] Furthermore, in the actual traffic environment of trajectory generation, right-turning vehicle-pedestrian interactions are often accompanied by perception errors, irregular pedestrian behavior, and sudden traffic interference. To address this characteristic, this application introduces a dual-path strategy generation mechanism in the trajectory generation stage based on the aforementioned trajectory generation method, while retaining the learned strategy. Historical reference strategy The system adaptively fuses data based on environmental uncertainty coefficients. When the traffic conditions are clear and the model confidence is high, the system prioritizes the learning strategy. However, when there is noise or changes in the environmental structure, the system automatically increases the weight of the reference strategy, thereby enhancing safety behaviors such as pedestrian avoidance and vehicle deceleration during the prediction process, which is in line with the "conservative decision-making" rule in actual traffic.

[0187] Step L7: Compare the predicted results with the actual trajectory, and evaluate and verify the accuracy of the KL-AIRL model's policy behavior based on specified indicators such as trajectory offset, acceleration deviation, and angular velocity offset.

[0188] The specific verification process at this point includes:

[0189] The results of policy transfer indices, including KL divergence, are used to evaluate changes in conflict rate, adaptability of avoidance behavior, and trajectory deviation after policy transfer. Policy transfer indices (such as KL divergence) can be used to analyze the learning effect and generalization ability of the policy, and are calculated using the following formula:

[0190]

[0191] in, As a strategy transferability index, For environmental quantity, In the environment The following strategy checks for all formatting issues throughout the entire document; in this example, parameter K=2.

[0192] The strategy transferability index here is specifically used for migration impact analysis. By calculating the KL divergence between strategy distributions under different traffic environments, it quantifies the degree of change of strategies under cross-intersection and cross-traffic flow conditions, and uses this to evaluate the traffic generalization ability and safety stability of the model in actual deployment. It can also characterize the impact of strategy migration on conflict rate, avoidance behavior and trajectory deviation from the perspective of traffic engineering, and provide quantitative basis for the application of the model under different intersection structures or traffic density conditions.

[0193] The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula:

[0194]

[0195] The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula:

[0196]

[0197] in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment;

[0198] The root mean square error is calculated to reflect the overall magnitude of the error, as follows:

[0199]

[0200] in, for Real-time acceleration for Predict acceleration at all times.

[0201] In this example, The training dataset was used to predict traffic trajectories using a hybrid approach combining MA-AIRL and Nash equilibrium. Based on the trajectory comparison results from step 7, the pedestrian trajectory index was determined as follows: average displacement error. The final displacement error is 0.1283m. The root mean square error is 0.3714m. The value is 0.079; vehicle trajectory index: final displacement error with average displacement error ADE = 0.2323m. The root mean square error is 0.2626m. It is 0.0221.

[0202] Based on the above example data, this method integrates adversarial reinforcement learning (AIRL) and NashQ-learning in the interaction trajectory prediction process, and introduces KL policy sampling and regularization mechanisms to constrain policy bias and improve the alignment of policy distribution with real behavior patterns, thereby effectively improving the model's performance in terms of prediction accuracy, behavior consistency and environment generalization ability.

[0203] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0204] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0205] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the method for predicting the interaction trajectory of right-turning vehicles and pedestrians at any of the intersections described in the above embodiments.

[0206] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0207] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0208] Memory, used to store computer programs;

[0209] The processor, when executing the program stored in memory, implements the aforementioned method for predicting the interaction trajectories of right-turning vehicles and pedestrians at intersections.

[0210] The communication bus mentioned in the above-mentioned electronic devices can be a standard bus for interconnecting peripheral components or an extended industrial standard structure bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0211] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0212] The memory may include random access memory or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0213] The processors mentioned above can be general-purpose processors, including central processing units, network processors, etc.; they can also be digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0214] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0215] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0216] Furthermore, it should be noted that if any directional indication (such as up, down, left, right, front, back, etc.) is involved in the embodiments of the present invention, the directional indication is only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0217] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, in the embodiments of this invention, "multiple" refers to two or more. Moreover, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

Claims

1. A method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection, characterized in that, Includes: S1. Obtain historical trajectory data and initial state information of multiple intelligent agents in a mixed traffic scenario, and construct an environment-coded state vector; S2. Input the historical trajectory data into the maximum entropy inverse reinforcement learning model to generate the maximum entropy probability distribution model of the trajectory, and learn the optimal reward function of the historical trajectory; S3. Constructing the KL-AIRL model to constrain the difference between the policy distribution and the expert distribution, using the optimal reward function of the maximum entropy probability distribution model; The S4.KL-AIRL model introduces the Nash Q-learning algorithm with multi-agent KL constraints to jointly optimize the policies of each agent, so as to achieve stable convergence and global optimal balance of the policies in the game interaction environment. S5. Combine the policy model trained by the KL-AIRL model with the initial state input behavior simulator, perform action sampling according to the fusion policy, and generate prediction results. S6. Compare the predicted results with the actual trajectory and verify the accuracy of the KL-AIRL model based on specified indicators; The specific optimization process of the Nash Q-learning algorithm for jointly optimizing the policies of each agent in step S4 includes: The formula for constructing multi-agent interaction is as follows: in, Represented as the current state Next Parameters of the reward function for each agent Influenced multi-agent joint actions The evaluation Q value, A collection of intelligent agents, For the first Actions of an agent The space of motion For the first The Q function of an agent, Defined in state Below, the reward parameters The influence of multi-agent local game relationships. For reward parameters, This is the current state. For joint action; Based on current reward parameters and local game Solve for a regularized Nash equilibrium hybrid strategy. ,in The regularized Nash equilibrium strategy represents the state of the agent in the next moment after performing an action. Based on current reward parameters It is confirmed that the following conditions exist: in, This indicates that when other agents all adopt their optimal strategies... At that time, intelligent agent The expected rewards that can be obtained For intelligent agents In state and actions Q value under, The optimal strategy for other agents For intelligent agents Reference strategy, For the optimal action combination of other agents under Nash equilibrium, For intelligent agents Candidate strategies, The regularization coefficient is . This indicates the calculation of the KL divergence value; state Initial expected cumulative return The calculation is as follows: in, For joint operations Expectations To implement the Nash equilibrium mixing strategy The action of performing a sampling operation To the current reward parameters The following is the set of optimal strategies for each agent obtained through game theory. For intelligent agents Q value; Next, the Q-value function is updated using the Bellman formula, as follows: in, The weight parameters that control the update intensity The weighting coefficients for the environmental classification branches; Finally, the reward parameters are updated based on the likelihood function of the historical demonstration trajectories. ,have: in, This is a collection of historical demonstration trajectories, serving as a database of real expert trajectory data samples used for model training. For trajectory The probability of represents the probability under the current reward parameter. Below, specific historical trajectories were observed. The possibility, For parameters Find the partial derivative. Here, is the learning rate parameter, and is a hyperparameter controlling the update step size. in The update is shown in the following equation: in, The average KL divergence among strategies. For environmental quantity, In the environment The following strategy.

2. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 1, characterized in that, In step S1, the environment coding state vector is set as follows: ,in, For inclusion An environment-coded state vector for each environment-coded state. , These represent the horizontal and vertical dimensions of the agent's planar coordinates, respectively. , , These represent the agent's velocity, acceleration, and direction of movement angle and angular velocity, respectively. It represents scene attributes and uses one-hot encoding.

3. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 2, characterized in that, The calculation formula for constructing the maximum entropy inverse reinforcement learning model in step S2 includes: in, For the reward parameter is Time, trajectory The probability of; Indicates the state ,action and environment The reward below, Indicates the first each state and this state Corresponding actions and environment The reward below, For trajectory The cumulative reward represents the optimal behavior in each state. Next action The total of the rewards Initial state The probability of the trajectory changing from state is represented by the probability of the trajectory changing from state . The possibility of starting The product of the probabilities of state transitions at each step in the trajectory represents the probability of performing an action. After state Transition to state The possibility; Here, is a normalization constant used to ensure that the sum of the probabilities of all possible trajectories is 1. Its calculation formula is as follows: Among them, the reward function Used to reflect the quality of a trajectory, determined by parameters. Define and update using stochastic gradient ascent. The updated calculation formula is as follows: in, It is a collection of historical trajectories. It is the learning rate parameter. It is a parameter The gradient.

4. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 3, characterized in that, In step S3, the discriminator is used to maximize the probability through a function, and its specific operation process includes: in, Represented as the mathematical expectation of the historical trajectory, This is expressed as the mathematical expectation of the predicted trajectory. The total time step, For historical strategy distribution, Step size For the state Next action Discriminator calculation, existence discriminator calculation As shown below: in, Step size For the state Next action The reward function, Feature vectors for environmental classification (in italics) The weight matrix for environmental classification. These are the weighting coefficients for the environmental classification branches. Step size The action distribution predicted by the policy network is determined by the sampling weights. To calculate the validity Sampling weight Using KL divergence fusion, the calculation is as follows: For the policy network learning rate, In strategy The expectations below The regularization coefficient is . This refers to calculating the KL divergence value, specifically measuring the current learning strategy. Historical reference strategy Indicators of distribution differences between them As a reference strategy in state Select action The probability, For policy network parameters Finding partial derivatives and dynamic reference strategies The update is as follows: in, For smoothing coefficients, This is the current strategy.

5. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 4, characterized in that, In step S5, a dual-path trajectory prediction mechanism is introduced during the motion sampling process of the simulator. The specific execution flow of this mechanism is as follows: When the system starts generating the predicted trajectory, it simultaneously invokes the trained policy network. Reference Strategy And construct the final action strategy through an environment-adaptive fusion formula. The fusion weights of the two strategies are determined by the environmental uncertainty coefficient. Dynamic adjustment: When the environment is known or the model has high confidence, use The output; When environmental noise, unobservable factors, or changes in interaction structure exceed a specified threshold are present, the reference strategy is improved. Dependency; After completing the strategy fusion, the system according to Motion sampling is performed, and the sampling results are input into the physics engine or dynamics model to update the state. This process is repeated until the complete trajectory is generated, as shown in the formula: in, In time step The chosen action As a fusion strategy, This represents the environmental uncertainty coefficient.

6. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 4, characterized in that, The specific operational process of the simulator performing motion sampling and finally generating the prediction result in step S5 includes: Input the initial state of the agent ; Sampling action at each time step h ; Update the state via the physics engine. until a complete predicted trajectory is generated. , Total duration; The pedestrian locations have been updated as follows: The pedestrian speed updates are as follows: The pedestrian angular velocity is updated as follows: in, express The lateral position of pedestrians at all times express The vertical position of pedestrians at all times express The speed of pedestrians at all times express Pedestrian heading angle at all times express The angular velocity of the pedestrian at any given moment Indicates time difference; Indicates pedestrian acceleration; The vehicle locations have been updated as follows: The vehicle speed updates are as follows: The vehicle angular velocity is updated as follows: in, express The lateral position of the vehicle at any given time. express The longitudinal position of the vehicle at any given time. express The speed of the vehicle at any time express Vehicle heading angle at any time express The angular velocity of the vehicle at any given moment. Indicates time difference, express Vehicle acceleration at any moment This indicates ambient noise.

7. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 4, characterized in that, The verification process in step S6 includes: The KL divergence calculation results are used to evaluate the changes in conflict rate, adaptability of avoidance behavior, and degree of trajectory deviation after strategy migration. The calculation formula is as follows: in, As a strategy transferability index, For environmental quantity, In the environment The following strategy checks all formatting issues across the entire document.

8. The method for predicting the interaction trajectory of right-turning vehicles and pedestrians at an intersection as described in claim 7, characterized in that, The verification process in step S6 also includes: The average Euclidean distance between the actual trajectory and the predicted trajectory within the predicted time step is calculated as the average displacement error, as shown in the following formula: The average Euclidean distance between the actual trajectory and the predicted trajectory at the final position is calculated as the final displacement error, as shown in the following formula: in, For the number of test samples, To predict duration, and The first One sample in The predicted and actual positions at any given moment; The root mean square error is calculated to reflect the overall magnitude of the error, as follows: in, for Real-time acceleration for Predict acceleration at all times.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 8.

Images