Unmanned bus trajectory optimization method for entering and leaving station based on inverse reinforcement learning

Abstract

The application relates to an unmanned bus station entry and exit trajectory optimization method based on inverse reinforcement learning, which comprises the following steps: analyzing the constraints of bus station entry and exit behaviors, defining state space variables of the station entry and exit behaviors, traversing the state space variables, and generating a candidate behavior trajectory curve; a maximum entropy inverse reinforcement learning model is constructed, a reward function of the model is composed of a group of base functions, the base functions are designed from the driving danger degree, the driving efficiency, the violation punishment, the influence on surrounding vehicles and the comfort degree, so that the bus station entry and exit behaviors are safe, efficient and comfortable, and meanwhile, the coefficients of the base functions are determined through inverse reinforcement learning, so that the behaviors are humanized; the maximum entropy inverse reinforcement learning model is trained based on human driver demonstration trajectories; and the candidate behavior trajectory curve is evaluated based on the trained model, and the optimal trajectory is selected. Compared with the prior art, the application has the advantages of high safety, high efficiency and good comfort.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving behavior decision-making, and in particular to an unmanned bus entering and leaving station trajectory optimization method based on inverse reinforcement learning. BACKGROUND

[0002] Automatic driving is a popular topic widely studied in academia and industry in recent years. The automatic driving car can replace the human driver to drive, which can save a lot of human resources and cost, especially in the field of public buses. With the development of automatic driving, it is foreseeable that in the near future, there will be more and more automatic driving cars and human drivers driving cars sharing the road. Therefore, the driving behavior of the automatic driving car should be as similar as possible to the driving behavior of the human driver, so that the driving behavior of the automatic driving car can be predicted by the human driver, ensuring road safety and efficiency. For the unmanned bus, safety is the primary factor to be considered, and therefore the driving behavior of the unmanned bus should be as similar as possible to the driving behavior of the human driver.

[0003] With the development of artificial intelligence, there are currently two main methods for learning human-like driving behavior: imitation learning and inverse reinforcement learning. Imitation learning learns the decision-making strategy of human drivers during driving by imitating human demonstration actions, i.e. a series of driving behaviors. However, imitation learning usually has poor generalization because it directly learns specific behaviors. Inverse reinforcement learning learns the reward function inside the behavior decision-making through human demonstration actions, and then learns the optimal behavior strategy. Since the reward function is more transferable than directly learning specific behaviors, inverse reinforcement learning generally has stronger generalization than imitation learning. However, inverse reinforcement learning involves learning a reward function and selecting the optimal strategy under the reward function in the iterative process, which has high model complexity and requires large amounts of computation, posing challenges to inverse reinforcement learning for driving behavior simulation.

[0004] Due to its nature, the bus needs to frequently enter and leave the station. The entering and leaving station behavior involves interacting with other vehicles, stopping, starting, etc., which poses a great challenge to the safety, efficiency and comfort of the unmanned bus operation, and is a problem that needs to be solved for the realization of unmanned buses. SUMMARY

[0005] The purpose of the present application is to provide an unmanned bus entering and leaving station trajectory optimization method based on inverse reinforcement learning, which takes into account the impact of frequent entering and leaving station behavior, and realizes a trajectory planning scheme that meets the safety, efficiency and comfort of the unmanned bus entering and leaving the station.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] An autonomous bus station entry and exit trajectory optimization method based on inverse reinforcement learning, comprising the following steps:

[0008] S1, analyze the constraints of bus station entry and exit behavior, define the state space variables of station entry / exit behavior, traverse the state space variables, and generate candidate behavior trajectory curves;

[0009] S2, construct a maximum entropy inverse reinforcement learning model, the reward function of the maximum entropy inverse reinforcement learning model is composed of a group of basis functions, the basis functions are designed from the driving risk degree, driving efficiency, violation penalty, influence on surrounding vehicles and comfort, to ensure the safety, efficiency and comfort of bus station entry and exit behavior, and the coefficients of the basis functions are determined through inverse reinforcement learning to ensure human-like driving;

[0010] S3, train the maximum entropy inverse reinforcement learning model based on human driver demonstration trajectory;

[0011] S4, evaluate the candidate behavior trajectory curve based on the trained maximum entropy inverse reinforcement learning model, and select the optimal trajectory.

[0012] Further, the step S1 comprises the following steps:

[0013] S11, assuming that for the station entry behavior, the initial state of the vehicle is uniform linear motion; for the station exit behavior, the target state of the vehicle is uniform linear motion;

[0014] S12, decouple the longitudinal and lateral motion of the vehicle for analysis, and determine 6 constraints of the longitudinal and lateral motion of the station entry and exit behavior:

[0015] Station entry:

[0016] Station exit:

[0017] Wherein, x s , v s , y s are the longitudinal position, longitudinal speed and lateral position of the vehicle at the beginning of the station entry process, i.e. the initial state, T1 is the end time of the station entry process, T ys , T ye are the start and end times of the lateral motion, x p , y p are the station positions, x e , v e , y e are the longitudinal position, longitudinal speed and lateral position of the vehicle at the end of the station exit process;

[0018] S13, respectively use polynomials with respect to time to represent the longitudinal and lateral motion of the station entry and exit driving trajectory:

[0019] Inbound:

[0020] Outbound:

[0021] where x1(t), y1(t) are the longitudinal and lateral position of the vehicle at time t when entering the station, x2(t), y2(t) are the longitudinal and lateral position of the vehicle at time t when leaving the station;

[0022] S14, according to the polynomial expression of the longitudinal and lateral motion of the inbound and outbound driving trajectory, for the inbound behavior, there are only four variables x s , v s , T ys , T ye , for the outbound behavior, there are only four variables x e , v e , T ys , T ye , y s , y e are the centerline positions of the adjacent lanes of the station, which are fixed. Discretize the variables to obtain the state space, that is, the state space of the inbound behavior is represented as (x s , v s , T ys , T ye ), and the state space of the outbound behavior is represented as (x e , v e , T ys , T ye );

[0023] S15, traverse the state space to generate all candidate inbound and outbound behavior trajectory curves.

[0024] Further, the objective of the maximum entropy inverse reinforcement learning model is to maximize the likelihood of the trajectory demonstrated by the human driver under the trajectory distribution by adjusting the weights of the basis functions in the reward function.

[0025] Further, in the maximum entropy inverse reinforcement learning model, the basis functions for measuring the degree of driving danger include a collision time-based basis function f risk (t) and a collision penalty-based basis function f collision (t), wherein the collision time represents the time required for two vehicles to collide if the front and rear vehicles maintain a constant speed, and is represented as:

[0026]

[0027] where d is the relative distance between the front and rear vehicles, v f , v rrespectively represent the speed of the front and rear vehicles; when the collision time is negative, the rear vehicle speed is smaller than the front vehicle speed, and the rear-end risk is smaller; when the collision time is positive, the value is closer to 0, and the rear-end risk is greater;

[0028] According to the characteristics of the collision time, the base function based on the collision time is determined as:

[0029]

[0030] wherein d(t) represents the relative distance between the target vehicle and the front vehicle at time t, v ego (t) and v f (t) represent the speed of the target vehicle and the front vehicle at time t, respectively; the greater the rear-end risk, the closer the value of the base function based on the collision time to 1, and vice versa, the closer to 0.

[0031] Considering that the generated candidate trajectory may collide with surrounding vehicles or road facilities, a base function based on collision penalty is constructed as:

[0032]

[0033] Further, in the maximum entropy inverse reinforcement learning model, the base function f efficiency (t) is constructed based on the longitudinal speed of the target vehicle:

[0034] f efficiency (t) = v xego (t)

[0035] wherein v xego (t) represents the longitudinal speed of the target vehicle.

[0036] Further, in the maximum entropy inverse reinforcement learning model, the base function f limit (t) is constructed based on the speed limit:

[0037]

[0038] wherein v limit represents the road speed limit.

[0039] Further, in the maximum entropy inverse reinforcement learning model, the base function f interaction (t) is constructed based on the longitudinal deceleration of the affected same-lane vehicle:

[0040]

[0041] wherein a xi represents the longitudinal deceleration of the affected same-lane rear vehicle i.

[0042] Further, in the maximum entropy inverse reinforcement learning model, the base function f measuring driving comfort comfort Based on the acceleration change rate, the longitudinal and lateral motions are decoupled, the longitudinal motion is measured by the absolute value of the longitudinal acceleration change rate, and the lateral motion is measured by the absolute value of the lateral acceleration change rate, that is:

[0043]

[0044]

[0045] wherein, represents the longitudinal acceleration change rate, represents the lateral acceleration change rate.

[0046] Further, the maximum entropy inverse reinforcement learning model adopts the gradient descent method to train the coefficients of the base function in the reward function, and the objective function is:

[0047]

[0048] r(τ)=ω T f(τ)

[0049] wherein, τ is a demonstration trajectory of a human driver, is a candidate trajectory, N is the total number of candidate trajectories, r(τ) is a reward function, f(τ) represents a base function group, ω is a coefficient vector of the base function group; before training, the value of each base function at each time is normalized to avoid the influence of different numerical ranges of the base functions on the coefficient learning.

[0050] Further, in the step S4, for the selection of the optimal trajectory in the candidate entry / exit trajectory, the reward function learned by the maximum entropy inverse reinforcement learning model is substituted to obtain the reward of each trajectory, and the probability of selecting the candidate trajectory is proportional to the exponential of the reward obtained by the candidate trajectory, that is:

[0051]

[0052] wherein, P(τ) represents the probability of selecting the candidate trajectory τ, r(τ) represents the reward obtained by the candidate trajectory τ, and N is the total number of candidate trajectories;

[0053] The candidate trajectory with the maximum probability P(τ) is the finally determined optimal trajectory.

[0054] Compared with the prior art, the present application has the following beneficial effects:

[0055] (1) The application realizes safe, efficient and comfortable bus station entry and exit behavior decision-making of an unmanned bus through a set of base functions designed from the perspectives of driving risk level, driving efficiency, rule violation punishment, influence on surrounding vehicles and comfort, and maximum entropy inverse reinforcement learning.

[0056] (2) The application learns base function coefficients instead of base functions themselves, designs base functions through mechanism knowledge, and makes the method highly explainable, thereby providing credibility for application in a vehicle terminal.

[0057] (3) The application generates candidate station entry and exit trajectories, replaces the process of finding an optimal strategy in an iteration process of inverse reinforcement learning, greatly reduces model complexity and computation, and provides a possibility for application in a vehicle terminal. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A method flowchart of the application.

[0059] Figure 2 A bus station entry and exit trajectory generation schematic diagram in an embodiment.

[0060] Figure 3 A bus station entry trajectory analysis diagram in an embodiment. DETAILED DESCRIPTION

[0061] The application will be described in detail below with reference to the drawings and specific embodiments. The embodiments are implemented on the premise of the technical solution of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0062] The embodiment assumes that the steps of bus driver making station entry and exit behavior decision-making are as follows: first, generating candidate station entry and exit trajectories; second, evaluating the reasonableness of the generated trajectories; and third, selecting a station entry and exit trajectory to follow. To simulate the station entry and exit behavior of a human driver, the simulated station entry and exit behavior is also divided into the three steps of generation, evaluation and selection, and the station entry and exit behavior is divided into two behaviors of station entry and station exit, and the parking time is regarded as fixed. Since the simulation methods of station entry and station exit behaviors are consistent, the following will not be distinguished.

[0063] The embodiment provides an unmanned bus station entry and exit trajectory optimization method based on inverse reinforcement learning, as shown in Figure 1 The method comprises the following steps:

[0064] S1, analyze the constraints of bus station entry and exit behavior, define state space variables of station entry / exit behavior, traverse the state space variables, and generate candidate behavior trajectory curves.

[0065] To apply the maximum entropy inverse reinforcement learning, the embodiment discretizes the continuous entering and exiting behaviors into discrete behaviors. Meanwhile, it assumes that the roads before and after entering and exiting are straight roads.

[0066] For the generation of candidate entering and exiting trajectories, the embodiment generates candidate lane-changing trajectories according to the current vehicle initial state and the vehicle target state. For the entering behavior, the vehicle target state is determined (bus stop), while the vehicle initial state is uncertain, i.e., what position to start the entering behavior is uncertain. While for the exiting behavior, the vehicle initial state is determined (bus stop), while the vehicle target state is uncertain, i.e., what position to end the entering behavior is uncertain. The candidate entering and exiting trajectories can finally be represented by a polynomial curve, which should be smooth and dynamically feasible, as shown in Figure 2 .

[0067] Specifically, the following steps are included:

[0068] S11, when a human driver stably drives on a straight road, the vehicle motion is basically uniform linear motion. Therefore, the embodiment assumes that for the entering behavior, the vehicle initial state is uniform linear motion; for the exiting behavior, the vehicle target state is uniform linear motion. To facilitate trajectory generation, the vehicle longitudinal motion and lateral motion are decoupled and analyzed, respectively represented by a polynomial about time.

[0069] S12, determine the 6 constraints of the longitudinal and lateral motions of the entering and exiting behaviors:

[0070] Entering:

[0071] Exiting:

[0072] where x s , v s , y s are the longitudinal position, longitudinal velocity and lateral position of the vehicle at the beginning of the entering process, i.e., the initial state; T1 is the end time of the entering process, which is a constant; T ys , T ye are the start and end times of the lateral motion (the entering and exiting behaviors do not need to be the same); x p , y p are the stop positions; x e , v e , y e are the longitudinal position, longitudinal velocity and lateral position of the vehicle at the end of the exiting process.

[0073] S13, represent the longitudinal and lateral motions of the entering and exiting driving trajectories by a 5th order polynomial about time, respectively:

[0074] Entering:

[0075] Outbound:

[0076] Wherein, x1(t), y1(t) are the longitudinal position and transverse position of the vehicle entering the station at time t respectively, x2(t), y2(t) are the longitudinal position and transverse position of the vehicle leaving the station at time t.

[0077] The entry and exit station trajectory analysis diagram is shown in Figure 3 From the diagram, it can be seen that the entry / exit is not a longitudinal movement throughout the journey, which also conforms to the entry and exit behavior of human drivers.

[0078] S14, from the above assumptions, for the entry behavior, there are only four variables x s , v s , T ys , T ye (y s fixed as the station adjacent lane center line position), for the exit behavior, there are also only four variables x e , v e , T ys , T ye (y p fixed as the station adjacent lane center line position). The variables can be determined according to the calculation amount and simulation accuracy to determine the number after discretization, and the larger number is conducive to the simulation accuracy, but also increases the calculation amount. After discretizing the variables, the state space can be obtained, that is, the state space of the entry behavior is represented as (x s , v s , T ys , T ye ), and the state space of the exit behavior is represented as (x e , v e , T ys , T ye ).

[0079] S15, traverse the state space to generate all candidate entry and exit behavior trajectory curves.

[0080] S2, construct a maximum entropy inverse reinforcement learning model, the reward function of the maximum entropy inverse reinforcement learning model is composed of a group of basis functions, and the basis functions are designed from the driving risk degree, driving efficiency, violation penalty, influence on surrounding vehicles and comfort angle to ensure the safety, efficiency and comfort of the bus entry and exit station behavior, and the coefficients of the basis functions are determined by inverse reinforcement learning to ensure the human-like driving.

[0081] The maximum entropy inverse reinforcement learning is adopted to learn the intrinsic driving reward function of human drivers from the demonstration trajectories of human drivers. The maximum entropy inverse reinforcement learning assumes that the reward function is a linear combination of a set of basis functions. The set of basis functions is constructed according to the mechanism knowledge. The goal of the maximum entropy inverse reinforcement learning is to maximize the likelihood of the trajectories demonstrated by human drivers under the trajectory distribution by adjusting the weights of the basis functions in the reward function. The maximum entropy inverse reinforcement learning is a mechanism combined with data model. Therefore, the selection of the basis functions is important for the maximum entropy inverse reinforcement learning. The embodiment designs a set of basis functions from the driving risk level, efficiency, violation penalty, influence on surrounding vehicles and comfort.

[0082] The reward function of the maximum entropy inverse reinforcement learning model is:

[0083] r(τ)=ω T f(τ)

[0084] wherein τ represents the inbound / outbound trajectory, r(τ) represents the reward of the trajectory, f represents a set of basis functions, ω is a coefficient vector of the basis functions (to-be-learned parameters),

[0085] f=(f risk ,f collision ,f efficiency ,f limit ,f interaction ,f comfortable )

[0086] (1) The basis function for measuring the driving risk level

[0087] The basis function for measuring the driving risk level includes the basis function f risk (t) based on the collision time and the basis function f collision (t) based on the collision penalty.

[0088] When driving, the human driver maintains a safe distance from the front vehicle to avoid dangerous situations such as rear-end collision. When entering or leaving the station, the interaction with other buses may be encountered, and it is very important to avoid rear-end collision. The collision time represents the time required for the front and rear vehicles to maintain uniform speed and collide, which is commonly used to evaluate the risk of rear-end collision, and is defined as:

[0089]

[0090] wherein d is the relative distance between the front and rear vehicles, v f and v r represent the speeds of the front and rear vehicles respectively; when the collision time is negative, the rear vehicle speed is smaller than the front vehicle speed, and the risk of rear-end collision is smaller; when the collision time is positive, the value is closer to 0, and the risk of rear-end collision is greater.

[0091] According to this feature of the collision time, the collision time-based basis function is determined as:

[0092]

[0093] where d(t) represents the relative distance between the target vehicle and the preceding vehicle at time t, v ego (t) and v f (t) represent the speeds of the target vehicle and the preceding vehicle at time t, respectively; the greater the rear-end risk, the closer the value of the collision time-based basis function to 1, and vice versa, the closer to 0.

[0094] At the same time, since the generated candidate trajectory may collide with surrounding vehicles or road facilities, a collision penalty-based basis function is introduced as:

[0095]

[0096] (2) Basis function for measuring driving efficiency

[0097] In most cases, the bus driver will complete the in-and-out station behavior as soon as possible within the safe driving range to improve the driving efficiency as much as possible. In this embodiment, the longitudinal speed of the target vehicle is used to measure the driving efficiency of the target vehicle, that is, the basis function f efficiency (t) for measuring driving efficiency is:

[0098] f efficiency (t) = v xego (t)

[0099] where v xego (t) represents the longitudinal speed of the target vehicle.

[0100] (3) Basis function for measuring violation penalty

[0101] Most buses have company-regulated speed limits, and even if there is no company-regulated speed limit, there is a speed limit on the road. Given the speed limit v limit , the bus should comply with this rule, that is, the basis function f limit (t) for measuring violation penalty is:

[0102]

[0103] (4) Basis function for measuring the impact on surrounding vehicles

[0104] The human driver will try to avoid the impact of his own behavior on other drivers in the entry and exit behavior to avoid dangerous situations or affect road efficiency. Specifically, avoid the situation of reducing the speed of the rear vehicle in the same lane. The embodiment uses the longitudinal deceleration of the affected vehicle in the same lane to represent the impact of the entry and exit behavior on the surrounding vehicles. Among them, the surrounding vehicles only include the rear vehicle in the same lane. Then, the base function f interaction (t) is:

[0105]

[0106] Where a xi represents the longitudinal deceleration of the affected rear vehicle i in the same lane.

[0107] (5) Base function for measuring driving comfort

[0108] The entry and exit behavior has a great impact on driving comfort because it involves acceleration and deceleration, so driving comfort is an important factor that needs to be considered in entry and exit behavior. The human driver will try to avoid uncomfortable driving behavior. Jerk is the rate of change of acceleration, which is an important indicator of driving comfort. When formulating the base function for measuring driving comfort, the longitudinal motion and the lateral motion are decoupled, the longitudinal motion is measured by the absolute value of the longitudinal jerk, and the lateral motion is measured by the absolute value of the lateral jerk, that is: The base function for measuring driving comfort f comfort is represented as:

[0109]

[0110]

[0111] Where, represents the longitudinal jerk, represents the lateral jerk.

[0112] The maximum entropy inverse reinforcement learning model uses gradient descent to train the coefficients of the base functions in the reward function, and the objective function is:

[0113]

[0114]

[0115] r(τ)=ω T f(τ)

[0116] Where τ is the demonstration trajectory of the human driver, wherein, P(τ) represents the probability of the candidate trajectory τ being selected, r(τ) represents the reward obtained by the candidate trajectory τ, and N is the total number of candidate trajectories; the candidate trajectory with the maximum probability P(τ) is the finally determined optimal trajectory.

[0117] S3, training a reward function in the maximum entropy inverse reinforcement learning model based on the human driver demonstration trajectory.

[0118] S4, evaluating the candidate behavior trajectory curve based on the trained maximum entropy inverse reinforcement learning model, and selecting an optimal trajectory.

[0119] For the evaluation of the candidate entry / exit station trajectory, the demonstration entry / exit station trajectory of the human driver is learned through the maximum entropy inverse reinforcement learning, so as to obtain the reward function, and the evaluation of the driving trajectory is realized based on the reward function.

[0120] For the selection of the optimal trajectory in the candidate entry / exit station trajectory, the reward of each trajectory is obtained by substituting the reward function learned by the maximum entropy inverse reinforcement learning model, and the probability of the candidate trajectory being selected is proportional to the exponential of the reward obtained by the candidate trajectory, that is:

[0121]

[0122] wherein, P(τ) represents the probability of the candidate trajectory τ being selected, r(τ) represents the reward obtained by the candidate trajectory τ, and N is the total number of candidate trajectories; the candidate trajectory with the maximum probability P(τ) is the finally determined optimal trajectory.

[0123] The application discloses an unmanned bus entry / exit station trajectory optimization method based on inverse reinforcement learning. The application divides the entry / exit station behavior into entry station behavior and exit station behavior for modeling. Firstly, the initial state space and the target state space of the entry / exit station behavior are analyzed to generate a candidate behavior trajectory. Then, the maximum entropy inverse reinforcement learning is used to learn a reward function from the entry / exit station behavior of a human driver, to evaluate the candidate behavior trajectory, to select a behavior trajectory with the maximum reward, and to realize entry / exit station behavior decision. The base function of the reward function is designed from five aspects of driving danger degree, efficiency, rule violation punishment, influence on surrounding vehicles and comfort, and can ensure the safety, driving efficiency and comfort of the bus entry / exit station.

[0124] The embodiment learns the base function coefficient instead of the base function itself, designs the base function through mechanism knowledge, so that the method has strong interpretability, and provides credibility for application in the vehicle end. Meanwhile, the embodiment generates the candidate entry / exit station trajectory, replaces the process of finding an optimal strategy in the iteration process of the inverse reinforcement learning, and greatly reduces the model complexity and the calculation amount.

[0125] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art in light of the teachings above without departing from the spirit of the present application. It is therefore to be understood that what is desired to be protected by letters patent is defined by the scope of the claims below and that on the basis of the teachings of the present application, obvious modifications and equivalents can be adopted by those skilled in the art in their possession of the teachings of the present application without departing from the spirit and scope of the application.

Claims

1. A method for optimizing an autonomous bus trajectory for entering and leaving a station based on inverse reinforcement learning, characterized in that, The method comprises the following steps: S1, analyzing the constraints of bus entry and exit station behaviors, defining state space variables of entry and exit station behaviors, traversing the state space variables, and generating candidate behavior trajectory curves; S2, constructing a maximum entropy inverse reinforcement learning model, wherein a reward function of the maximum entropy inverse reinforcement learning model is composed of a group of basis functions, the basis functions are designed from the perspectives of driving risk degree, driving efficiency, rule violation punishment, influence on surrounding vehicles and comfort, so as to ensure that the bus entry and exit station behaviors are safe, efficient and comfortable, and coefficients of the basis functions are determined through inverse reinforcement learning, so as to ensure human-like driving; S3, training the maximum entropy inverse reinforcement learning model based on human driver demonstration trajectories; S4, evaluating the candidate behavior trajectory curves based on the trained maximum entropy inverse reinforcement learning model, and selecting an optimal trajectory.

2. The method of claim 1, wherein, The step S1 comprises the following steps: S11, assuming that, for entry station behaviors, the initial state of the vehicle is uniform linear motion, and for exit station behaviors, the target state of the vehicle is uniform linear motion; S12, decoupling and analyzing the longitudinal motion and lateral motion of the vehicle, and determining six constraints of the longitudinal and lateral motions of the entry and exit station behaviors; Inbound: Outbound: wherein x s , v s , y s are the longitudinal position, longitudinal speed and lateral position of the vehicle at the beginning of the entry process, i.e. the initial state, T1 is the end of the entry process, T ys , T ye are the start and end of the lateral movement, x p , y p are the station positions, x e , v e , y e are the longitudinal position, longitudinal speed and lateral position of the vehicle at the end of the exit process. S13, respectively representing the longitudinal and lateral motions of the entry and exit station driving trajectories by polynomials with respect to time: Inbound: Outbound: wherein x1(t) and y1(t) are respectively the longitudinal position and lateral position of the vehicle entering the station at time t, and x2(t) and y2(t) are respectively the longitudinal position and lateral position of the vehicle exiting the station at time t; S14, according to the polynomial expression of the longitudinal and lateral motion of the entry and exit driving track, for the entry behavior, there are only four variables x s , s , ys , ye , for the exit behavior, there are only four variables x e , e , ys , ye , y s , e , the center line position of the adjacent lane of the station, the state space is obtained by discretizing the variables, that is, the state space expression of the entry behavior is (x s , v s , T ys , T te ), and the state space expression of the exit behavior is (x e , v e , T ys , T ye ); S15, traversing the state space to generate all candidate entry and exit station behavior trajectory curves.

3. The method of claim 1, wherein, The maximum entropy inverse reinforcement learning model aims to maximize the likelihood of the trajectories demonstrated by human drivers under the trajectory distribution by adjusting the weights of the basis functions in the reward function.

4. The method of claim 1, wherein, In the maximum entropy inverse reinforcement learning model, the base function for measuring the driving risk degree includes a base function f risk (t) based on collision time collision (t) based on collision penalty, where the collision time represents the time required for the two vehicles to collide if the front and rear vehicles maintain uniform speed driving, and is expressed as: where d is the relative distance between the front and rear vehicles, v f , v r represent the speed of the front and rear vehicles respectively; when the collision time is negative, the rear vehicle speed is smaller than the front vehicle speed, and the rear-end risk is smaller; when the collision time is positive, the value is closer to 0, and the rear-end risk is greater; According to the characteristics of the collision time, the basis function based on the collision time is determined as: Where d(t) represents the relative distance between the target vehicle and the vehicle in front at time t, and v ego (t), v f (t) represents the speeds of the target vehicle and the vehicle in front at time t, respectively; the greater the risk of a rear-end collision, the closer the value of the basis function based on the collision time is to 1, and vice versa, the safer it is, the closer it is to 0; Considering that the generated candidate trajectory may collide with surrounding vehicles or road facilities, a basis function based on collision punishment is constructed as:

5. The method of claim 1, wherein, In the maximum entropy inverse reinforcement learning model, the base function f measuring driving efficiency efficiency (t) is constructed based on the longitudinal speed of the target vehicle: f efficiency (t) = v xego (t) where v xego (t) denotes the target vehicle longitudinal speed.

6. The method of claim 1, wherein, In the maximum entropy inverse reinforcement learning model, the base function f measuring the violation penalty limit (t) Construction based on speed limit: where v limit represents the road speed limit.

7. The method of claim 1, wherein, In the maximum entropy inverse reinforcement learning model, the base function f measuring the influence on surrounding vehicles interaction (t) constructing based on the longitudinal deceleration of the affected same-lane vehicle: where a xi denotes the longitudinal deceleration of the affected same-lane vehicle behind vehicle i.

8. The method of claim 1, wherein, In the maximum entropy inverse reinforcement learning model, the base function f measuring driving comfort comfort Based on the acceleration change rate, the longitudinal and lateral motions are decoupled, the longitudinal motion is measured by the absolute value of the longitudinal acceleration change rate, and the lateral motion is measured by the absolute value of the lateral acceleration change rate, that is: wherein denotes the longitudinal acceleration rate of change, denotes the lateral acceleration rate of change.

9. The method of claim 1, wherein, The maximum entropy inverse reinforcement learning model adopts the gradient descent method to train the coefficients of the basis functions in the reward function, and the objective function is: r(τ) = ω T f(τ) where τ is the demonstrated trajectory of a human driver, is the candidate trajectory, N is the total number of candidate trajectories, r(τ) is the reward function, f(τ) represents the basis function set, and ω is the coefficient vector of the basis function set; before training, the value of each basis function at each time is normalized to avoid the influence of different numerical ranges of basis functions on coefficient learning.

10. The method of claim 1, wherein, In the step S4, for the selection of the optimal trajectory in the candidate entry and exit station trajectory, the reward of each trajectory is obtained by substituting the reward function learned by the maximum entropy inverse reinforcement learning model, and the probability of the selection of the candidate trajectory is proportional to the exponential of the reward obtained by the candidate trajectory, that is: wherein P(τ) represents the probability of the selection of the candidate trajectory τ, r(τ) represents the reward obtained by the candidate trajectory τ, and N is the total number of candidate trajectories; The candidate trajectory with the maximum probability P(τ) is the finally determined optimal trajectory.

Images