A method and system for dissipating shock waves of overlapping traffic on expressways in a vehicle-road cooperative environment

Abstract

The present invention discloses a method and system for dissipating overlapping traffic shock waves on fast roads in a vehicle-road cooperative environment. A traffic shock wave model based on shock wave theory is established to calculate the optimal time for an intelligent networked vehicle to arrive at an intersection. Then, a vehicle trajectory optimization controller based on a deep reinforcement learning model is used to construct a networked autonomous driving vehicle intelligent body, and the optimal vehicle trajectory optimization controller is trained to control the intelligent networked vehicle. The present invention uses a deep reinforcement learning framework to perform multi-objective trajectory optimization control on networked autonomous driving vehicles, guides upstream traffic to slowly enter the influence range of traffic shock waves with target trajectories, and thereby suppresses the propagation of traffic shock waves upstream. The present invention can reduce the queuing and parking time of vehicles at intersections, is conducive to improving the overall traffic efficiency of urban trunk roads, and reduces the potential risk of traffic accidents and fuel consumption at intersections. It has practical engineering application value and significance in alleviating traffic congestion problems at urban intersections.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent traffic management and control, and specifically relates to a method and system for dissipating shock waves of overlapping traffic on a fast road in a vehicle-road cooperative environment. Background Art

[0002] Traffic shock waves refer to the stop-and-go phenomenon, which is the process of vehicles repeatedly slowing down, stopping and starting. This phenomenon generally occurs in unstable traffic environments and continues to spread to upstream traffic flows, easily developing into serious traffic congestion, which has an adverse impact on traffic efficiency, safety and energy efficiency. Therefore, dissipating traffic shock waves is a key task in solving traffic congestion problems.

[0003] Signalized intersections are one of the main causes of urban traffic congestion. The red light phase causes periodic traffic oscillations, causing vehicles to queue up at the entrance. At present, the main solution to signalized intersection congestion is to optimize the control signal strategy, which reduces the vehicle queue backlog by improving the utilization rate of the green light phase, such as traffic green wave bands, multi-scheme dynamic timing, and adaptive signal timing. However, this type of control method based on macro traffic parameters has low accuracy. On the one hand, the control points are located at discrete intersections, and the spatial resolution of the control output is low; on the other hand, the driver's random driving behavior is difficult to achieve the optimal control effect of the strategy. Congestion absorption driving strategy can achieve traffic shock wave dissipation based on refined vehicle trajectories to solve the above problems. This strategy avoids more vehicles from falling into the downstream traffic shock wave by specifying vehicles to perform "slow entry" and "fast exit" behaviors, thereby preventing the shock wave from propagating upstream.

[0004] With the development of vehicle networking technology and autonomous driving technology, refined control of vehicle trajectories has become possible. In the process of congestion dissipation, vehicle networking technology provides autonomous driving vehicles with continuous, stable and beyond-visual-range traffic status information. Based on rich traffic status information, networked autonomous driving vehicles can plan more effective traffic shock wave dissipation trajectories. At the same time, autonomous driving vehicles have a highly controllable central control system that can help vehicles accurately execute dissipation trajectories. The driving behavior of autonomous driving vehicles further constrains the following behavior of conventional human-driven vehicles, thereby improving the overall traffic congestion problem. Therefore, the congestion absorption driving strategy based on the networked autonomous driving environment has broad application prospects for solving urban traffic congestion. Summary of the invention

[0005] The purpose of the present invention is to propose a method and system for dissipating overlapping traffic shock waves on fast roads in a vehicle-road cooperative environment. Based on the prediction of overlapping traffic shock waves at intersections, networked autonomous driving vehicles execute congestion absorption driving strategies to suppress the propagation of traffic shock waves, thereby reducing the risk of traffic accidents and traffic congestion caused by oscillation waves.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for dissipating overlapping traffic shock waves on a fast road in a vehicle-road cooperative environment, comprising the following steps:

[0007] S1. Collect environmental data of the target road section, including traffic signal system data and vehicle operation data, and then generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems;

[0008] S2. Constructing a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculating the traffic prediction state, that is, the optimal time for the intelligent networked vehicle to arrive at the intersection; specifically, the following sub-steps are included:

[0009] S201. Construct a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles;

[0010] S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time t P,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows:

[0011]

[0012] In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed;

[0013] S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is:

[0014]

[0015] x F,n =x light -(ni)×s

[0016]

[0017]

[0018] Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles;

[0019] S3, the vehicle trajectory optimization controller based on the deep reinforcement learning model builds the networked autonomous driving vehicle intelligent body, specifically: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the agent optimization objectives include: vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; the reinforcement learning controller includes the actor neural network model and the critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model;

[0020] S4. Construct the historical experience data set of the intelligent agent, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

[0021] Furthermore, in the aforementioned step S1, the traffic signal system data includes: the signal strategy of the intersection, the current signal phase information, and the upstream vehicle information; the vehicle operation data includes the operation data information of the own vehicle and the preceding vehicle.

[0022] Furthermore, in the aforementioned step S3, the traffic signal status includes the distance between the vehicle and the intersection, the signal cycle, the current phase, and the remaining time of the current phase; the traffic prediction status is the optimal time for the vehicle to arrive at the intersection; the leading vehicle status is the leading vehicle speed; and the own vehicle status is the own vehicle speed and the distance to the leading vehicle.

[0023] Furthermore, in the aforementioned step S3, the optimization objective is expressed as an action reward function, including the speed reward r speed 、Safety Reward safe , time reward time , acceleration reward r a , as follows:

[0024] (1) Speed ​​reward r speed is the vehicle speed of the connected autonomous vehicle n at time t The corresponding reward is within the speed limit range [v min ,v max ]Inside, The larger the r speed The higher the value, the calculation is as follows:

[0025]

[0026] (2) Safety Reward saf e Based on the distance between the connected autonomous driving vehicle n and its preceding vehicle n-1 at time t Whether it meets the expected safety distance Calculate as follows:

[0027]

[0028] In the formula, a max is the maximum acceleration of the connected autonomous driving vehicle; τ is the system delay time; is the distance between vehicle n and vehicle n-1;

[0029] (3) Time reward r time It is used to encourage vehicles to avoid reaching the intersection stop line during the red light phase and not be affected by traffic shock waves, as shown in the following formula:

[0030] r time =r p +r g

[0031]

[0032] In the formula, r p , r g They are the traffic shock wave reward and the signal phase reward; Maintain the current speed for vehicle n The time it takes to reach the intersection; is the distance between vehicle n and the intersection at time t; T c is the traffic signal cycle; t g_start , t y_end The green light start time and the yellow light end time are respectively;

[0033] (4) Acceleration reward r a is the acceleration at time t The corresponding reward value is used to ensure the smoothness of vehicle driving and is calculated as follows:

[0034] r a =-(a n t / a max ) 2

[0035] (5) The total reward value is as follows:

[0036]

[0037] Where ω1, ω2, ω3, and ω4 are reward weights.

[0038] Furthermore, in the aforementioned step S4, constructing the agent historical experience data set specifically includes: the agent independently interacts with the environment in the simulation to obtain experience data And store them in the experience data set. The specific process of obtaining an experience data is as follows: Agent n obtains the environment state at time t Based on status Output the acceleration action of agent n through the trajectory optimization model The action is executed and the simulation runs to time t+1, and the environment state is updated to By status Get action rewards

[0039] Furthermore, in the aforementioned step S4, a reinforcement learning controller is trained with a historical experience data set, specifically: a single intersection environment is initialized on a simulation platform, including road section length, number of lanes, speed limit, flow, traffic detector location, traffic signal strategy, and vehicle model parameters. The agent runs in the simulation environment and collects experience data to construct a historical experience data set for the agent. In each round of training, a fixed number of experience data are randomly selected from the experience data set to construct a training set. The policy gradient and valuation error are calculated and the control model is updated until the error converges to obtain the optimal vehicle trajectory optimization controller.

[0040] The present invention also provides a system for dissipating overlapping traffic shock waves on a fast road in a vehicle-road cooperative environment, comprising:

[0041] Data information acquisition module, vehicle communication module, and vehicle control module;

[0042] The data information acquisition module is used to collect environmental data of the target road section; the data information acquisition module includes a road-side data acquisition unit and a vehicle-side data acquisition unit; the road-side data acquisition unit is used to collect traffic signal system data, and the vehicle-side data acquisition unit is used to collect vehicle operation data.

[0043] A vehicle communication module, used to generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems;

[0044] The vehicle control module includes a traffic shock wave predictor and an autonomous vehicle trajectory controller;

[0045] The traffic shock wave predictor is used to construct a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculate the optimal time for the intelligent networked vehicle to arrive at the intersection; the traffic shock wave predictor is configured to perform the following actions, including steps S201 to S203:

[0046] S201. Construct a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles;

[0047] S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time t P,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows:

[0048]

[0049] In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed;

[0050] S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is:

[0051]

[0052] x F,n =x light -(ni)×s

[0053]

[0054] Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles;

[0055] Autonomous driving vehicle trajectory controller, used to build a connected autonomous driving vehicle agent based on the vehicle trajectory optimization controller of the deep reinforcement learning model: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the optimization objectives are: including vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; the reinforcement learning controller includes an actor neural network model and a critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model; then construct the intelligent agent historical experience data set, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

[0056] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0057] 1. In the mixed traffic flow of autonomous driving, the real-time trajectory status of a large number of regular people driving is unknown, and thus the traffic shock wave status at the intersection is also unknown. Therefore, the present invention constructs a real-time prediction model for traffic shock waves based on the data of the upstream traffic detector of the intersection, and its predicted shock wave information is used as the input of trajectory planning so that the vehicle trajectory meets the requirements of dissipating shock waves.

[0058] 2. The present invention develops a trajectory planning model based on multiple optimization objectives, including safety performance, traffic efficiency and driving comfort, to ensure that the output shock wave dissipation trajectory meets the requirements of traffic safety and transportation efficiency. At the same time, the deep reinforcement learning framework is used to train and optimize the planning model, so that it has high adaptability and real-time output stability in random and complex environments.

[0059] 3. The present invention relies on the environmental adaptability of the trajectory planning model. In an autonomous driving mixed traffic environment, through the continuous updating of congestion absorbing vehicles and the collaborative absorption of multiple vehicles, it can suppress the periodic traffic shock waves caused by traffic signals while effectively alleviating the secondary traffic shock wave problem caused by the deceleration behavior of the absorption process. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the decision-making process of the autonomous driving vehicle of the present invention.

[0061] Figure 2 It is a schematic diagram of traffic shock wave prediction of the present invention.

[0062] Figure 3 It is a flow chart of deep reinforcement learning training in the present invention.

[0063] Figure 4 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0064] like Figure 4 As shown, the present invention provides a method for dissipating overlapping traffic shock waves on a fast road in a vehicle-road cooperative environment, comprising the following steps:

[0065] S1. Collect environmental data of the target road section, including traffic signal system data and vehicle operation data, and then generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems;

[0066] S2. Constructing a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculating the traffic prediction state, that is, the optimal time for the intelligent networked vehicle to arrive at the intersection; specifically, the following sub-steps are included:

[0067] S201. Construct a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles;

[0068] S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time tP,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows:

[0069]

[0070] In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed;

[0071] S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is:

[0072]

[0073] x F,n =x light -(ni)×s

[0074]

[0075] Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles;

[0076] S3, the vehicle trajectory optimization controller based on the deep reinforcement learning model builds the networked autonomous driving vehicle intelligent body, specifically: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the agent optimization objectives include: vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; the reinforcement learning controller includes the actor neural network model and the critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model;

[0077] S4. Construct the historical experience data set of the intelligent agent, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

[0078] As a preferred embodiment of the present invention, in step S3, the traffic signal status includes the distance between the vehicle and the intersection, the signal cycle, the current phase, and the remaining time of the current phase; the traffic prediction status is the optimal time for the vehicle to arrive at the intersection; the leading vehicle status is the leading vehicle speed; and the own vehicle status is the own vehicle speed and the distance to the leading vehicle.

[0079] In step S3, the optimization objective is expressed as an action reward function, including the speed reward r speed 、Safety Reward safe , time reward time , acceleration reward r a , as follows:

[0080] (1) Speed ​​reward r speed is the vehicle speed of the connected autonomous vehicle n at time t The corresponding reward is within the speed limit range [v min ,v max ]Inside, The larger the r speed The higher the value, the calculation is as follows:

[0081]

[0082] (2) Safety Reward saf e Based on the distance between the connected autonomous driving vehicle n and its preceding vehicle n-1 at time t Whether it meets the expected safety distance Calculate as follows:

[0083]

[0084] In the formula, a max is the maximum acceleration of the connected autonomous driving vehicle; τ is the system delay time; is the distance between vehicle n and vehicle n-1;

[0085] (3) Time reward r timeIt is used to encourage vehicles to avoid reaching the intersection stop line during the red light phase and not be affected by traffic shock waves, as shown in the following formula:

[0086] r time =r p +r g

[0087]

[0088] In the formula, r p , r g They are the traffic shock wave reward and the signal phase reward; Maintain the current speed for vehicle n The time it takes to reach the intersection; is the distance between vehicle n and the intersection at time t; T c is the traffic signal cycle; t g_start , t y_end They are the green light start time and the yellow light end time respectively.

[0089] (4) Acceleration reward r a is the acceleration at time t The corresponding reward value is used to ensure the smoothness of vehicle driving and is calculated as follows:

[0090]

[0091] (5) The total reward value is as follows:

[0092]

[0093] Where ω1, ω2, ω3, and ω4 are reward weights.

[0094] Further, as a preferred embodiment of the present invention, constructing the agent historical experience data set specifically includes: the agent independently interacts with the environment in the simulation to obtain experience data And store them in the experience data set. The specific process of obtaining an experience data is as follows: Agent n obtains the environment state at time t Based on status Output the acceleration action of agent n through the trajectory optimization model The action is executed and the simulation runs to time t+1, and the environment state is updated to By status Get action rewards

[0095]

[0096] Furthermore, as a preferred embodiment of the present invention, in step S4, a reinforcement learning controller is trained with a historical experience data set, specifically: a single intersection environment is initialized on a simulation platform, including road section length, number of lanes, speed limit, flow, traffic detector location, traffic signal strategy, and vehicle model parameters. The agent runs in the simulation environment and collects experience data to construct a historical experience data set for the agent. In each round of training, a fixed number of experience data are randomly selected from the experience data set to construct a training set. The policy gradient and valuation error are calculated and the control model is updated until the error converges to obtain the optimal vehicle trajectory optimization controller.

[0097] like Figure 1 The present invention also provides a system for dissipating overlapping traffic shock waves on a fast road in a vehicle-road cooperative environment, comprising:

[0098] Data information acquisition module, vehicle communication module, and vehicle control module;

[0099] The data information acquisition module is used to collect environmental data of the target road section; the data information acquisition module includes a road-side data acquisition unit and a vehicle-side data acquisition unit; the road-side data acquisition unit is used to collect traffic signal system data, and the vehicle-side data acquisition unit is used to collect vehicle operation data.

[0100] A vehicle communication module, used to generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems;

[0101] The vehicle control module includes a traffic shock wave predictor and an autonomous vehicle trajectory controller;

[0102] The traffic shock wave predictor is used to construct a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculate the optimal time for the intelligent networked vehicle to arrive at the intersection; the traffic shock wave predictor is configured to perform the following actions, including steps S201 to S203:

[0103] S201. Construct a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles;

[0104] S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time t P,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows:

[0105]

[0106] In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed;

[0107] S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is:

[0108]

[0109] x F,n =x light -(ni)×s

[0110]

[0111] Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles;

[0112] Autonomous driving vehicle trajectory controller, used to build a connected autonomous driving vehicle agent based on the vehicle trajectory optimization controller of the deep reinforcement learning model: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the optimization objectives are: including vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; the reinforcement learning controller includes an actor neural network model and a critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model; then construct the intelligent agent historical experience data set, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

[0113] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment, characterized in that: The following steps are involved: S1. Collect environmental data of the target road section, including traffic signal system data and vehicle operation data, and then generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems; S2. Constructing a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculating the traffic prediction state, that is, the optimal time for the intelligent networked vehicle to arrive at the intersection; specifically, the following sub-steps are included: S201, constructing a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles; S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time t P,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows: In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed; S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is: x F,n =x light -(n-i)×s Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles; S3, the vehicle trajectory optimization controller based on the deep reinforcement learning model builds the networked autonomous driving vehicle intelligent body, specifically: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the agent optimization objectives include: vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; the reinforcement learning controller includes the actor neural network model and the critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model; S4. Construct the historical experience data set of the intelligent agent, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

2. According to the method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment according to claim 1, it is characterized in that: In step S1, the traffic signal system data includes: the signal strategy of the intersection, the current signal phase information, and the upstream vehicle information; the vehicle operation data includes the operation data information of the own vehicle and the preceding vehicle.

3. The method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment according to claim 1, characterized in that: In step S3, the traffic signal status includes the distance between the vehicle and the intersection, the signal cycle, the current phase, and the remaining time of the current phase; the traffic prediction status is the optimal time for the vehicle to arrive at the intersection; the leading vehicle status is the leading vehicle speed; and the own vehicle status is the own vehicle speed and the distance to the leading vehicle.

4. The method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment according to claim 1, characterized in that: In step S3, the optimization objective is expressed as an action reward function, including the speed reward r speed 、Safety Reward safe , time reward time , acceleration reward r a , as follows: (1) Speed ​​reward r speed is the vehicle speed of the connected autonomous vehicle n at time t The corresponding reward is within the speed limit range [v min ,v max ]Inside, The larger the r speed The higher the value, the calculation is as follows: (2) Safety Reward safe Based on the distance between the connected autonomous driving vehicle n and its preceding vehicle n-1 at time t Whether it meets the expected safety distance Calculate as follows: In the formula, a max is the maximum acceleration of the connected autonomous driving vehicle; τ is the system delay time; is the distance between vehicle n and vehicle n-1; (3) Time reward r time It is used to encourage vehicles to avoid reaching the intersection stop line during the red light phase and not be affected by traffic shock waves, as shown in the following formula: r time ＝r p +r g In the formula, r p , r g They are the traffic shock wave reward and the signal phase reward; Maintain the current speed for vehicle n The time it takes to reach the intersection; is the distance between vehicle n and the intersection at time t; T c is the traffic signal cycle; t g_start , t y_end The green light start time and the yellow light end time are respectively; (4) Acceleration reward r a is the acceleration at time t The corresponding reward value is used to ensure the smoothness of vehicle driving and is calculated as follows: (5) The total reward value is as follows: Where ω1, ω2, ω3, and ω4 are reward weights.

5. The method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment according to claim 1, characterized in that: In step S4, the construction of the agent historical experience data set is specifically: the agent interacts with the environment independently in the simulation to obtain experience data And store them in the experience data set. The specific process of obtaining an experience data is as follows: Agent n obtains the environment state at time t Based on status Output the acceleration action of agent n through the trajectory optimization model The action is executed and the simulation runs to time t+1, and the environment state is updated to By status Get action rewards 6. The method for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment according to claim 1, characterized in that: In step S4, the reinforcement learning controller is trained with the historical experience data set, specifically: a single intersection environment is initialized on the simulation platform, including the length of the road section, the number of lanes, the speed limit, the flow rate, the location of the traffic detector, the traffic signal strategy, and the vehicle model parameters. The intelligent agent runs in the simulation environment and collects experience data to construct the historical experience data set of the intelligent agent. In each round of training, a fixed number of experience data are randomly selected from the experience data set to construct a training set, the policy gradient and the valuation error are calculated, and the control model is updated until the error converges to obtain the optimal vehicle trajectory optimization controller.

7. A system for dissipating overlapping traffic shock waves on expressways in a vehicle-road cooperative environment based on claim 1, characterized in that: include: Data information acquisition module, vehicle communication module, and vehicle control module; Data information collection module, used to collect environmental data of the target road section; The data information acquisition module includes a road-side data acquisition unit and a vehicle-side data acquisition unit; the road-side data acquisition unit is used to collect traffic signal system data, and the vehicle-side data acquisition unit is used to collect vehicle operation data. A vehicle communication module, used to generate information transmission data between vehicles and information transmission data between vehicles and traffic signal systems; The vehicle control module includes a traffic shock wave predictor and an autonomous vehicle trajectory controller; The traffic shock wave predictor is used to construct a traffic shock wave model based on shock wave theory according to the information transmission data between vehicles and the information transmission data between vehicles and traffic signal systems, and calculate the optimal time for the intelligent networked vehicle to arrive at the intersection; the traffic shock wave predictor is configured to perform the following actions, including steps S201 to S203: S201, constructing a general computing unit based on a mixed fleet: the mixed fleet includes N vehicles; {vehicle 1, vehicle 2, vehicle 3, ... vehicle n ..., vehicle N}, the leading vehicle and the trailing vehicle are intelligent network-connected vehicles, and the vehicles between the leading vehicle and the trailing vehicle are conventional human-driven vehicles; S202: Determine the type of the mixed convoy as an intermittent convoy or a continuous convoy based on the traffic signal system data and the vehicle operation data. Specifically, there is a vehicle i in the convoy that arrives at the intersection at time t P,i If the vehicle is within the red light phase range, the convoy is an intermittent convoy, otherwise it is a continuous convoy. The time it takes for the vehicle to arrive at the intersection is calculated as follows: In the formula, is the time when vehicle n passes the detector; x light is the location of the traffic light; x det is the location of the traffic detector; is the speed of vehicle n when it passes the detector; t E is the time it takes for the last vehicle in the previous convoy to pass the traffic light; max is the maximum speed of the vehicle; l is the expected safety distance at the maximum speed; S203, calculate the optimal arrival time of the last vehicle: when the fleet is a continuous queue, the optimal arrival time of the last vehicle t D =t P,N , when the queue is an intermittent convoy, the optimal arrival time of the last vehicle is: x F,n =x light -(n-i)×s Where, t F,n , x F,n are the time and position of the vehicle resuming driving after stopping under the influence of the queuing shock wave; t C , x C are the time and position at which the connected autonomous driving vehicle ends its shock wave absorption driving behavior; t G is the green light start time; w is the intersection dissipation wave speed; s is the safe parking distance for vehicles; Autonomous driving vehicle trajectory controller, used to build a connected autonomous driving vehicle agent based on the vehicle trajectory optimization controller of the deep reinforcement learning model: For input, Including traffic signal status, traffic prediction status, front vehicle status and own vehicle status; vehicle acceleration The control action is the output, and the optimization objectives are: including vehicle speed, vehicle operation safety, vehicle arrival time, and trajectory smoothness; The reinforcement learning controller includes an actor neural network model and a critic neural network model. The actor neural network model is used to Output the control action of the intelligent agent, and the critic neural network model outputs the value of the intelligent agent strategy to evaluate the actor neural network model; Then construct the agent historical experience dataset, including: state space Vehicle acceleration Next state space Next Moment Reward The reinforcement learning controller is trained with historical experience data sets to obtain the optimal vehicle trajectory optimization controller; then the optimal vehicle trajectory optimization controller is used to control the intelligent connected vehicle.

Images