Expressway road cooperative control system and method based on deep reinforcement learning

Abstract

The invention discloses an expressway road cooperative control system and method based on deep reinforcement learning, wherein the system comprises a traffic information interaction module, a trafficcontrol module, a deep learning network training module, and a plurality of variable speed limit and ramp control units. The traffic status of ta road is obtained through the information interaction module, and then the traffic status is transmitted to the traffic control module. And the control strategy is continuously optimized through the training module, and the stability of the training process is ensured by adopting a deep reinforcement learning algorithm with an actor-critic architecture. All traffic control units in the system can be controlled at the same time, the problems of trafficstatus space explosion and the like cannot be caused, it can be guaranteed that vehicles pass through bottleneck road sections at a high speed, and passing of surrounding road vehicles cannot be affected by queuing and the like.

Description

technical field [0001] The invention relates to the technical field of traffic control and intelligent transportation, in particular to a system and method for cooperative control of expressway main roads and entrance ramps based on deep reinforcement learning. Background technique [0002] Expressways present frequent, periodic, and long-distance traffic congestion during peak hours, among which, the entrance ramps and adjacent main roads of expressways have become typical bottleneck areas of expressways. Since the early road network planning may be unreasonable, and road reconstruction is difficult, the coordinated management and control of expressway ramps and adjacent main roads is an important way to improve road traffic efficiency and improve driving safety. [0003] The existing cooperative control methods are mainly model predictive control or feedback control methods. In the model predictive control method, the characteristic variables are generally extracted from ...

AI Technical Summary

Problems solved by technology

However, deep reinforcement learning has the following problems when dealing with cooperative control: (1) Synchronous control problem when multi-agents cooperate

For example, the period of the ramp signal light and the period of the variable speed limit control are inconsistent, how to unify the two; (2) the existing reward function is easily affected by random disturbances in the traffic environment; On-ramp queuing issues, which can lead to congestion on surrounding roads

(4) The traditional deep reinforcement learning technology has inherent defects, and it is easy to cause problems such as behavior space state explosion when dealing with multi-agent cooperative control

Images