Actor-critic algorithm-based distributed traffic signal lamp joint control method

Abstract

The invention discloses an actor-critic algorithm-based distributed traffic signal lamp joint control method. The method includes the steps of: conducting mathematical modeling on a network composed of multiple agents; modeling a Markov decision process of a single traffic intersection in a distributed traffic signal lamp control system, and defining a state set, an action set and a single-step reward value; constructing a multi-agent joint control mode, and establishing communication connection between the agents to exchange respective information; establishing a flexible dominant actor-critic algorithm, adding a strategy entropy of a next state into the single-step reward value, constructing a value function, and adding a dominant function; based on the flexible dominant actor-critic algorithm, taking the purpose of minimizing the average waiting time of vehicles, adopting a joint flexible dominant actor-critic algorithm by the agents of each traffic intersection for learning and control of signal lamps. According to the invention, through cooperative control of signal lamps at different traffic intersections, the overall road smoothness of the traffic network is improved.

Description

technical field [0001] The invention relates to the technical field of adaptive traffic signal control (Adaptive Traffic Signal Control, ATSC), in particular to a method for joint control of distributed traffic signal lights based on actor-critic algorithm. Background technique [0002] With the deepening of urbanization, most cities are facing the huge problem of traffic congestion. The congested road traffic environment not only causes great damage to the environment, but also has a huge negative impact on the social economy. Due to the small space for road expansion reserved in urban planning and the large impact on the construction of transportation infrastructure in the city, coupled with the continuous increase in the number of vehicles per capita, the problem will become more difficult. In this case, optimizing the control technology of signal lights is an easy and economical way to alleviate the problem. Compared with the traditional timing scheme for adjusting dif...

AI Technical Summary

Problems solved by technology

[0003] Previous studies mainly focused on the optimal control of a single traffic intersection, ignoring the fact that the traffic flows at different intersections in the urban traffic network often affect each other

On the other hand, the existing research is mainly based on Q-learning, and there are problems such as unstable convergence value, poor computing power due to too large Q-value table, and inability to adapt to infinite Markov decision chains.

Images