Single-point intersection signal control method based on deep cyclic Q learning

Abstract

The invention discloses a single-point intersection signal control method based on deep cycle Q learning. The method comprises the following steps: learning an optimal signal control strategy at a single intersection by using a deep cyclic Q learning (DRQN) algorithm, wherein according to the DRQN algorithm, an LSTM neural network is introduced on the basis of DQN, the characteristic that the LSTMcan memorize time axis information is utilized, the current intersection input state is comprehensively expressed by combining the states of the intersection at the previous several moments instead of the state of the intersection at the current moment and thus the influence of the POMDP characteristics of the intersection on deep Q learning performance is reduced. The performance of the improvedDRQN algorithm provided by the invention is superior to that of the DQN algorithm and is also superior to that of a traditional intersection timing control method. When the traffic flow is close to saturation and supersaturation, the DRQN algorithm can observe the state of the intersection at each moment and make an optimal opportunity choice, thereby improving the traffic efficiency of the intersection.

Description

technical field [0001] The invention relates to the technical field of deep reinforcement learning and traffic signal control, in particular to a single-point intersection signal control method based on deep loop Q-learning. Background technique [0002] With the rapid development of the economy, the problem of urban traffic congestion has become very serious, causing huge economic losses, especially in China. The short-term traffic demand at road intersections has the characteristics of time-varying, nonlinear, and complexity, and it is difficult to establish an accurate mathematical model. Simple timing control and induction control methods are difficult to adapt to the dynamic, complex, and rapid changes in traffic flow. Ineffective. The development of intelligent transportation, using artificial intelligence knowledge to strengthen the effective control of urban traffic signals, can effectively alleviate urban congestion. [0003] In recent years, with the rapid develo...

AI Technical Summary

Problems solved by technology

[0003] In recent years, with the rapid development of deep learning, deep reinforcement learning has been applied to traffic control as an emerging technology, which can control and optimize simple traffic signal timing, but for the traditional deep Q-learning algorithm, although It can use the neural network to perceive and learn the useful features, does not need to manually extract features, improves the accuracy of the state representation, and also solves the problem of the input dimension explosion of the traditional reinforcement learning algorithm. Data loss during data transmission will cause the real-time state value to be different from the real value, resulting in a decrease in the accuracy of the state input perceived by deep Q-learning, thus affecting the performance of the algorithm

Images