Traffic light control method based on distributed deep cycle Q network

Abstract

The invention discloses a traffic light control method based on a distributed deep cycle Q network, and mainly solves the problems that in the prior art, the cooperation among multiple intersections is difficult to realize and the state space is excessively large in multi-intersection modeling. The implementation scheme is as follows: 1) urban traffic road network information is read, vehicle traffic state sets of all the intersections are established, and the read urban traffic road network information is converted into an adjacent matrix to be stored by adopting a collaborative diagram; 2) universal performance evaluation indexes are set according to waiting time of vehicles in a road network and the number of the vehicles reaching a destination; 3) state sets, action sets and action reward values of all the intersections at all moments are obtained from the vehicle traffic state sets of all the intersections; and 4) a distributed deep cycle Q network-based traffic light control model is built, and urban road network traffic lights are controlled according to the model. According to the method, the cooperation among the intersections can be realized; the problem of state space explosion is avoided; and the method can be used for urban traffic management to reduce urban traffic congestion.

Description

technical field [0001] The invention belongs to the field of traffic control, in particular to a traffic light control method, which can be used for urban traffic management and reduce urban traffic congestion. Background technique [0002] The use of reinforcement learning to solve urban traffic light control has appeared in the 20th century. Sutton R.RS and others successfully applied SARSA to traffic light control. This is the first academic application of reinforcement learning in traffic light control algorithms in the world. Balaji PG uses the distributed multi-intersection s-model to solve the problem of traffic light control. Each intersection has an independent Q table to learn and judge the execution phase. The experiment proves the effectiveness of Q learning in the traffic light control algorithm. Wiering M A uses a reinforcement learning model for vehicle traffic state modeling to determine the optimal action phase by calculating the maximum gain at traffic ligh...

AI Technical Summary

Problems solved by technology

[0006] In the paper published by Lun Libao in 2013, he used heuristic reinforcement learning to calculate the income of traffic lights at intersections to obtain the execution phase of a single intersection at the next moment, and used the concept of collaboration graph and multi-intersection s collaboration to consider the collaboration of traffic lights between multiple intersections Deployment, experiments show that although the multi-intersection s cooperative algorithm using reinforcement learning is superior to algorithms such as timing systems and max-plus, it still has the problem of state space explosion

Images