Adaptive signal control method and system based on reinforcement learning and phase competition

Abstract

The invention relates to a self-adaptive signal control method and system based on reinforcement learning and phase competition, and the method comprises the steps: obtaining an intersection state through interaction with a simulation environment, obtaining a decision through the output of a strategy network pi theta, collecting rewards and a state at a next moment after the decision, and obtaining a sample simulation track; and training and parameter updating are performed on the PPO network based on the simulation trajectory, and the operation is repeated for a plurality of rounds until convergence. And after convergence, the model can perform signal adjustment and signal control based on the real-time state of the traffic flow. The method can be suitable for different intersections while ensuring the signal control effect. The method can be widely applied to the field of urban traffic signal control.

Description

technical field [0001] The invention relates to the field of urban traffic signal control, in particular to an adaptive signal control method and system based on deep reinforcement learning and phase competition. Background technique [0002] Since the 21st century, with the rapid development of my country's economic level, the living standards of residents have been greatly improved. Therefore, how to reasonably and effectively improve the efficiency of traffic travel, so as to alleviate the problem of traffic congestion, has become the focus of government departments in recent years. The intersection has always been the main cause of traffic congestion, so it has become a research hotspot in recent years to alleviate traffic congestion by optimizing its signal timing. [0003] Most of the traditional signal control methods are modeled and optimized based on the knowledge in the field of traffic engineering, such as Webster, GreenWave, SCATS, SCOOT and other methods. Howe...

AI Technical Summary

Problems solved by technology

However, most current models based on deep reinforcement learning have two disadvantages: 1. Poor generalization

The control effect of the model trained under one traffic flow data is likely to be greatly reduced under another traffic flow data

2. Most of the current models concatenate all the states into vectors and input them directly, but this makes the model no longer applicable when the topology or phase setting of the intersection changes, due to the change of the vector dimension

Images