A collaborative control method for regional signal lights based on multi-view encoding transfer reinforcement learning

Abstract

The invention discloses a method for collaborative control of regional signal lights based on multi-view code transfer reinforcement learning, which belongs to the intersecting field of machine learning and intelligent transportation. Methods include a multi-view state encoder and a transfer reinforcement learning framework. The proposed multi-view state encoder integrates the one-dimensional and two-dimensional state of the intersection and the state information from the neighboring intersection, and uses the result as the actual input of the intersection agent. In the proposed migration reinforcement learning framework, first independently train several expert agents focusing on fitting ability; then use the transferred expert agents to jointly guide training a seed agent focusing on generalization ability; finally, the seed agent The parameters of the agent are migrated to each intersection for adaptive training, and the difference between these agents and the expert agent is calculated to determine whether to perform iterative training. The final agent has better decision-making ability and generalization performance at the same time, effectively alleviating traffic congestion.

Description

technical field [0001] The invention belongs to the intersecting technical field of machine learning and intelligent transportation, and relates to a method for collaborative control of regional signal lights based on multi-view coding transfer reinforcement learning. Background technique [0002] With the acceleration of urbanization, the number of cars per capita is also increasing, and traffic congestion has become one of the major problems affecting urban economic development. Reasonable control of traffic lights plays an important role in improving traffic congestion and reducing traffic accidents. [0003] There are many ways to control traffic lights. Traditional timing signal control methods cycle through a predefined phase setting periodically. This method mainly sets the phase and time of the rotation based on the historical road condition information at the intersection, and lacks the ability to dynamically process the road conditions. Once the road conditions c...

AI Technical Summary

Problems solved by technology

However, the information from the neighborhood may play a positive or negative role in the coordinated control of regional signal lights. Direct use of the original neighborhood information may cause information redundancy and affect the effect of coordinated control.

[0007] In addition, the methods mentioned above either only use the one-dimensional information of the intersection, or only use the two-dimensional information of the intersection as the state input, they do not effectively use the information that the intersection itself can obtain

Moreover, in a multi-intersection environment, all agents learn and make decisions at the same time, so agents will also affect each other

An unreasonable decision made by an agent may cause the agent in the neighborhood to face road conditions that it has never experienced, thus causing its decision to fail, that is, the agent's generalization ability is poor

[0008] In order to effectively utilize the traffic information obtained at each intersection and alleviate the problems of poor generalization ability of the trained intersection agent, the present invention proposes a collaborative control method for regional signal lights based on multi-view encoding transfer reinforcement learning (Multi-agent Transfer Soft Actor -Criticwith the Multi-view State Encoder, MT-SAC)

Images