Traffic Signal Phase Control Using Spatiotemporal RL Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing reinforcement learning (RL)-based traffic signal control (TSC) methods face challenges such as inefficient online exploration leading to congestion, poor model performance during training, and suboptimal control due to lack of dynamic spatial dependency modeling in complex traffic flow patterns.
Innovation Solution
Combining offline learning, sequence modeling, and spatiotemporal dependency modeling to capture dynamic spatial dependencies between traffic signals, utilizing a traffic signal prediction model with a spatiotemporal encoder and return-based action decoder trained through return-based contrastive learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If online RL method is used for traffic signal control, then the model can achieve optimal control effect through iterative optimization, but it causes serious traffic congestion or accident risks during training due to extensive exploration in real environment
Solution Approach 1:
The patent pre-trains the RL model in a simulated environment using historical trajectory data before deploying it to real traffic intersections. This preliminary training phase allows the model to learn optimal control strategies without causing actual traffic congestion or safety issues, while still achieving iterative optimization through the simulated environment.
2Object-affected harmful factors
If offline RL method is used to avoid real-time interaction risks, then safety is improved, but model performance deteriorates due to lack of iterative optimization of data distribution
Solution Approach 1:
The patent implements a feedback mechanism where the pre-trained model's predictions are used to generate synthetic trajectory data, which is then used to re-train and improve the model. This closed-loop feedback process allows iterative optimization of the data distribution without requiring real-time interaction with the actual traffic environment, thus maintaining both safety and performance.
3Productivity
If sequence modeling-based TSC method is used to predict action based on historical trajectory data, then competitive performance is achieved, but it fails to capture dynamic spatial dependency between data samples from different intersections
Solution Approach 1:
The patent merges sequence modeling with graph neural networks to create a hybrid architecture. The sequence modeling component captures temporal dependencies in historical trajectory data, while the graph neural network component models the spatial relationships and correlations between different intersections. This combination allows the system to leverage both temporal and spatial information for improved prediction performance and adaptability.
Data Source
AI summary
Provided are a reinforcement learning (RL)-based traffic signal control (TSC) method and apparatus, a device, a medium, and a product. The TSC method includes: obtaining traffic state data of a target intersection at a current time point and a road network graph, where the traffic state data includes a quantity of lanes at the target intersection and a traffic flow of each of the lanes; inputting the traffic state data and the road network graph into a preset traffic signal prediction model, and obtaining a target phase action output by the traffic signal prediction model, where the traffic signal prediction model includes a spatiotemporal encoder and a return-based action decoder, and the traffic signal prediction model is obtained through training based on return-based contrastive learning; and controlling, based on the target phase action, a traffic light at the target intersection to execute the target phase action.