Adaptive Cycle-Level Traffic Signal Control Using PPO
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing adaptive traffic signal controllers operate on a second-by-second basis, which poses challenges in terms of safety, predictability, and computational requirements, and lack the ability to generate phase durations for a traffic signal cycle over a continuous range of values, limiting their flexibility and accuracy.
Innovation Solution
An intelligent adaptive cycle-level traffic signal controller using Proximal Policy Optimization (PPO), a reinforcement learning algorithm, that operates within a continuous action space, generating continuous values for phase durations, enabling more flexible and accurate traffic signal control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If existing adaptive traffic signal controllers operate on a second-by-second basis, then they can respond quickly to traffic changes, but they face challenges in safety, predictability, and computational requirements
Solution Approach 1:
The patent segments the traffic signal control into two distinct levels: cycle-level control (determining phase durations) and phase-level control (managing individual phase transitions). This segmentation allows the system to make stable, predictable cycle-level decisions while still responding to real-time traffic conditions through phase-level adjustments, thereby resolving the contradiction between fast response and reliability.
Solution Approach 2:
The patent implements preliminary action by determining complete phase duration values for entire traffic signal cycles in advance, before the cycles execute. This preliminary planning at the cycle level provides predictability and safety, while the system remains adaptable to real-time conditions through the phase-level controller that operates within the predetermined cycle framework.
2Speed
If existing adaptive traffic signal controllers operate on a second-by-second basis, then they can respond quickly to traffic changes, but computational requirements become excessive
Solution Approach 1:
The patent segments the control architecture into cycle-level and phase-level components, where the computationally intensive cycle-level optimization determines phase durations based on traffic flow patterns, while the phase-level controller handles real-time transitions with minimal computation. This segmentation reduces overall computational requirements while maintaining responsive performance.
Solution Approach 2:
By performing preliminary optimization at the cycle level to determine phase durations before execution, the system reduces real-time computational burden. The heavy lifting of optimization is done in advance, allowing the system to respond quickly to traffic changes without excessive computational requirements during actual operation.
3Ease of manufacture
If existing controllers generate phase durations for discrete time intervals, then they are simpler to implement, but they lack flexibility and accuracy in optimizing traffic flow
Solution Approach 1:
The patent implements dynamics by allowing phase durations to be continuously adjustable within each phase rather than being fixed to discrete time intervals. This dynamic approach enables the traffic signal controller to optimize traffic flow more accurately by adjusting phase lengths based on real-time traffic conditions, while the underlying framework remains implementable through standard traffic signal infrastructure.
Solution Approach 2:
The patent applies parameter changes by transitioning from discrete time interval-based phase durations to continuous parameter-based control. This allows the system to achieve higher flexibility and accuracy in optimizing traffic flow by adjusting phase duration parameters continuously based on traffic flow measurements, while maintaining ease of implementation through the cycle-level/phase-level separation architecture.
Data Source
AI summary
Methods, systems, and processor-readable media for adaptive cycle-level traffic signal control are described. An adaptive cycle-level traffic signal controller and control method that operate within a continuous action space. A reinforcement learning algorithm called Proximal Policy Optimization (PPO), which is a type of actor-critic model for reinforcement learning, may be used to generate signal cycle phase durations selected from a continuous range of values. The controller thus does not treat the action space as discrete, but instead produces continuous values as output. The generated phase durations may define a full traffic signal cycle. The inputs to the controller may indicate current and past states of the traffic environment. The average duration of delay of vehicles in the traffic environment may be used to calculate the reward for the reinforcement learning model that drives the behavior of the controller.