Traffic Light and Unmanned Vehicle Coordination Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In complex mixed traffic scenarios, intelligent traffic lights and unmanned vehicles often make decisions independently, leading to inaccurate decisions due to a lack of coordination between their state information, resulting in inefficient traffic management.
Innovation Solution
A method and apparatus that generate reinforced traffic light and vehicle state parameters through vehicle state representation information and current traffic light states, allowing for coordinated traffic light control and unmanned vehicle navigation actions, using reinforcement learning models to optimize decisions based on cumulative rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If intelligent traffic lights and unmanned vehicles make decisions independently, then each system can operate autonomously, but decision accuracy deteriorates due to lack of coordination between their state information
Solution Approach 1:
The patent merges the decision-making systems of traffic lights and unmanned vehicles by establishing bidirectional communication between them. The traffic light control end and vehicle navigation end share state information (vehicle flow data, navigation status) and coordinate their actions through joint training of reinforcement learning models, transforming independent autonomous decisions into coordinated collaborative decisions that maintain automation while improving accuracy
Solution Approach 2:
The patent implements feedback mechanisms where the traffic light control end receives vehicle state information from unmanned vehicles, and the vehicle navigation end receives traffic light state information. This bidirectional feedback loop allows both systems to adjust their decisions based on real-time information from the other, improving decision accuracy while maintaining autonomous operation through coordinated feedback-driven adjustments
2Productivity
If traffic lights control based on vehicle flow situation, then traffic management efficiency improves, but coordination with unmanned vehicle navigation deteriorates due to independent decision-making
Solution Approach 1:
The patent makes the traffic light control system multi-functional by enabling it to serve both traditional traffic flow management and unmanned vehicle navigation coordination simultaneously. The same control end processes both vehicle flow data for traffic efficiency optimization and navigation state data for coordination reliability, allowing one system to fulfill multiple functions without compromising either objective
3Device complexity
If reinforcement learning models are trained separately for traffic lights and vehicles, then model training complexity is reduced, but overall system performance deteriorates due to lack of coordinated optimization
Solution Approach 1:
The patent merges the training processes of separate reinforcement learning models by implementing joint training where the traffic light control model and vehicle navigation model are trained together using combined state information and shared reward mechanisms. This coordinated training approach increases complexity but achieves superior system performance through optimized coordination between components
Data Source
AI summary
A method for controlling a traffic light, a method and apparatus for navigating an unmanned vehicle and a method and apparatus for training a model are provided. An implementation comprises: generating a reinforced traffic light state parameter according to vehicle state representation information of an unmanned vehicle currently contained in a preset area of a target traffic light and a current traffic light state parameter of the target traffic light; and generating a traffic light control action according to the reinforced traffic light state parameter; where the reinforced traffic light state parameter is used to cause an unmanned vehicle navigation end to generate a reinforced vehicle state parameter according to a reinforced traffic light state and a current vehicle state parameter of a target unmanned vehicle, and generate an unmanned vehicle navigation action according to the reinforced vehicle state parameter.