Minimal Risk Maneuver Selection via Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in dynamically selecting optimal maneuvers to navigate safely and avoid accidents, as existing systems may choose unnecessary or unsafe actions due to lack of real-time environmental and vehicle state feedback.
Innovation Solution
The implementation of a system that uses sensors to monitor vehicle and environmental parameters, predicts future states, and employs a Markov Decision Process with reinforcement learning to select and train optimal minimal risk maneuvers (MRMs) based on continuous data feedback, assigning rewards for safe and efficient operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional autonomous vehicle control systems are used to navigate and avoid obstacles, then the vehicle can operate autonomously, but the system may choose unnecessary or unsafe maneuvers due to lack of real-time feedback
Solution Approach 1:
The patent implements a feedback mechanism where the Markov Decision Process continuously receives real-time observations of the environment and vehicle state, processes this information, and updates maneuver selections accordingly. This closed-loop feedback ensures the system has current information about both the environment and vehicle condition, preventing unnecessary or unsafe maneuvers based on outdated assumptions.
2Reliability
If the vehicle continuously monitors and updates maneuvers based on real-time data, then the safety and efficiency improve, but the computational complexity and processing requirements increase
Solution Approach 1:
The system dynamically adjusts its computational approach by using a Markov Decision Process that can adapt the horizon length and detail of environmental modeling based on current situation urgency. In high-risk scenarios, the system expands its computational scope to evaluate more potential maneuvers, while in safe conditions, it uses simplified models to reduce processing load. This dynamic adaptation maintains safety while managing computational complexity.
3Device complexity
If the system selects from a limited set of predefined maneuvers, then the control logic is simpler, but the adaptability to diverse and changing environments is reduced
Solution Approach 1:
The patent segments the maneuver selection process into hierarchical layers: a core layer of essential maneuver types (proceed, stop, reverse, turn) and an environmental modeling layer that adapts to specific situations. The Markov Decision Process combines these segmented elements dynamically, allowing the system to maintain simple control logic for basic maneuvers while achieving high adaptability through contextual combination and sequencing of these maneuvers based on real-time environmental assessment.
Data Source
AI summary
Provided are methods for selection of optimal minimal risk maneuver, which can include receiving at least one first parameter associated with a characteristic of a vehicle and at least one second parameter associated with at least one object external to the vehicle, generating at least one future state for at least one of the first and second parameters, selecting at least one maneuver from a plurality of maneuvers based on the generated future state, determining at least one reward value associated with the selected maneuver, updating the selected maneuver based on the determined reward value to generate an updated maneuver, and operating the vehicle based on the updated maneuver. Systems and computer program products are also provided.