Autonomous Driving Decision Search Under Future State Uncertainty
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current driving decision-making in autonomous vehicles based on reinforcement learning algorithms like DQN fails to consider the impact of driving actions on subsequent vehicle behavior, leading to suboptimal decision-making outcomes.
Innovation Solution
A Monte Carlo tree search method is employed, utilizing a stochastic model to predict future driving environment states and determine a favorable action sequence, incorporating dropout-based forward propagation and an episodic memory to enhance robustness and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reinforcement learning (DQN) algorithm is used for driving decision-making, then the system can learn driving actions from environment states, but the impact of driving actions on subsequent driving is not considered leading to suboptimal decisions
Solution Approach 1:
The patent performs preliminary actions by pre-building the Monte Carlo tree structure and pre-calculating possible future states before making the final driving decision. This allows the system to consider long-term impacts of actions without exceeding computational time constraints during real-time decision-making.
Solution Approach 2:
The decision-making process is segmented into discrete steps: building the Monte Carlo tree, performing rollouts, calculating value functions, and selecting actions. This segmentation allows the system to manage computational complexity by breaking down the complex task of considering future impacts into manageable stages.
2Reliability
If Monte Carlo tree search is performed to consider future driving states, then decision-making robustness is improved, but computational complexity increases
Solution Approach 1:
The patent performs a limited number of rollouts (e.g., 100-1000 simulations) rather than exhaustively exploring all possible future states. This partial action approach provides sufficient robustness for safe driving decisions while keeping computational complexity manageable for real-time autonomous vehicle systems.
Solution Approach 2:
The system adjusts parameters such as the number of rollouts, tree depth, and simulation steps to balance decision-making robustness with computational complexity. These parameter changes allow the Monte Carlo tree search to be adapted to different computational resources and safety requirements.
3Reliability
If stochastic model with dropout-based forward propagation is used to predict future states, then uncertainty in future environments is considered improving robustness, but measurement precision of predicted states may be reduced
Solution Approach 1:
The patent uses dropout-based forward propagation during the rollout process to generate multiple possible future states, creating a distribution of outcomes rather than a single prediction. This feedback mechanism allows the system to account for environmental uncertainty and make more robust decisions while the aggregate of multiple predictions maintains overall accuracy.
Data Source
AI summary
The present disclosure relates to driving decision-making methods, apparatuses, and chips. One example method includes constructing a Monte Carlo tree based on a current driving environment state, where the Monte Carlo tree includes a root node and N−1 non-root nodes, each node represents one driving environment state, and a driving environment state represented by any non-root node is predicted by a stochastic model of driving environments. Based on at least a value function of each node in the Monte Carlo tree, a node sequence that starts from the root node and ends at a leaf node is determined. A driving action sequence is determined based on a driving action corresponding to each node in the node sequence.