Maneuver Planning With Updated State Transitions for Automated Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing maneuver planning systems for automated vehicles and robots struggle to adapt to real-world, dynamic environments, as they often rely on predefined assumptions and lack the ability to learn from real-world transitions between states.
Innovation Solution
A method and device that utilize a Markov decision process to describe the environment in discrete form, record actual transitions, and update transition probabilities based on real-world data, integrating reinforcement learning to optimize maneuver planning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If predefined Markov decision problems are used for maneuver planning, then the planning process has a structured framework, but the system cannot adapt to real-world dynamic environments
Solution Approach 1:
The system records actual transitions between states during vehicle operation and uses this feedback to continuously update the transition probabilities in the Markov decision problem. This feedback mechanism allows the predefined framework to adapt to real-world conditions while maintaining its structured planning approach.
Solution Approach 2:
The transition probabilities in the Markov decision problem are made dynamic through continuous updating based on recorded actual transitions. Instead of static predefined probabilities, the system adapts the transition model to reflect real-world dynamics, resolving the contradiction between structured framework and adaptability.
2Productivity
If reinforcement learning methods are used to learn optimal actions, then decision-making can be optimized, but the system requires thorough examination of surroundings which increases computational complexity
Solution Approach 1:
The system performs preliminary action by recording actual transitions during normal operation and updating transition probabilities in advance. This preparation reduces the computational burden during real-time decision-making, as the reinforcement learning agent can rely on pre-updated transition models rather than examining all possibilities from scratch.
Solution Approach 2:
Instead of requiring thorough examination of all possible surroundings and transitions, the system uses partial action by leveraging the updated transition probabilities from recorded data. This allows the reinforcement learning method to make optimized decisions without the full computational overhead of exhaustive examination.
3Adaptability or versatility
If transition probabilities are updated using recorded actual transitions, then the Markov decision problem adapts to real conditions, but data processing and model updating increase system complexity
Solution Approach 1:
The system performs self-service by automatically recording actual transitions during operation and updating its own transition probabilities without external intervention. This self-updating mechanism enables adaptation to real-world conditions while minimizing the need for complex external data processing infrastructure.
Solution Approach 2:
The system creates a simplified copy of real-world transitions by recording actual state transitions and representing them as updated probabilities in the Markov decision problem. This copying approach allows adaptation to real conditions while maintaining the simplified discrete state space structure, avoiding the complexity of processing full real-world data.
Data Source
Figure 1
Figure 2
AI summary
The invention relates to a method for supporting maneuver planning for a vehicle (50) driving with at least partial automation or for a robot; wherein a state space (10) of an environment of the vehicle (50) or of the robot is described in discrete form by means of a specified Markov decision process (20); wherein maneuver planning for the vehicle (50) or for the robot, on the basis of the Markov decision process (20), is supported by the execution of at least one optimization method; wherein actually occurring transitions between states (11) of the state space (10) are recorded during operation of the vehicle (50) or of the robot; and wherein, by evaluation of the frequency of the recorded transitions, probabilities of transitions between the states (11) of the Markov decision process (20) are determined and updated. The invention also relates to a device (1) for supporting maneuver planning for a vehicle (50) driving with at least partial automation or for a robot, and to a back-end server (60).