Autonomous Vehicle J-Turn Control Under Tire-Road Uncertainty
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for designing safe control policies for autonomous vehicles performing extreme maneuvers, such as J-turns, face challenges due to unpredictable vehicle dynamics, uncertainties in tire-road interactions, and the difficulty in generalizing analytical models, leading to potential safety issues in real-world implementations.
Innovation Solution
A computer-implemented method using a constrained Markov decision processing (CMDP) model trained with fuzzy instructions and dynamic models to simulate and execute stunt maneuvers, incorporating worst-case scenario uncertainty modeling to ensure safety and agility, allowing for the detection of stimuli to initiate maneuvers and applying action sequences to control systems for successful execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reinforcement learning with random exploration is used for control policy search, then the system can learn control policies, but the learning efficiency is highly inefficient and might completely fail
Solution Approach 1:
The patent applies preliminary action by incorporating expert instructions and suggestions before the policy search algorithm begins. These expert guidelines pre-shape the exploration space, directing the RL agent toward promising control strategies rather than relying on purely random exploration. This preliminary guidance significantly improves learning efficiency and success rate while maintaining the autonomy of the RL framework.
2Productivity
If instructed reinforcement learning is used to reduce exploration, then learning efficiency improves, but the control policy may perform well in simulations but poorly in real world practice
Solution Approach 1:
The patent implements feedback mechanisms that continuously monitor the control policy's performance and adjust the expert instructions accordingly. The system uses simulation results to refine the expert guidelines, creating a closed-loop learning process. This feedback-driven approach ensures that the instructed RL policy generalizes well to real-world scenarios by iteratively improving the expert instructions based on observed performance gaps between simulation and reality.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the weights and priorities of different expert instructions based on their effectiveness. The system modifies the instruction parameters during training to optimize the balance between following expert guidance and exploring new strategies. This adaptive parameter adjustment allows the system to maintain high learning efficiency while ensuring robust real-world performance.
3Ease of manufacture
If analytical models are used to design autonomous extreme maneuvers, then specific maneuvers can be designed, but the models are difficult to generalize for other maneuvers and show uncertainties in tire-road interactions
Solution Approach 1:
The patent achieves universality by creating a unified framework that combines analytical models with reinforcement learning. The system uses a generalizable RL architecture that can adapt to different maneuver types while leveraging analytical models for specific maneuver design. This hybrid approach allows the same system to design and execute various extreme maneuvers (J-turns, drifts, etc.) by switching between model-based planning and RL-based adaptation, thereby achieving both specificity and generalization.
Data Source
AI summary
Techniques disclosed herein relate to applying a trained constrained Markov decision process (CMDP) to control an autonomous vehicle to perform a stunt maneuver, such as a J-turn, in a safe and agile manner. The CMDP may implement a set of fuzzy logic instructions that correspond to actions needed to execute the stunt maneuver. While training the CMDP, the techniques disclosed herein may utilize a dynamic model of the autonomous vehicle that includes a model of the uncertainty introduced when implementing the stunt maneuver, such as the uncertainty in the tire-road mechanics. By utilizing a worst case scenario measure of the uncertainty during training, safe performance of the stunt maneuver is guaranteed when the trained model is applied in the real world.