Autonomous Vehicle Model Training Using Simulated Driving Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in training machine learning models due to causality and feedback issues, where learned control policies overweight correlated features rather than causal ones, and overweight past actions as indicative of future actions, leading to suboptimal performance in real-world scenarios.
Innovation Solution
The method involves training a machine learning model using simulated episodes of autonomous vehicle locomotion, where the model processes initial state instances and generates predictions, implementing these predictions to create subsequent state instances, and updating the model based on losses compared to ground truth labels, allowing the model to adapt and improve its predictions iteratively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the machine learning model is trained using traditional methods on real-world data, then the model learns from actual driving scenarios, but the model overweights correlated features rather than causal ones and overweights past actions as indicative of future actions
Solution Approach 1:
The patent creates a simulated environment that copies real-world driving scenarios, allowing the model to be trained on synthesized data that mimics actual driving conditions without the causality and feedback problems present in traditional real-world training methods
Solution Approach 2:
The simulation environment acts as an intermediary between real-world data and model training, enabling the generation of training data with known ground truth labels while avoiding the direct causality issues present in real-world observational data
2Productivity
If the model is trained to improve performance in real-world scenarios, then the model achieves better autonomous control, but the model develops suboptimal behavior due to causality and feedback issues in traditional training
Solution Approach 1:
By copying real-world driving scenarios into a simulation environment, the system enables comprehensive training that improves model performance while maintaining reliability through controlled synthesis of training data with known ground truth
Solution Approach 2:
The system performs preliminary training in a controlled simulation environment before deployment, allowing the model to learn optimal control policies in advance without the negative effects of traditional training methods
3Ease of manufacture
If traditional training methods are used, then the training process is simpler, but the model fails to generalize well to unique real-world scenarios
Solution Approach 1:
The simulation environment provides a universal training platform that can generate diverse driving scenarios while maintaining controlled conditions, enabling the model to generalize across multiple scenarios without sacrificing training simplicity
Data Source
AI summary
Systems and methods for training a machine learning (“ML”) model for use in controlling an autonomous vehicle (“AV”) are described herein. Implementations can obtain an initial state instance from driving of a vehicle, obtain ground truth label(s) for subsequent state instance(s) that each indicate a corresponding action of the vehicle for a corresponding time instance, perform, for a given time interval, a simulated episode, of locomotion of a simulated AV, generate, for each of a plurality of time instances of the given time interval, subsequent simulated state instance(s) that differ from the subsequent state instance(s), determine, using the ML model, and for each of the time instances, a predicted simulated action of the simulated AV based on the subsequent simulated operation instance(s), generate loss(es) based on the predicted simulated actions and the ground truth labels, and update the ML model based on the loss(es).