Autonomous Vehicle Logged-Data Augmentation for Realistic Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicle technology faces challenges in acquiring sufficient and high-quality training data to accurately represent various driving conditions, as real-world sensor data alone is insufficient, and existing simulation data from video game-like simulators lacks realism.
Innovation Solution
A method and system that generate simulation data from logged data of autonomous vehicles, including receiving logged data, generating augmented data describing actors and their behavior, and creating simulation scenarios to train machine learning models for perception, planning, and control subsystems, using a simulation data generator that includes a data mapping engine, augmentation engine, and scenario production engine.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If simulation data from video game-like simulators is used to train machine learning models, then the quantity of training data is increased, but the quality and realism of the data deteriorates
Solution Approach 1:
The patent uses real logged sensor data from autonomous vehicles as the source truth, and creates simulation data by copying and augmenting this real data rather than generating it from scratch using video game-like simulators. This ensures the simulation data maintains the realism and quality characteristics of real-world data while still providing the quantity needed for training.
Solution Approach 2:
The patent introduces an intermediary process that transforms real logged data into simulation scenarios. This intermediary transformation pipeline includes mapping logged data to simulation coordinates, augmenting the data, and generating simulation scenarios that preserve the underlying realism while enabling scalable data generation for training machine learning models.
2Reliability
If only real-world sensor data from autonomous vehicle operation is collected, then the quality and realism of training data is maintained, but the quantity of training data is insufficient
Solution Approach 1:
The patent performs preliminary actions by collecting and storing logged sensor data from autonomous vehicle operations in advance. This logged data serves as a foundation that can be repeatedly used and augmented to generate multiple simulation scenarios, thereby multiplying the effective training data quantity from a single set of real-world recordings.
Solution Approach 2:
The patent dynamically generates multiple simulation scenarios from static logged data by applying various transformations, coordinate mappings, and augmentations. This dynamic process creates diverse training examples from a single real-world data source, effectively increasing the quantity of training data while preserving its quality.
3Adaptability or versatility
If more diverse driving conditions and scenarios are represented in training data, then the adaptability of machine learning models is improved, but the complexity of data collection and processing increases
Solution Approach 1:
The patent creates a universal data processing pipeline that can handle multiple types of logged data (sensor data, localization data, tracking data) and generate diverse simulation scenarios covering various driving conditions. This multi-functional system efficiently produces adaptable training data without requiring separate complex collection systems for each scenario type.
Solution Approach 2:
The patent achieves diverse driving conditions by changing parameters in the simulation scenarios generated from logged data, such as modifying actor types, motion behavior characteristics, and environmental conditions. This parameter-based approach to creating diversity is more efficient than physically collecting data for each scenario, reducing overall system complexity.
Data Source
AI summary
Logged data from an autonomous vehicle is processed to generate augmented data. The augmented data describes an actor in an environment of the autonomous vehicle, the actor having an associated actor type and an actor motion behavior characteristic. The augmented data may be varied to create different sets of augmented data. The sets of augmented data can be used to create one or more simulation scenarios that in turn are used to produce machine learning models to control the operation of autonomous vehicles.