Hybrid Sensor Data Augmentation for Rare Autonomous Driving Scenarios
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in testing and validation of driver assistance technologies and autonomous vehicles lies in simulating rare and risky scenarios, as existing simulations lack perfect sensor models and traffic models, resulting in unrealistic data.
Innovation Solution
The system generates improved scenarios by augmenting recorded sensor data with simulated data, using sensor models and object models to create realistic simulations for training machine learning models, which can detect obstacles and predict collisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If detailed simulations are used to prove out rare scenarios, then the scenarios can be tested, but the data generated does not perfectly mimic real-world imperfections due to lack of perfect sensor models and traffic models
Solution Approach 1:
The patent combines real sensor recordings with simulated sensor data to create hybrid training datasets. This merging approach allows the system to leverage the realism of actual sensor imperfections while incorporating the controlled variability of simulated data, thereby improving both reliability and measurement precision simultaneously
Solution Approach 2:
The patent creates copies of real sensor data through simulation, generating synthetic sensor recordings that replicate real-world scenarios. These copied datasets preserve the imperfections and characteristics of real sensors while providing additional training variations that enhance model robustness without requiring perfect sensor models
2Measurement precision
If a wide variety of real-world test cases are encountered, then real-world performance is validated, but rare corner cases are insufficiently covered
Solution Approach 1:
The patent dynamically adjusts the composition of training data by combining real recordings with simulated scenarios. This dynamic approach allows the system to adapt the training dataset to cover both common real-world cases and rare corner cases, improving scenario coverage while maintaining real-world validation accuracy
Solution Approach 2:
The patent performs preliminary simulation of rare corner cases before actual deployment. By pre-generating simulated training data for edge cases that are difficult to capture in real-world testing, the system prepares robust models in advance, ensuring both real-world accuracy and comprehensive scenario coverage
Data Source
AI summary
Original sensor data is received from one or more sensors of a vehicle. Free space around the vehicle is identified according to the sensor data, such as by identifying regions where data points have a height below a threshold. A location for an object model is selected from the free space. A plane is fitted to sensor data around the location and the object model is oriented according to an orientation of the plane. Sensing of the object model by a sensor of the vehicle is simulated to obtain simulated data, which is then added to the original sensor data. Sensor data corresponding to objects that would have been obscured by the object model is removed from the original sensor data. Augmented sensor data may be used to validate a control algorithm or train a machine learning model.