Augmenting Vehicle Sensor Data with Simulated Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in testing and validation of driver assistance technologies and autonomous vehicles lies in simulating rare and risky scenarios, as existing simulations lack perfect sensor models and traffic models, resulting in unrealistic data.
Innovation Solution
The system generates improved scenarios by augmenting recorded sensor data with simulated data, using sensor models and object models to create realistic and diverse testing environments for training machine learning models, particularly for LIDAR, camera, and RADAR sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If detailed simulations are used to prove out rare scenarios, then the ability to test rare and risky scenarios is improved, but the realism of generated data deteriorates due to lack of perfect sensor models and traffic models
Solution Approach 1:
The patent merges real sensor recordings with simulated sensor data to create augmented datasets. Real recordings provide authentic sensor characteristics and environmental nuances, while simulations contribute controlled rare scenarios and object instances. This combination resolves the contradiction by maintaining data realism through real recordings while enabling rare scenario testing through simulation augmentation.
Solution Approach 2:
The patent creates synthetic copies of rare scenarios using simulation, then augments these copies with real sensor data characteristics. The simulation generates foundational scenario structures (objects, positions, relationships) that are then enriched with real sensor measurements, allowing rare scenarios to be tested while preserving measurement precision through real data contamination.
2Measurement precision
If real-world testing is performed to gather diverse data, then the realism of sensor data is improved, but the safety and feasibility deteriorate due to rare and risky scenarios
Solution Approach 1:
The patent uses simulation as an intermediary layer between real-world testing and rare scenario generation. Simulation acts as a safe intermediary that can model rare scenarios without physical risk, while still providing realistic data through augmentation with real sensor characteristics. This mediator approach allows diverse data collection without exposing systems to actual safety risks.
Solution Approach 2:
The patent performs preliminary simulation of rare scenarios before real-world testing. By pre-generating and validating rare scenario handling in simulation, the system prepares safe test cases that can then be executed in the real world with reduced risk. The preliminary simulation action filters out potentially harmful scenarios before they reach physical testing.
Data Source
AI summary
Original sensor data is received from one or more sensors of a vehicle. Free space around the vehicle is identified according to the sensor data, such as by identifying regions where data points have a height below a threshold. A location for an object model is selected from the free space. A plane is fitted to sensor data around the location and the object model is oriented according to an orientation of the plane. Sensing of the object model by a sensor of the vehicle is simulated to obtain simulated data, which is then added to the original sensor data. Sensor data corresponding to objects that would have been obscured by the object model is removed from the original sensor data. Augmented sensor data may be used to validate a control algorithm or train a machine learning model.