Synthetic Sensor Image Generation for Realistic NIR Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating training data for machine learning models in vehicle interior sensing face challenges such as high cost and time consumption in manual annotation, and significant domain shift when transitioning from simulated to real-world data, particularly with Near-Infrared sensors.
Innovation Solution
A system comprising a simulator block and a generator block, where the simulator block creates simulated image data with label metadata, and the generator block uses generative AI to manipulate characteristics to align with real-world sensor data, reducing domain shift and automating annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual annotation of real sensor data is used, then data accuracy and reliability are improved, but time consumption and cost increase significantly
Solution Approach 1:
The patent creates synthetic copies of real-world sensor data through simulation environments. These synthetic data copies replicate the characteristics and patterns of real sensor data while being automatically generated, eliminating the need for manual annotation. The simulation engine generates realistic sensor readings, images, and metadata that can be used for training machine learning models without requiring human annotators to process actual real-world data.
Solution Approach 2:
The patent performs data generation and annotation in advance through simulation before the actual need for training data arises. The simulation environment pre-generates diverse scenarios, edge cases, and varying conditions that would be time-consuming to capture and annotate manually in the real world. This preliminary generation of annotated data ready for model training significantly reduces the time required when deployment is needed.
2Loss of time
If simulated data is used for training, then annotation time and cost are reduced, but domain shift between simulated and real-world data increases
Solution Approach 1:
The patent systematically varies simulation parameters to match real-world conditions more closely. This includes adjusting sensor characteristics, environmental conditions, lighting scenarios, and physical parameters in the simulation to replicate the statistical properties and distributions of real sensor data. By carefully tuning these parameters, the synthetic data maintains domain alignment with real-world data while preserving the benefits of automated generation.
Solution Approach 2:
The patent implements feedback loops where the performance of models trained on simulated data is evaluated against real-world data performance. This feedback is used to iteratively improve the simulation parameters and data generation processes. The system learns from the domain gap and adjusts the simulation to better replicate real-world characteristics, continuously reducing the domain shift over time.
3Adaptability or versatility
If diverse training scenarios are generated manually, then data coverage and model robustness are improved, but resource requirements and complexity increase
Solution Approach 1:
The patent creates a universal simulation platform that can generate multiple types of sensor data (images, depth maps, thermal data, LIDAR points) and simulate various scenarios (different lighting conditions, weather, occlusions, edge cases) through a single integrated system. This multi-functional simulation engine replaces the need for multiple separate data collection systems and manual scenario design processes, reducing overall system complexity while maintaining comprehensive data coverage.
Solution Approach 2:
The patent employs dynamic scenario generation where the simulation automatically creates and varies training scenarios based on predefined parameters and randomization. Instead of manually designing each scenario, the system dynamically generates diverse situations including rare edge cases, varying environmental conditions, and different object configurations. This dynamic approach ensures comprehensive coverage without requiring complex manual setup for each scenario type.
Data Source
Figure 1~2
AI summary
System (1) for creating image data (10). A simulator block (2) is provided for outputting simulated image data (3) of at least one scene in a simulated environment, wherein the simulated image data (3) comprises simulated sensor data (4) of one or more simulated sensors in the simulated environment and label data (5) providing information associated with the respective scene. A generator block (9) for generating, from the simulated image data (3), image data (10) of one or more scene variants of the at least one scene, wherein the generator block (9) is configured to manipulate parts of the simulated sensor data (3) to increase, in the one or more scene variants, characteristics associated with real-world sensor data.