Synthetic Radar and Lidar Data From 3D Scenes for AV Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The development of control algorithms for autonomous vehicles is hindered by the limited nature of sample sensor data, which is often restricted to specific environments and cannot effectively evaluate or test algorithms for safe and robust operation, especially under unsafe conditions.
Innovation Solution
A system and method for generating synthetic sensor data, such as synthetic radar, lidar, and sonar data, from custom-designed 3D scene data, allowing for the creation, evaluation, and verification of controller designs in a virtual environment, including the use of a 3D scene generator, synthetic sensor data generator, and modeling environment to simulate various scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If physical sensor data collection is used, then data represents real-world conditions, but data diversity and coverage of unsafe conditions are limited
Solution Approach 1:
The patent creates virtual copies of physical sensors (radar, lidar, sonar) within a 3D simulated environment. These virtual sensors replicate the detection capabilities of physical sensors while operating in a virtual world, allowing unlimited data generation without physical constraints. The virtual sensors process 3D scene data to generate synthetic sensor outputs that mirror real sensor behavior.
Solution Approach 2:
The system dynamically changes environmental parameters in the 3D simulation including weather conditions (rain, snow, fog), lighting conditions, object positions, and sensor configurations. This allows generation of diverse training data covering edge cases and unsafe conditions that would be difficult or dangerous to capture in the physical world.
2Reliability
If more sample sensor data is collected to improve algorithm robustness, then more physical testing is required, but time and resource consumption increase
Solution Approach 1:
The system performs preliminary action by pre-generating comprehensive training datasets in the virtual environment before actual algorithm deployment. The 3D scenes, objects, and sensor data are prepared in advance, allowing algorithms to be trained extensively on diverse scenarios without requiring time-consuming physical data collection campaigns.
Solution Approach 2:
By using virtual copies of sensors and environments, the system can generate unlimited training data instantaneously. Multiple parallel simulations can run simultaneously, creating large datasets in fractions of the time required for physical data collection, thereby accelerating algorithm development and validation.
3Ease of operation
If physical data collection is used, then sensor behavior is accurate, but control over testing conditions and safety is reduced
Solution Approach 1:
The 3D simulation environment serves multiple functions: it acts as both the scene generator and the sensor model. The same virtual environment that contains 3D objects and scenes also hosts virtual sensors that detect these objects, providing a universal platform for generating both ground truth data and sensor data with complete control over testing conditions.
Solution Approach 2:
The patent introduces an intermediary layer of virtual sensing between the physical world and algorithm testing. Instead of directly using physical sensors, the system uses virtual sensors in a 3D simulation as an intermediary that preserves sensor detection principles while allowing complete control over environmental conditions and eliminating safety risks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods generate synthetic sensor data, such as synthetic radar, lidar, and/or sonar data from three dimensional (3D) scene data that may be custom designed. Reflectivity coefficients in the radar, lidar, and/or sonar spectrums may be determined for objects included in the 3D scene data. The reflectivity coefficients may be utilized by a game engine for computing the synthetic sensor data. The synthetic sensor data may be used in the creation, evaluation, and/or verification of a design for a controller or other system that utilizes such data.