Simulated Environments for Autonomous Vehicle DNN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for training deep neural networks (DNNs) in autonomous vehicles rely on real-world data, which is limited, dangerous, and time-consuming, and do not guarantee universally accurate results, especially for safety-critical tasks like obstacle avoidance.
Innovation Solution
Utilize simulated environments to train, test, and verify DNNs using physical and virtual sensor data, allowing for the creation of challenging and dangerous scenarios, and ensure the DNNs perform accurately by replicating real-world conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-world data is used to train DNNs, then the training data reflects actual driving conditions, but the amount of available data is limited and dangerous scenarios are difficult to reproduce
Solution Approach 1:
The patent creates virtual copies of real-world driving environments through simulation. Synthetic training data is generated by replicating real driving scenarios, including dangerous situations, in a virtual environment. This allows unlimited reproduction of rare and hazardous events without needing to physically recreate them, solving the contradiction between data representativeness and data quantity.
Solution Approach 2:
The system performs preliminary training in simulated environments before deploying to real-world conditions. By pre-training DNNs on synthetic data that includes edge cases and dangerous scenarios, the system prepares the model in advance for situations that would be difficult or unsafe to encounter during actual driving, thus expanding the effective training dataset.
2Measurement precision
If real-world testing is conducted to verify DNN safety, then actual performance can be evaluated, but the testing process is dangerous and time-consuming
Solution Approach 1:
The patent uses virtual copies of the autonomous vehicle and its environment for testing purposes. By evaluating DNN performance on synthetic data and in simulated scenarios, the system can repeatedly test edge cases and dangerous situations without risking physical safety or consuming real-world driving time, thus achieving precise performance measurement efficiently.
Solution Approach 2:
The system performs preliminary verification and validation of DNN safety in simulation before real-world deployment. By conducting extensive safety testing in the virtual environment, including rare and hazardous scenarios, the system reduces the need for prolonged real-world testing, thereby decreasing time loss while maintaining evaluation accuracy.
3Reliability
If extensive real-world testing is performed to achieve acceptable safety levels, then DNN reliability improves, but the process becomes increasingly difficult and dangerous
Solution Approach 1:
The patent replaces physical testing with virtual testing by creating detailed simulations of the autonomous vehicle and its operating environment. This allows comprehensive safety verification including dangerous scenarios without exposing the actual vehicle to harm, thus improving reliability assessment while eliminating testing risks.
Solution Approach 2:
The system uses simulation to cushion against potential real-world failures by pre-identifying and addressing safety issues in the virtual environment. By testing extensively in simulation before deployment, the system prepares for and mitigates potential hazards without the need for dangerous real-world trial-and-error, thereby improving safety while reducing testing risks.
Data Source
AI summary
In various examples, physical sensor data may be generated by a vehicle in a real-world environment. The physical sensor data may be used to train deep neural networks (DNNs). The DNNs may then be tested in a simulated environment—in some examples using hardware configured for installation in a vehicle to execute an autonomous driving software stack—to control a virtual vehicle in the simulated environment or to otherwise test, verify, or validate the outputs of the DNNs. Prior to use by the DNNs, virtual sensor data generated by virtual sensors within the simulated environment may be encoded to a format consistent with the format of the physical sensor data generated by the vehicle.