Image Augmentation With Simulated Road Objects for AV Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous vehicle systems face challenges in training deep neural networks to accurately detect road debris and objects due to the scarcity of relevant training data, as collecting real-world images of road debris is difficult and dangerous, and requires extensive computational power and human effort.
Innovation Solution
The system augments real-world images with simulated objects to train machine learning models, allowing for a larger dataset with rich detail, enabling higher accuracy in object detection and reducing training time by using zero-shot learning to identify potential hazards without precise object classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-world images of road debris are collected for training, then training data authenticity is improved, but data quantity and collection safety deteriorate
Solution Approach 1:
The patent creates synthetic copies of road debris objects using 3D modeling and rendering techniques. These virtual objects are then integrated into real-world images to generate training data. This copying approach allows unlimited generation of training samples without the safety risks and scarcity issues of collecting actual road debris images.
Solution Approach 2:
The patent introduces simulated objects as an intermediary between real-world road debris and training data. Instead of directly using rare and dangerous real debris images, the system uses virtual representations that can be freely generated and manipulated, serving as a safe intermediary for training purposes.
2Measurement precision
If real-world road debris images are collected, then detection accuracy is improved, but collection difficulty and danger increase
Solution Approach 1:
The system creates virtual copies of road debris with controlled properties and scenarios. These synthetic images provide consistent, diverse, and safely generated training data that maintains detection accuracy requirements while eliminating collection difficulties and dangers associated with real-world debris.
Solution Approach 2:
The patent performs preliminary generation of diverse road debris scenarios in virtual environments before actual deployment. By pre-generating comprehensive training data covering various edge cases and conditions in simulation, the system prepares robust models without needing to collect dangerous real-world data during deployment.
3Measurement precision
If thousands of training data instances are collected, then DNN training accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The system generates synthetic training data copies that can be created rapidly through computational rendering rather than physical collection. This approach provides unlimited training instances with diverse scenarios without the time cost of actual field collection, enabling comprehensive training data generation on demand.
Solution Approach 2:
The patent pre-generates diverse training scenarios and objects in virtual environments before model training begins. This preliminary preparation creates a ready-to-use comprehensive dataset that covers edge cases and various conditions, eliminating the need for time-consuming data collection during the training process.
Data Source
AI summary
In various examples, systems and methods are disclosed that preserve rich, detail-centric information from a real-world image by augmenting the real-world image with simulated objects to train a machine learning model to detect objects in an input image. The machine learning model may be trained, in deployment, to detect objects and determine bounding shapes to encapsulate detected objects. The machine learning model may further be trained to determine the type of road object encountered, calculate hazard ratings, and calculate confidence percentages. In deployment, detection of a road object, determination of a corresponding bounding shape, identification of road object type, and/or calculation of a hazard rating by the machine learning model may be used as an aid for determining next steps regarding the surrounding environment—e.g., navigating around the road debris, driving over the road debris, or coming to a complete stop—in a variety of autonomous machine applications.