AR Synthetic Data Rendering for Realistic ML Training Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models trained on low-quality synthetic images exhibit poorer performance compared to those trained on real imagery due to the lack of realism in synthetic data, necessitating improved methods for generating photo-realistic synthetic data.
Innovation Solution
Utilizing augmented reality (AR) techniques to generate synthetic training data by combining 3D models of virtual objects with real-world environments, automatically labeling the data with object positions and orientations, and storing the augmented images in machine learning datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If typical techniques are used to generate synthetic images, then the generation process is economical and scalable, but the quality and realism of the synthetic images deteriorate
Solution Approach 1:
The patent introduces an AR rendering engine as an intermediary between simple synthetic image generation and complex real image collection. This engine combines 3D virtual objects with photorealistic background images to create intermediate synthetic images that maintain both generation efficiency and improved visual realism, resolving the contradiction between productivity and manufacturing precision
Solution Approach 2:
The patent creates copies of real-world scenes by rendering 3D virtual objects into photorealistic backgrounds. These copied scenes preserve the visual characteristics of real images while maintaining the scalability and cost-effectiveness of synthetic generation, addressing both the need for realism and efficient production
2Productivity
If low-quality synthetic images are used for training, then data generation is fast and inexpensive, but machine learning model performance deteriorates
Solution Approach 1:
The AR rendering engine serves as a mediator that produces intermediate-quality synthetic images with improved realism compared to traditional synthetic methods. These images maintain fast generation speeds while providing sufficient visual fidelity for effective model training, thus preserving both productivity and reliability
Solution Approach 2:
The patent changes key parameters of synthetic image generation by incorporating photorealistic backgrounds, lighting effects, and camera parameters into the rendering process. These parameter changes significantly improve image quality and model training effectiveness while maintaining the efficiency benefits of synthetic data generation
Data Source
AI summary
The present disclosure is directed to systems and methods for generating synthetic training data using augmented reality (AR) techniques. For example, images of a scene can be used to generate a three-dimensional mapping of the scene. The three-dimensional mapping may be associated with the images to indicate locations for positioning a virtual object. Using an AR rendering engine, implementations can generate an augmented image depicting the virtual object within the scene at a position and orientation. The augmented image can then be stored in a machine learning dataset and associated with a label based on aspects of the virtual object.