3D Engine Synthetic Dataset Generation for AI Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current synthetic imagery and datasets lack diversity in backgrounds and object pose and lighting conditions, making them unsuitable for training deep learning models, especially for rare targets or those difficult to find in real-world scenarios, and existing augmentation methods are labor-intensive or result in unrealistic imagery.
Innovation Solution
An end-to-end pipeline using a 3D engine to generate synthetic datasets by combining real satellite imagery with procedurally generated 3D objects, processed by a generative neural network to maintain global image consistency and realism, allowing for the creation of diverse and realistic training datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional image augmentation methods (geometric transforms, CGI, physics-based synthesis) are used, then synthetic data can be generated, but background diversity remains limited and the process is labor-intensive
Solution Approach 1:
The patent transitions from 2D image manipulation to 3D scene generation using Unreal Engine. By creating three-dimensional environments with procedurally generated objects and real-time rendering, the system achieves diverse backgrounds and object poses that cannot be obtained through traditional 2D geometric transforms. The 3D scene graph allows infinite variations in object placement, orientation, and environmental context.
Solution Approach 2:
The patent uses real satellite imagery as source material and creates synthetic versions by compositing 3D rendered objects onto real background images. This copying approach preserves the authentic texture and lighting characteristics of real-world backgrounds while adding procedurally generated objects, thereby maintaining background diversity without requiring entirely synthetic environments.
2Quantity of substance
If existing data augmentation approaches are used, then data generation is possible, but the imagery becomes unrealistic
Solution Approach 1:
The patent merges real satellite imagery with procedurally generated 3D objects in a unified rendering pipeline. By combining authentic photographic backgrounds with consistently rendered 3D objects that match the lighting and perspective, the system creates synthetic images that maintain photorealistic quality. The compositing process integrates shadows, reflections, and lighting effects to ensure visual consistency.
Solution Approach 2:
The patent employs photometric rendering with adjustable lighting parameters, camera angles, and environmental conditions. By changing these rendering parameters to match real-world physics and lighting models, the synthetic imagery achieves realistic appearance. The Unreal Engine allows precise control over illumination, material properties, and atmospheric effects to maintain authenticity.
3Measurement precision
If massive amounts of labeled examples are collected from real-world data, then training accuracy improves, but the process becomes labor-intensive and time-consuming
Solution Approach 1:
The system generates its own training data autonomously through procedural generation. The Unreal Engine automatically creates diverse 3D scenes with labeled objects based on predefined templates and parameters, eliminating the need for manual annotation. The pipeline self-generates synthetic datasets with ground truth labels embedded in the 3D scene graph, significantly reducing human labor requirements.
Solution Approach 2:
The patent performs preliminary actions by pre-rendering 3D objects and environments with embedded labels before training begins. The procedural generation system creates ready-to-use training data with annotations already incorporated into the 3D scene structure, eliminating the need for subsequent manual labeling. This preliminary data preparation accelerates the overall training process.
Data Source
AI summary
Embodiments regard improving machine learning (ML) training dataset generation. A method includes receiving or generating, by a scene editor, an image, augmenting, using procedural generation, the image to include a three-dimensional (3D) model of an object resulting in a synthetic image, and generating, by a generative model and based on the synthetic image, a realistic image.