Synthetic Image Generation Using GAN Object Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating synthetic digital images for visual object detectors fail to efficiently create training data with strict connections and spatial relations between objects, limiting the robustness of these detectors in scenarios requiring precise object relationships.
Innovation Solution
A method using a Generative Adversarial Network (GAN) to generate synthetic objects by replacing objects in original images with new objects, defined by binary segmentation masks and textures, to create realistic training data with precise object relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rendering engines are used to generate synthetic digital images with strict spatial relations between objects, then the realism and domain relevance of training data is improved, but the complexity and computational cost of the rendering process increases
Solution Approach 1:
The patent uses copy-paste operations to transfer object segments between images, creating synthetic images with correct spatial relationships through direct copying rather than complex rendering simulations. This maintains spatial accuracy while avoiding rendering complexity.
Solution Approach 2:
The patent segments objects from source images using segmentation masks, then recombines these segments into target images. This segmentation approach simplifies the process of creating images with strict spatial relations by working with discrete object segments rather than full rendering pipelines.
2Reliability
If large amounts of annotated training data are collected manually, then the training quality of visual object detectors is improved, but the time and cost consumption increases
Solution Approach 1:
The patent automatically generates synthetic training data by copying and pasting segmented objects into different scenes, eliminating the need for manual annotation of large datasets. This provides unlimited training data without annotation time costs.
Solution Approach 2:
The system performs self-annotation by automatically generating synthetic images with known ground truth labels through the copy-paste process, eliminating the need for external human annotators and providing infinite training data automatically.
3Reliability
If simple data augmentation techniques are applied to training data, then the robustness of visual object detectors is improved, but the domain relevance of the augmented data decreases
Solution Approach 1:
The patent merges simple augmentation operations with domain-specific copy-paste operations, combining the robustness benefits of augmentation with the domain relevance of synthetic scene generation. This creates augmented data that is both robust and domain-specific.
Data Source
AI summary
There is provided mechanisms for rendering a synthetic digital image. A method is performed by an image processing device. The method comprises obtaining (S102) an original digital image. The original digital image represents a depiction of a visual scene that comprises at least one object. The method comprises identifying (S104) a first object in the original digital image. The first object is bounded by a bounding box in the original digital image. The method comprises generating (S106) a synthetic object by a Generative Adversarial Network processing the first object and a second object. The synthetic object has a shape defined by a binary segmentation mask as applied to the first object. The synthetic object has a texture and colour based on the second object. The method comprises rendering (S108) the synthetic digital image by, in the original digital image, replacing the first object in the bounding box with the synthetic object.