Image Training Data Recoloring to Bridge the Synthetic-Real Domain Gap
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for image processing, particularly those trained on synthetic data, often struggle with generalization to real-world images due to a domain gap, and lack sufficient training examples, leading to poor performance in inference tasks.
Innovation Solution
Augment training data by generating recolored images using colorization models, particularly image-to-image diffusion models, to create a diverse and photorealistic set of training examples, including grayscale and RGB images, to enhance the training dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If training data is augmented by generating recolored images using colorization models, then the variety and realism of training examples increases, but the complexity of the data processing pipeline increases
Solution Approach 1:
The patent applies preliminary action by pre-training colorization models on large datasets of real images before using them to augment the training data. This preliminary training of the colorization model allows the system to automatically generate photorealistic recolored versions of synthetic training images without requiring manual intervention, thereby increasing training data variety while managing pipeline complexity through automation.
2Productivity
If image processing models are trained on synthetic data, then training efficiency increases, but generalization performance to real-world images deteriorates
Solution Approach 1:
The patent applies color changes by using colorization models to generate multiple recolored versions of synthetic training images. This process transforms the appearance of training images to better match real-world photorealistic images while maintaining the underlying semantic content. The colorization step bridges the domain gap between synthetic and real images, improving generalization performance without sacrificing training efficiency.
3Reliability
If the training dataset size is increased by adding recolored images, then model performance improves, but the computational resources required increase
Solution Approach 1:
The patent applies copying by generating multiple recolored versions of each synthetic training image using colorization models. Instead of collecting new real images (which would be resource-intensive), the system creates synthetic copies with varied color appearances that better represent real-world diversity. This approach increases effective training data size and improves model performance while maintaining computational efficiency by reusing the original synthetic images as bases.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating, using one or more coloraF_ization models, an augmented set of training data. One of the methods includes receiving a plurality of training examples for training an image processing model, each training example comprising an image and a corresponding ground-truth output for the image; generating, for each image, a respective grayscale image; generating, for each respective grayscale image, one or more recolored images using one or more colorization models; and generating an augmented set of training data for training the image processing model that comprises a plurality of additional training examples.