Synthetic Shadow Training Data for Diverse Image Scene Generalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing shadow detection and removal methods face challenges due to inefficient use of network capacity and limited dataset size, which affects generalization across various scenarios, and the creation of training data is time-consuming and lacks variety in scenery and shadow types.
Innovation Solution
A method for generating a training dataset by collecting images from various sources, extracting shadow-free patches, and synthesizing shadows on these patches, using a pre-trained model to create a synthetic dataset that is diverse and effective for pretraining machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If physical photography method is used to create training data by photographing objects with and without shadows, then shadow detection and removal training data can be obtained, but the method lacks variety in scenery types and shadow shapes, and requires the camera to be perfectly still to capture the same scene
Solution Approach 1:
The patent uses synthetic image generation to create training data pairs instead of physical photography. A shadow-free reference image is copied and modified by overlaying synthesized shadow masks to create corresponding shadowed images, eliminating the need for physical rephotography while maintaining pixel-level correspondence
Solution Approach 2:
The patent varies shadow parameters (position, shape, intensity, opacity) independently of the reference image to generate diverse training data. By changing shadow mask parameters rather than physical camera parameters, the system achieves both pixel correspondence and scenery variety
2Measurement precision
If a small real-life benchmark dataset is used for training, then the dataset can be obtained with real-world accuracy, but the dataset size is limited which harms generalization across various scenarios
Solution Approach 1:
The patent copies real shadow-free reference images and applies synthesized shadow masks to create expanded training datasets. This maintains the authenticity of the base images while generating multiple varied training examples from each reference image
Solution Approach 2:
The patent pre-computes and stores a library of shadow masks with varying parameters before training. These pre-prepared shadow masks are then applied to reference images to efficiently generate diverse training data without requiring real-time shadow synthesis during training
3Reliability
If machine learning methods are used for shadow detection and removal, then state-of-the-art performance can be achieved in computer vision tasks, but a large amount of training data is required which is difficult and time-consuming to produce
Solution Approach 1:
The patent copies existing shadow-free images and applies synthesized shadow masks to create training pairs automatically. This eliminates the time-consuming process of physical photography while maintaining image quality and pixel correspondence
Solution Approach 2:
The patent replaces the mechanical photography process with computational image synthesis. Instead of physically photographing scenes with and without shadows, the system uses algorithmic shadow mask generation and overlay, dramatically reducing data production time
Data Source
AI summary
Broadly speaking, embodiments of the present techniques provide methods for performing shadow detection and/or removal in images. In particular, the present techniques provide a computer-implemented method for generating a synthetic training dataset for training a machine learning, ML, model using federated learning to perform shadow detection (and optionally removal), and methods for training the ML model using the generated training dataset. Advantageously, the method to generate a training dataset enables a diverse training dataset to be generated while maintaining user data privacy, where the diversity refers to the variety of scenes containing shadows.