3D Background Mask Generation for Automated Image Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating labeled image datasets for machine learning and computer vision systems require significant human-hours for image capture and labeling, and using physical green screens introduces limitations such as color bias and inefficiency in data collection.
Innovation Solution
A system that generates background replacement masks in 3D mesh scenes using augmented reality, allowing users to manually or automatically identify background regions and replace them with new backgrounds, reducing the need for manual labeling and overcoming limitations of physical green screens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If physical green screens are used for background replacement, then background uniformity is improved, but device complexity and operational difficulty increase
Solution Approach 1:
The patent uses a 3D mesh representation of the scene as a virtual copy of the physical environment. Instead of requiring physical green screens, the system creates a digital model that can be manipulated and processed computationally, achieving background uniformity through software-based mask generation rather than physical hardware constraints
Solution Approach 2:
The patent replaces the mechanical/physical green screen system with a computational approach using 3D mesh processing and image processing algorithms. The background replacement is achieved through software-based mask identification and pixel value manipulation rather than physical color-controlled environments
2Measurement precision
If manual labeling of each image is performed, then labeling precision is improved, but loss of time increases
Solution Approach 1:
The system performs self-labeling by automatically generating labels through 3D mesh processing and mask-based pixel value assignment. The 3D annotations and mask data enable the system to autonomously create labeled images without requiring human annotators to manually process each image, significantly reducing time consumption while maintaining precision through algorithmic consistency
Solution Approach 2:
The patent performs preliminary actions by capturing 3D mesh data and generating masks in advance, before the actual labeling process. The 3D annotations and background masks are prepared beforehand, allowing rapid generation of labeled images through simple pixel value updates rather than requiring time-consuming manual analysis of each image
3Productivity
If 3D mesh processing is used for background identification, then productivity is improved, but device complexity increases
Solution Approach 1:
The 3D mesh representation serves multiple functions simultaneously: it provides spatial understanding of the scene, enables mask generation for background identification, supports pixel value manipulation for background replacement, and facilitates annotation projection. This multi-functionality increases productivity by consolidating multiple processing steps into a single unified representation rather than requiring separate complex systems for each function
Data Source
AI summary
A system captures, by a recording device, a scene with physical objects, the scene displayed as a three-dimensional (3D) mesh. The system marks 3D annotations for a physical object and identifies a mask. The mask indicates background pixels corresponding to a region behind the physical object. Each background pixel is associated with a value. The system captures a plurality of images of the scene with varying features, wherein a respective image includes: two-dimensional (2D) projections corresponding to the marked 3D annotations for the physical object; and the mask based on the associated value for each background pixel. The system updates the value of each background pixel with a new value. The system trains a machine model using the respective image as generated labeled data, thereby obtaining the generated labeled data in an automated manner based on a minimal amount of marked annotations.