Polarized Light Alpha Matte Generation for Accurate Video Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital image and video matting systems suffer from inaccuracies, inefficiencies, and inflexibilities in generating composite images, requiring excessive time, computing resources, and manual labor for large-scale video datasets, especially with fine details or rapidly moving objects.
Innovation Solution
A video matte preparation system utilizing polarized light and digital videos to generate alpha matte animations and composite videos by spatially and temporally cropping, correcting intensity values, and harmonizing foreground subjects with replacement backgrounds, employing machine-learning models for efficient and accurate matte generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional chroma key compositing is used to generate composite images, then the process is simple to implement, but the accuracy and realism of the composite images deteriorate
Solution Approach 1:
The patent changes the physical parameter of light polarization angle to differentiate foreground from background. By capturing images at multiple polarization angles (0°, 45°, 90°, 135°) and using polarized lighting, the system creates distinct intensity variations that enable accurate matte generation without relying on color keying, thereby improving composite image accuracy while maintaining manageable system complexity
Solution Approach 2:
The patent introduces polarized light as an intermediary medium between the camera and the scene. The polarized lighting setup acts as a mediator that encodes spatial information about foreground and background objects through polarization states, allowing the matting algorithm to accurately separate subjects from backgrounds without direct color or luminance-based methods
2Productivity
If conventional matting systems process large-scale video datasets, then comprehensive coverage is achieved, but the processing time and computing resources increase excessively
Solution Approach 1:
The patent applies preliminary spatial and temporal cropping to video frames before processing. By identifying and removing irrelevant regions (such as uniform background areas) and leveraging temporal coherence between consecutive frames, the system reduces the amount of data that requires intensive computation, thereby significantly decreasing processing time and computing resource requirements while maintaining matte quality
Solution Approach 2:
The patent segments the video processing task into distinct stages: polarization-based matte generation, spatial cropping, temporal filtering, and composite generation. This segmentation allows each stage to be optimized independently, with the polarization method handling the computationally intensive separation task efficiently and subsequent stages refining the results with minimal additional processing
3Measurement precision
If polarized light with multiple angles is used to generate alpha mattes, then the accuracy of matte generation is improved, but the device complexity and processing requirements increase
Solution Approach 1:
The patent uses periodic polarization angle variations (0°, 45°, 90°, 135°) in a systematic sequence to capture multiple polarized images. This periodic approach allows the system to extract foreground-background separation information through trigonometric relationships in the intensity variations, achieving high precision alpha matte generation while keeping the lighting and capture system relatively simple and repetitive
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system improves accuracy and efficiency in generating high-quality alpha mattes and composite videos, reducing processing demands and manual interactions, enabling rapid creation of large-scale training datasets with fine details and complex objects.
Implementation Method 1
a polarized light source to generate marked and unmarked frames of polarized digital videos having different polarization angles
Data Source
AI summary
The present disclosure relates to systems, non-transitory computer-readable media, and methods for efficiently automating the preparation of accurate alpha matte animations and modified digital videos utilizing polarized light. For example, the disclosed systems obtain a plurality of polarized digital videos portraying an animation of a foreground subject backlit by a polarized light source. In some embodiments, the disclosed systems generate a plurality of corrected polarized digital videos by adjusting intensity values of the plurality of polarized digital videos based on intensity differences across the plurality of polarized digital videos. The disclosed systems generate an alpha matte animation comprising a plurality of alpha mattes from the plurality of corrected polarized digital videos or from the plurality of polarized digital videos. Utilizing the alpha matte animation, the disclosed systems generate a modified digital video by combining the animation of the foreground subject and a replacement background.