Alpha Matte Generation Using Polarized Light
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing alpha matting techniques struggle with accurately extracting foreground objects from images and video, particularly when dealing with fine structures, semi-transparent objects, and optical and motion blur, leading to inaccurate masks and visual artifacts.
Innovation Solution
The use of a polarization camera capturing multiple polarized images with different polarization angles allows for the generation of highly accurate alpha mattes by modeling polarized reflections and applying intensity correction, thereby capturing fine details and motion without requiring additional post-processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional alpha matting techniques are used, then the processing is simpler and faster, but the mask accuracy deteriorates especially for fine structures and semi-transparent objects
Solution Approach 1:
The patent introduces polarized light as an intermediary medium between the camera and the scene. By using polarized backlighting and capturing images through polarizing filters at different angles, the system creates intermediate polarized images that encode depth and transparency information. This intermediary approach enables accurate extraction of fine structures and semi-transparent objects without requiring complex post-processing algorithms.
Solution Approach 2:
The patent adds the polarization dimension to traditional imaging. Instead of only capturing intensity information, the system captures polarized images at multiple angles (0°, 45°, 90°, 135°), transforming a 2D intensity problem into a 3D polarization space problem. This dimensional expansion provides additional constraints that improve mask accuracy for difficult cases while maintaining computational tractability.
2Measurement precision
If multiple polarized images are captured and processed, then the alpha matte accuracy improves, but the processing time and energy consumption increase
Solution Approach 1:
The patent performs preliminary organization of the four polarized images into specific groups (first and second groups with orthogonal relationships) before solving the linear system. The method pre-establishes the mathematical relationships and constraints based on polarization theory, allowing the alpha matte to be solved directly through a systematic linear equation approach rather than requiring iterative optimization, thus reducing processing time.
3Reliability
If polarized reflections are modeled and constrained, then the visual artifacts are reduced, but the system complexity increases
Solution Approach 1:
The patent incorporates feedback mechanisms by using the known polarization states of the backlight and filters to constrain the unknown polarized reflections from the scene. The system uses the measured intensities from multiple polarized images to infer and remove polarized reflection components, providing feedback that improves visual quality by eliminating artifacts while maintaining a relatively simple system architecture.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach results in highly accurate alpha mattes that capture fine details and motion, reducing visual artifacts and eliminating the need for manual post-production edits, while also saving processing time and energy.
Implementation Method 1
a polarization camera includes polarization filters integrated with the sensor of the camera such that multiple polarized images are captured simultaneously. Each image has a different polarization angle.
Implementation Method 2
When a subject is backlit by a polarized light source, the backgrounds of the polarized images vary depending on the angle of polarization.
Data Source
AI summary
Embodiments are disclosed for generating an instant mask from polarized input images. A method of generating the mask includes obtaining, by a user interface manager, a set of polarized input images, the set of polarized images including background images depicting a background and subject images depicting a foreground object in front of the background and backlit by a polarized light source, the set of polarized input images including more than two polarized images, each associated with a different polarization angle, determining, by a matte optimization manager, an at least partially polarized reflection on the foreground object based on the set of polarized input images, determining, by the matte optimization manager, an alpha matte based at least on intensity values associated with each of the set of polarized input images such that the alpha matte isolates the foreground object and removes the at least partially polarized reflection, and generating, by a matte generator, an output matte based on the alpha matte.