Smartphone Image Compositing via Digital Matting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital imaging technologies, such as chroma keying, require a uniformly colored background and are limited to two-layer compositing, leading to issues like halos and incomplete separation of foreground and background, especially in self-portrait photography where the user's arm is often included.
Innovation Solution
A mobile device with dual cameras and a processor that can differentiate foreground from background using digital matting operations, allowing for the manipulation and composition of images within the device, including automatic insertion of foreground into background scenes, and enabling advanced adjustments like resizing and repositioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If chroma keying is used to separate foreground from background, then background removal is achieved, but the technique requires a uniformly colored background and is limited to two-layer compositing
Solution Approach 1:
The patent segments the image into multiple layers (foreground, background, and intermediate layers) rather than simple two-layer chroma keying. This allows sophisticated separation of foreground objects from complex backgrounds with multiple colors and textures, enabling precise background removal without requiring uniform coloring while maintaining high compositing flexibility.
Solution Approach 2:
The patent introduces depth information as an additional dimension to传统的 two-dimensional chroma keying approach. By incorporating depth maps or depth estimation, the system can separate foreground and background based on spatial depth rather than color alone, enabling accurate separation without uniform background requirements and supporting multi-layer compositing with different depth relationships.
2Adaptability or versatility
If traditional film-based matting is used, then multiple film strips can be combined, but registration errors cause halos and edge artifacts
Solution Approach 1:
The patent replaces the mechanical film-based matting system with a digital image processing system. Instead of physically aligning multiple film strips through mechanical means, the system uses digital image registration algorithms, pixel-level processing, and computational methods to combine multiple layers with precise alignment, eliminating halos and edge artifacts caused by mechanical registration errors.
3Ease of operation
If a single background color is used for chroma keying, then background segmentation is simplified, but the segmentation is limited to two layers only with no further separation possible
Solution Approach 1:
The patent applies multi-layer segmentation that divides the image into foreground, background, and intermediate layers with different depth or importance values. This sophisticated segmentation approach maintains operational simplicity through automated processing while enabling extensive layer separation capability, allowing independent manipulation of multiple layers without requiring complex manual segmentation.
Solution Approach 2:
The patent adds a depth or layer hierarchy dimension to the traditional single-layer chroma keying approach. By organizing pixels into multiple layers with depth relationships, the system simplifies operation through automated depth-based separation while enabling versatile layer manipulation, including independent transformation, compositing, and rendering of each layer.
Data Source
AI summary
Digital imagery is composed or composited within a smartphone or other portable electronic device to merge foreground and background subject matter acquired simultaneously with different cameras or acquired at different times with the same camera. A processor stores the digital imagery along with information derived or used by an operation to differentiate the foreground subject matter from the background scene, which may be a digital matting operation, and/or may involve finding an edge or an outline of the foreground subject matter. The foreground subject matter and the background scene may be stored as different graphical layers in the memory. A user control such as a touchscreen may enable a user to select and resize or reposition the foreground subject matter on a background scene of a separately acquired image. The digital imagery may constitute a still image or a sequence of images representative of motion video.