Dynamic Chroma Key Using Depth Maps and Least Used Color
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Chroma key processing in image sensors often fails to effectively separate foreground from background when the color of the foreground subject is similar to the background, causing the subject to disappear in images or videos.
Innovation Solution
The method involves generating a depth map and color histogram from frame data captured by image sensors, identifying the least used or not used color in the foreground, and dynamically replacing the background with this color to prevent foreground objects from disappearing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If chroma key processing is used to replace background based on color, then background replacement is achieved, but foreground objects with similar colors to the background disappear or blend into the background
Solution Approach 1:
The patent transitions from 2D color-based chroma key processing to 3D depth-aware processing by incorporating depth maps. The system uses depth information to distinguish foreground objects from background even when their colors are similar, adding a spatial dimension to the background replacement process. This resolves the contradiction by enabling accurate background replacement without causing foreground objects to disappear.
Solution Approach 2:
The patent dynamically adjusts the background color selection by analyzing the color histogram of foreground objects and identifying the least used color. Instead of using a fixed background color, the system adapts the background color parameter based on the specific scene content, ensuring that the replacement background color does not conflict with foreground object colors. This dynamic parameter adjustment maintains foreground visibility while achieving effective background replacement.
2Ease of manufacture
If a fixed background color is used for replacement, then processing is simple, but it cannot adapt when foreground object colors change or conflict with the background color
Solution Approach 1:
The patent implements dynamic background color adaptation by continuously analyzing the color histogram of foreground objects and selecting the least used color as the background color. This dynamic approach allows the background replacement system to adapt to changing scene conditions and foreground object colors, resolving the contradiction between processing simplicity and adaptability. The system automatically adjusts to different scenes without requiring manual reconfiguration.
Solution Approach 2:
The system performs self-adjustment by automatically analyzing the captured image data, generating depth maps, creating color histograms, and selecting appropriate background colors without external intervention. The algorithm autonomously identifies the least used color in the foreground and uses it as the background color, enabling the system to adapt to different scenes independently. This self-service capability maintains ease of use while achieving high adaptability.
Data Source
AI summary
A method, a device, and a computer program product for background image replacement in image data. The method includes receiving, from at least one image sensor of a device, frame data. The device generates a first depth map based on the received frame data. The device generates first foreground image data and first background image data, based on the first frame data and the first depth map. The method further includes determining a least used color in the first foreground image data. The least used color is identified as a first background color. The method also includes generating modified frame data by replacing the first background image data with the first background color and storing the modified frame data to a memory of the device.