Occlusion Key Generation Using Background Pixel Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating occlusion keys in video match moving, such as chroma-keying and deep neural networks, fail to accurately differentiate foreground and background pixels in dynamic backgrounds like digital LED screens, leading to suboptimal results.
Innovation Solution
A system that creates an occlusion key using a background pixel map by transforming current images into a stationary stream, segmenting them into foreground and background pixels, and generating a history of background pixel colors, which is then used to determine pixel classification in real-time images for accurate occlusion key creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If chroma-keying or deep neural networks are used to generate occlusion keys, then the process can be automated, but accuracy deteriorates in dynamic backgrounds like digital LED screens
Solution Approach 1:
The patent applies preliminary action by capturing and storing background pixel data before the actual occlusion key generation process. A background pixel map is created in advance by collecting pixel color information from multiple historical frames at the same spatial location, establishing a reference database that is then used to accurately classify foreground and background pixels in real-time without requiring complex automated analysis during the main process
Solution Approach 2:
The patent uses copying by creating a stationary stream of images that replicates the background scene at the target location across multiple frames. This copied stationary stream serves as a reference model that can be compared against current frames to identify foreground objects, replacing the need for complex neural network-based automated segmentation
2Device complexity
If traditional computer vision techniques are used, then the system remains simple, but accuracy deteriorates with dynamic backgrounds
Solution Approach 1:
The patent maintains system simplicity while improving accuracy by performing preliminary background capture and creating a background pixel map before processing. This pre-established reference data allows accurate foreground-background differentiation in dynamic scenes without requiring complex real-time analysis algorithms
Solution Approach 2:
The patent introduces an intermediary element - the background pixel map - that mediates between the simple camera input and the occlusion key generation process. This intermediary stores historical background pixel information and enables accurate classification by comparing current pixels against the stored reference, bridging the gap between simple system architecture and high accuracy requirements
3Measurement precision
If a background pixel map with historical data is used, then accuracy improves over time, but data processing requirements increase
Solution Approach 1:
The patent applies taking out by extracting only the essential background pixel color information from historical frames and storing it in a compact background pixel map. Instead of processing and storing entire historical image sequences, the system extracts and retains only the relevant pixel color data at the target location, reducing data volume while maintaining classification accuracy
Solution Approach 2:
The patent implements local quality by focusing data collection and processing only at the specific target location where the occlusion key is needed, rather than processing the entire image frame. This localized approach collects background pixel data from a specific region of interest, significantly reducing the quantity of data that must be stored and processed while maintaining high accuracy for the intended application
Data Source
AI summary
A system and method of obtaining an occlusion key using a background pixel map is disclosed. A target image containing a target location suitable for displaying a virtual augmentation is obtained. A stream of current images are transformed into a stationary stream having the camera pose of the target image. These are segmented using a trained neural network. The background pixel map is then the color values of background pixels found at each position within the target location. An occlusion key for a new current image is obtained by first transforming it to conform to the target image and then comparing each pixel in the target location with the color values of background pixels in the background pixel map. The occlusion key is then transformed back to conform to the current image and used for virtual augmentation of the current image.