Background Replacement Using Pixel Magnitude Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video communication systems often capture unwanted background images of participants, compromising privacy and making it difficult to use branded or promotional backgrounds, especially when using 2D cameras with backgrounds that are not predominantly green.
Innovation Solution
A telecommunication device with a background modifier that segments captured images into foreground and background sets based on spatial coordinates, replaces the background with a template set of pixels, and combines it with the foreground to form a modified image, allowing for semi-background replacement and artistic visual effects, while maintaining privacy and accommodating non-green backgrounds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If background replacement is performed using traditional green screen methods, then background can be replaced, but the participant must be in front of a predominantly green background which compromises privacy and limits versatility
Solution Approach 1:
The patent segments the captured image into foreground (participant) and background regions using spatial coordinates and pixel magnitude analysis. This segmentation allows selective replacement of only the background portion while preserving the participant, eliminating the need for green screen environments and thereby protecting privacy while enabling versatile background replacement.
Solution Approach 2:
The patent applies different processing treatments to different regions of the image: the foreground region (participant) is preserved with minimal modification, while the background region is replaced with template pixels. This local quality approach allows background replacement without affecting the participant, achieving both privacy protection and versatility.
2Adaptability or versatility
If complete background replacement is performed, then privacy is protected and branded backgrounds can be used, but image quality may deteriorate with artifacts and loss of natural appearance
Solution Approach 1:
The patent implements semi-background replacement rather than complete replacement, substituting only a portion of the background pixels with template pixels while retaining some original background elements. This partial action maintains image quality and natural appearance while still achieving privacy protection and branded background capabilities.
Solution Approach 2:
The patent modifies pixel magnitudes in the background region by replacing them with template pixel magnitudes, while preserving the spatial coordinates and overall image structure. This parameter change approach enables branded backgrounds while maintaining image quality through careful selection and blending of template pixels.
3Productivity
If rough segmentation based on spatial coordinates is used, then processing speed is improved for real-time application, but segmentation precision may be reduced compared to pixel magnitude-based methods
Solution Approach 1:
The patent segments the image based on spatial coordinates into foreground and background regions, which can be processed in real-time. This spatial segmentation approach provides sufficient precision for background replacement while enabling real-time processing speed required for video communication applications.
4Device complexity
If 2D cameras are used to capture participant images, then device complexity is reduced, but the ability to perform accurate background separation is limited compared to 3D or depth-sensing cameras
Solution Approach 1:
The patent processes 2D camera images by analyzing and transforming pixel magnitude parameters to separate foreground from background. This parameter transformation approach enables accurate background separation using simple 2D cameras, avoiding the need for complex 3D or depth-sensing camera systems while maintaining separation accuracy.
Data Source
AI summary
A telecommunication device includes an image capture system that captures an image of a local participant in a telecommunication session, the image comprising foreground and background images defined by plural pixels, each of the pixels having a pixel magnitude related to a sample of the image at a spatial location of the respective pixel and a background modifier that segments plural pixels of the captured image into foreground and background sets of pixels, replaces the background set of pixels with a template set of pixels to form a new background set of pixels, selected pixels in the template set of pixels having a different magnitude than a magnitude of the corresponding pixel in the background set of pixels replaced by the selected pixel, and combines the new background set of pixels with the foreground set of pixels to form modified image information for transmission to a remote endpoint. A background selector selects the template set of pixels from among multiple template sets of pixels based on an attribute of a remote endpoint or remote participant associated with the remote endpoint.