User-Controlled XR Masking for Video Background Occlusion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video conferencing systems lack user control over which entities or objects in the physical background are occluded by virtual backgrounds, leading to compliance and privacy concerns, as well as a negative user experience due to blurry or obscured foreground objects.

Innovation Solution

A system and method that allow users to control the inclusion or exclusion of entities in the video feed using masks to determine where an extended reality (XR) portion should be inserted, ensuring that specified entities are not occluded, and dynamically adapting to changing environmental conditions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Object-affected harmful factors

If a virtual background is applied to replace the entire background, then the user's privacy is improved and the professional appearance is enhanced, but other entities in the physical background are occluded and blurred

Engineering Contradiction:

Improveprivacy protectionVSAvoidvisibility of other entities

Core Design Contradiction:

Object-affected harmful factorsVSLoss of information

Solution Approach 1:

The patent divides the background into multiple regions using masks, allowing selective application of virtual background to specific areas while preserving visibility of other entities in different regions. This segmentation enables users to control which portions of the background are replaced and which remain visible.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different background treatments to different spatial regions - virtual background replacement in some areas and original background preservation in other areas. This local differentiation allows simultaneous achievement of privacy protection and entity visibility based on user-controlled mask definitions.

Inventive Principle:

Principle #3Local quality

2Object-affected harmful factors

If a virtual background is applied to replace the entire background, then the professional appearance is improved, but the user loses control over which objects remain visible

Engineering Contradiction:

Improveprofessional appearanceVSAvoiduser control over visibility

Core Design Contradiction:

Object-affected harmful factorsVSEase of operation

Solution Approach 1:

The patent introduces dynamic mask editing capabilities that allow users to interactively adjust which entities are included or excluded from the virtual background replacement. Users can add, remove, or modify mask regions based on real-time needs, providing granular control over visibility while maintaining professional appearance.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system provides visual feedback by displaying the current mask configuration and previewing the resulting video feed before finalizing the virtual background application. This feedback mechanism enables users to verify and adjust their control settings to achieve desired visibility outcomes.

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If a virtual background is applied, then the background replacement is achieved, but the user experience is degraded due to blurry or obscured foreground objects

Engineering Contradiction:

Improvebackground replacement capabilityVSAvoiduser experience

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent segments the video feed into foreground entities and background regions, applying virtual background replacement only to appropriate background areas while preserving sharpness and clarity of foreground objects. This segmentation prevents blurring of important entities while maintaining background replacement functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent uses masks as intermediary structures that mediate between the virtual background replacement process and the visibility of entities. Masks act as selective filters that guide the background replacement process to avoid obscuring important foreground objects, thereby maintaining user experience quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12010157B2Systems and methods for enabling user-controlled extended reality

Publication Date: 2024.06.11 ADEIA GUIDES INC

AI summary

Systems and methods are described for enabling user-controlled extended reality (XR). The provided systems and methods may determine an XR portion should be included in a video feed that comprises a depiction of a first user, and the XR portion may be configured to occlude the entity in the video feed. An entity may be detected in the video feed, and user input may be received specifying that a detected entity in the video feed should not be occluded by the XR portion. Based on the user input, the video feed may be modified to exclude the XR portion from a portion of the video feed depicting the detected entity, wherein the portion is included at a different portion of the video feed, and the depiction of the first user is included in the video feed, and transmit the modified video feed to a second user device.