User-Controlled XR Background Masking for Video Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video conferencing systems lack user control over which entities or objects are occluded by virtual backgrounds, leading to compliance and privacy concerns, as well as a poor user experience due to blurred or obscured foreground objects.
Innovation Solution
A system and method that allow users to control the inclusion or exclusion of entities in a video feed by generating masks for detected entities and combining them with the physical background, enabling the user to specify which parts of the environment should not be occluded by the XR portion, and dynamically adapting to changing conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual background is applied to replace the entire background, then the user's privacy is improved and professional appearance is enhanced, but other entities (people, objects) in the background become occluded and invisible to other participants
Solution Approach 1:
The patent segments the background into multiple regions: fully occluded regions (for privacy-sensitive areas), partially transparent regions (for selective visibility), and fully visible regions (for important entities). This allows different portions of the background to have different levels of occlusion based on their importance and privacy requirements.
Solution Approach 2:
The patent applies different occlusion qualities to different regions of the background. Instead of uniform occlusion, the system dynamically adjusts the transparency and visibility characteristics of each background region based on detected entities and their importance, allowing critical entities to remain visible while maintaining privacy in other areas.
2Ease of manufacture
If a static virtual background is set before the session, then the implementation is simple and straightforward, but the system cannot adapt to changing dynamic environmental conditions such as moving people or objects
Solution Approach 1:
The patent transforms the static virtual background into a dynamic system that automatically detects changes in the environment (such as new people or objects entering the frame) and adjusts the occlusion strategy in real-time. The system continuously monitors the background and re-segments regions as needed to maintain both privacy and visibility of important entities.
Solution Approach 2:
The system automatically detects entities in the background and self-adjusts the occlusion parameters without requiring manual user intervention. The algorithm autonomously determines which regions should be occluded and which should remain visible, adapting to environmental changes independently.
3Reliability
If the virtual background occludes all background entities, then the user's physical environment is fully protected, but compliance and security concerns arise because other participants cannot verify the authenticity of the video feed
Solution Approach 1:
The patent introduces an intermediary approach where the virtual background acts as a selective filter rather than a complete barrier. By allowing certain background entities to remain visible through controlled transparency, the system maintains a balance between privacy protection and authenticity verification, reducing compliance and security risks.
4Ease of manufacture
If the system uses chroma-key or green screen background removal methods, then the virtual background can be applied effectively, but foreground objects outside the main participant border become rendered blurry or obscured
Solution Approach 1:
The patent segments the video feed into multiple layers: the main participant, foreground objects, and background regions. By separately processing and masking these layers, the system preserves the clarity of foreground objects while applying the virtual background, avoiding the blurring effect caused by traditional chroma-key methods.
Data Source
AI summary
Systems and methods are described for enabling user-controlled extended reality (XR). The systems and methods may access one or more frames, depicting an environment surrounding a first user, of a video feed captured by a first user's device, and classify each portion of the frames as a physical background of the environment, as depicting the first user, or as depicting an additional entity. First and second masks may be generated for the depictions of the first and second users, respectively, and such masks may respectively define boundaries of the first user's depiction and the second user's depiction at which the XR portion should not be inserted. The frames may be modified by combining portions of the frames belonging to the physical background with the first mask and the second mask, and the video feed comprising the modified frames may be encoded and transmitted to the second user device.