Virtual Meeting Background Freeze for Stable Auto-Framing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual meeting platforms fail to align facial features and scale the size of participants consistently, leading to an unrealistic visual display and discomfort due to background movement, causing meeting fatigue and queasy feelings.
Innovation Solution
Implement a method to freeze the background of a virtual meeting participant's video stream while using auto-framing to keep the participant centered and at a constant size, employing an AI model to superimpose the participant's image onto a static background.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If auto-framing is used to keep the participant centered and at constant size, then the participant's positioning is improved, but the background moves causing discomfort and fatigue
Solution Approach 1:
The video stream is segmented into the participant (foreground) and background components. The participant is extracted using auto-framing techniques while the background is separated and processed independently, allowing the participant to be repositioned without moving the background, thus resolving the contradiction between positioning flexibility and background stability
Solution Approach 2:
A virtual background or placeholder is introduced as an intermediary between the participant and the original background. The participant is composited onto this intermediary background, which remains static while the participant can be freely positioned using auto-framing, eliminating background movement discomfort
2Object-affected harmful factors
If the background is made static to reduce discomfort, then user comfort is improved, but the visual display becomes less adaptive to participant movement
Solution Approach 1:
The system dynamically switches between two modes: in portrait mode, the background is frozen for comfort; in landscape mode, the background remains dynamic and adaptive. This dynamic adaptation allows the system to optimize for different use cases, resolving the contradiction between stability and adaptability
Solution Approach 2:
Different regions of the visual display have different properties: the participant region allows free movement and repositioning, while the background region maintains static properties in portrait mode. This local differentiation enables simultaneous adaptability of the participant and stability of the background where needed
3Reliability
If the background is frozen using AI models, then visual realism is improved, but processing complexity and computational resources increase
Solution Approach 1:
The background freezing and AI model generation is performed in advance during setup or low-activity periods, rather than in real-time during the meeting. This preliminary action reduces the computational burden during actual use while maintaining visual realism, thus resolving the contradiction between reliability and complexity
Data Source
AI summary
Systems and methods for virtual meeting background freeze may include determining that a background of a video stream of a first client device of a participant of a virtual meeting is to be modified; identifying a first frame of the video stream as a candidate for the background of the video stream; and for each of one or more second frames of the video stream, generating a composite image by superimposing an image of a participant depicted in a respective second frame of the one or more second frames of the video stream on the background of the first frame using a location and a size of the image of the participant with respect to the respective second frame, and causing the composite image to be presented in a virtual meeting user interface (UI) on a second client device in place of the respective second frame.