Video Reframing Based on Participant Presence and Crop Offsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video conferencing applications often result in participants being partially or fully cropped out due to default center cropping, leading to an incomplete view of participants in a conference room.
Innovation Solution
Implementing techniques for repositioning and reframing participants by detecting their positions within the video frame and adjusting their placement to ensure they are fully visible, either at the sending endpoint or through signaling in the encoded stream, without requiring changes to existing decoding or server processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If default center cropping is applied to fit video frames in layout, then the number of pictures per page increases, but participants may be cropped out or not fully visible
Solution Approach 1:
The system performs face detection and calculates crop offsets before applying the crop, ensuring that the crop region is pre-determined to include all detected faces. This preliminary action prevents participant cropping while maximizing the number of visible participants in the layout.
Solution Approach 2:
Instead of uniform center cropping, the system applies localized cropping based on detected face positions. Each video frame is cropped differently according to the specific face locations, allowing optimal visibility of participants in different regions of the frame.
2Ease of manufacture
If center-based cropping is applied to preserve the center portion, then the layout fitting is improved, but portions of participant's body are cropped out
Solution Approach 1:
Face detection is performed before cropping to determine the appropriate crop region. The system calculates offsets based on detected face positions, ensuring that the crop preserves complete participant visibility while maintaining layout compatibility.
Solution Approach 2:
The crop region parameters (offsets and dimensions) are dynamically adjusted based on detected face positions. Instead of fixed center cropping, the system modifies crop parameters to align with actual participant locations, preserving body visibility while adapting to different scenarios.
3Loss of information
If repositioning processing is applied at sender endpoint, then participant visibility is improved, but processing complexity increases
Solution Approach 1:
The sender endpoint autonomously performs face detection and calculates crop offsets without requiring server processing or receiver endpoint computation. This self-service approach shifts the processing burden to the sender, reducing server load while maintaining participant visibility.
Solution Approach 2:
All repositioning and cropping decisions are made in advance at the sender endpoint before video transmission. The face detection and offset calculation are performed preliminarily, eliminating the need for complex real-time processing at the receiver or server端.
4Loss of information
If face detection and repositioning are implemented, then participant composition is improved, but signaling overhead may increase
Solution Approach 1:
The system extracts only the necessary crop offset information from face detection results and transmits it as compact signaling data. By taking out only the essential repositioning parameters rather than transmitting full detection results, the signaling overhead is minimized while maintaining composition quality.
Data Source
AI summary
Systems and methods are described herein for automatically reframing a video for a conference participant. Video of the participant is captured, and a first position of the participant is detected. An offset for the first position of the participant is then calculated to determine a relative distance from the center of the video frame. The captured video is modified based on the offset and is then presented in the video conference.