Video Group Framing Using Head Detection for Conference Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video systems capture whole room views in conference settings, leading to a far-end experience where users see excessive room environment and small heads, detracting from an optimal viewing experience.
Innovation Solution
Implementing a machine learning-based head detection model to identify group sizes and orientations, generating a group frame definition to crop and zoom on the group, excluding unnecessary room areas, and adjusting frames in real-time to maintain focus on the group.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If whole room view is captured, then all participants and environment are visible, but the viewing experience deteriorates due to excessive room environment and small heads
Solution Approach 1:
The patent extracts the group of people from the whole room view by detecting heads and generating bounding boxes, then crops the video stream to show only the relevant group area. This removes the excessive room environment while maintaining visibility of all participants, directly resolving the contradiction between comprehensive coverage and viewing quality.
Solution Approach 2:
The patent applies local quality by focusing the video stream on the specific local area where the group is located, rather than displaying the entire room uniformly. The framing dynamically adjusts to highlight the region of interest (the group) while excluding irrelevant background areas, improving viewing experience without losing participant visibility.
2Ease of operation
If head detection model is implemented, then group focus is achieved, but device complexity increases
Solution Approach 1:
The patent introduces a head detection model as an intermediary component that processes video frames to identify heads and generate bounding boxes. This intermediary enables automatic group detection and framing without manual intervention, achieving group focus capability while the complexity is encapsulated within the detection model rather than the main system architecture.
Solution Approach 2:
The system performs self-service by automatically detecting heads and generating appropriate framing without user input. The head detection model processes the video stream autonomously to identify the group and determine the optimal frame definition, reducing the need for manual configuration and simplifying user interaction despite the underlying complexity.
3Ease of operation
If real-time frame adjustment is performed, then group focus is maintained, but processing time increases
Solution Approach 1:
The patent implements continuous frame adjustment by processing each video frame in real-time through the head detection model and frame generation logic. The system maintains group focus continuously as participants move, ensuring the video stream always displays the current group position and size, which is essential for maintaining engagement despite the processing time required.
Solution Approach 2:
The system performs preliminary actions by pre-defining the group frame definition based on detected head positions and bounding boxes before displaying the video stream. The frame definition is calculated in advance for each frame, allowing the video processing to be optimized and reducing the time required during actual display, thus maintaining real-time performance.
Data Source
AI summary
A method may include obtaining, using a head detection model and for an image in a video stream, head detection information on the video stream, identifying a group vertical size of a group on a vertical axis using the head detection information, and identifying a group horizontal size of the group on a horizontal axis using the head detection information. Method may in addition include determining an orientation of the group using the group horizontal size and group vertical size, creating a group frame definition using the orientation, and outputting the video stream using the group frame definition.