Video Group Framing Using Face Motion Upper Body Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In videoconferencing, face detection is challenging when faces are small, and participant movement causes instability in group framing, leading to a negative user experience due to high false detection rates and ineffective motion detection.
Innovation Solution
A method that combines face detection with motion detection and upper body detection using deep learning techniques, employing a strict motion detection mode to filter out fake faces and a normal mode for postprocessing, with upper body detection improving accuracy and reducing processing load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If face detection is performed on small faces in video data, then participant identification is attempted, but detection accuracy deteriorates when face size is below 20 pixels
Solution Approach 1:
The patent combines face detection with motion detection and upper body detection into a unified participant identification system. Instead of relying solely on face detection, the system merges multiple detection modalities to identify participants, thereby overcoming the limitation of small face sizes and improving overall detection accuracy.
Solution Approach 2:
The patent transitions from two-dimensional face detection to incorporating temporal dimension through motion detection and spatial context through upper body detection. By adding these additional dimensions of detection, the system can identify participants even when their faces are too small for reliable detection alone.
2Reliability
If only face detection is used for group framing, then participant identification is attempted, but reliability deteriorates when participants move or do not face the camera
Solution Approach 1:
The patent implements dynamic detection by performing motion detection to track participant movement over time. The system adapts to participant movement by continuously updating their positions and maintaining identification even when participants move or change orientation, thereby improving reliability in dynamic videoconference environments.
Solution Approach 2:
The patent creates a multi-functional detection system that can identify participants through multiple means: face detection when faces are visible, motion detection when participants move, and upper body detection as an alternative when faces are not detectable. This universal approach ensures reliable participant identification across various scenarios.
3Measurement precision
If multiple detection types are performed, then detection accuracy is improved, but processing load and device complexity increase
Solution Approach 1:
The patent segments the detection process into distinct stages: face detection first, then motion detection for saved faces, and finally upper body detection as needed. This segmentation allows the system to perform multiple detection types without processing the entire video stream with all detection algorithms simultaneously, thereby managing complexity while maintaining accuracy.
Solution Approach 2:
The patent applies partial action by performing full face detection on all video data, but only performing motion detection and upper body detection on saved detected faces rather than the entire video stream. This selective application of detection algorithms reduces processing load while maintaining high identification accuracy for relevant participants.
Data Source
AI summary
A videoconferencing endpoint is described that uses a combination of face detection, motion detection, and upper body detection for selecting participants of a videoconference for group framing. Motion detection is used to remove fake faces as well as to detect motion in regions around detected faces during postprocessing. Upper body detection is used in conjunction with the motion detection in postprocessing to allow saving faces that have been initially detected by face detection for group framing even if the participant has turned away from the camera, allowing the endpoint to keep tracking the participants region better than would be possible based only on an unstable result coming from face detection.