Camera Control Using Participant Gaze and Priority
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conference systems are limited in automatically focusing on objects of interest to remote participants, as they primarily rely on active speaker detection and do not account for participant orientation, leading to an enhanced conference experience.
Innovation Solution
The system utilizes participant orientation information, including gaze and pose, to automatically control the camera to focus on the highest priority object or participant that participants are looking at, even if no active speaker is detected, by determining the priority and orientation of participants and objects in the conference room.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If active speaker detection is used to control camera focus, then the camera can automatically track speakers, but it cannot focus on objects of interest like whiteboards when no one is speaking
Solution Approach 1:
The camera control system is enhanced to perform multiple functions: it can track active speakers through audio detection and simultaneously detect participant gaze direction through computer vision. This multi-functional approach allows the camera to automatically focus on either speakers or objects of interest like whiteboards depending on the situation, resolving the limitation of speaker-only detection
2Measurement precision
If the camera focuses only on active speakers, then speaker tracking is improved, but participant orientation and object of interest information is lost
Solution Approach 1:
The camera control functionality is divided into separate independent modules: one module detects active speakers through audio analysis, another module detects participant gaze direction through computer vision analysis of facial features and head pose, and a third module integrates these inputs to determine final camera focus. This segmentation allows each module to optimize its specific detection task while preserving all information for integrated decision-making
3Ease of operation
If manual camera control is used to capture objects of interest, then flexibility is improved, but automation and ease of operation deteriorate
Solution Approach 1:
The camera control system operates autonomously by automatically detecting participant gaze direction using computer vision algorithms that analyze facial landmarks, head pose, and eye position. The system self-determines when participants are looking at objects like whiteboards and automatically adjusts camera focus without requiring manual intervention, while still providing the option for manual override when needed
Data Source
AI summary
A video conference system operates to identity and to determine a location of things in a conference room, and to determine an orientation of video conference participants towards the identified things, and uses an assigned priority of the things that the participants are oriented towards to automatically control a camera to focus on the highest priority thing.