Director Camera Tracking Using Speech History and Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video conferencing scenarios, director cameras frequently and unnecessarily switch between speakers due to reliance on acoustic source localization, leading to disruptions and inefficiencies.
Innovation Solution
A camera tracking method and apparatus that utilizes both video and audio information to determine historical speech patterns and current speaker importance, allowing for precise control of the director camera's shooting angle and focal length to minimize unnecessary switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If acoustic source localization technology is used to automatically switch the director camera to the currently speaking participant, then the camera can automatically track speakers, but frequent and unnecessary switching of the camera lens occurs
Solution Approach 1:
The system introduces feedback mechanisms by analyzing historical speech information and current speech states to dynamically adjust camera switching decisions. The director device continuously monitors speech patterns and uses this feedback to determine whether switching is necessary, thereby reducing unnecessary camera lens switching while maintaining automatic tracking capability
Solution Approach 2:
The system performs preliminary analysis of historical speech information before making camera switching decisions. By pre-processing speech data and identifying speech patterns in advance, the system can make more informed switching decisions that avoid frequent and unnecessary camera lens changes
2Measurement precision
If the director camera switches to the currently speaking participant based on acoustic source localization, then the current speaker can be captured, but the switching causes disruptions and inefficiencies in video conferencing
Solution Approach 1:
The system performs preliminary analysis of historical speech information to establish speech patterns and importance levels before making camera switching decisions. This pre-processing allows the system to identify when switching is truly necessary, reducing unnecessary switches that disrupt video conferencing efficiency while maintaining accurate speaker identification
Solution Approach 2:
The system uses feedback from historical speech analysis to optimize camera switching timing and frequency. By continuously learning from speech patterns, the system reduces unnecessary switching operations that cause disruptions, thereby improving overall video conferencing efficiency while maintaining accurate speaker tracking
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
This application provides a camera tracking method and apparatus, and a device. The method includes: determining, by a director device, historical speech information in a preset time period based on first video information collected by a first camera in the preset time period and first audio information collected by a plurality of microphones MICs in the preset time period, where the first camera is configured to collect a local video; determining, by the director device, a current speaker based on second video information collected by the first camera at a current moment and second audio information collected by the plurality of MICs at the current moment; and controlling, by the director device, a director status of at least one director camera based on the historical speech information, the current speaker, and a speaker shot by the at least one director camera at the current moment, where the director camera is configured to send a director video stream to another conference site. This avoids frequent and unnecessary switching of the director camera.