Dual-Sensory Speech Detection for Video Conference Mic Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video conferencing systems face challenges with excessive background noise and reduced audio quality due to background noise suppression techniques, which compromise privacy and security, and existing echo cancellation methods fail to effectively manage microphone states during multi-participant calls.
Innovation Solution
A dual sensory input speech detection system that combines audio and visual cues, using lip movement and speech analysis to dynamically control microphone attenuation, ensuring accurate prediction of participant speech and reducing background noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If background noise suppression techniques are applied to clean the speech signal, then speech quality is improved, but audio fidelity is reduced and privacy/security is compromised
Solution Approach 1:
The patent extracts and removes only the harmful background noise components while preserving the speech signal and audio fidelity. By using dual sensory input (audio and video) to identify speech segments, the system can selectively attenuate noise without applying broad-based suppression that degrades audio quality.
Solution Approach 2:
The patent introduces an intermediary processing layer that combines audio and video signals to create a more accurate representation of speech activity. This intermediary approach allows for more precise noise suppression by cross-referencing visual lip movement with audio signals, enabling selective noise reduction that preserves audio fidelity.
2Object-generated harmful factors
If echo cancellation techniques are employed to remove double talk echo effects, then echo noise is reduced, but microphone state control during multi-participant calls becomes problematic
Solution Approach 1:
The patent implements dynamic microphone state control that adapts to changing speech conditions in real-time. By continuously monitoring both audio and video inputs, the system can dynamically adjust microphone attenuation levels for each participant, enabling effective echo cancellation while maintaining ease of operation through automated state management.
Solution Approach 2:
The patent employs feedback mechanisms where the system continuously monitors audio and video signals to detect speech activity and adjusts microphone states accordingly. This feedback loop enables the system to automatically manage echo cancellation and microphone states without requiring manual intervention, resolving the contradiction between echo reduction and operational ease.
3Measurement precision
If dual sensory input speech detection is used to accurately predict participant speech, then speech detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent merges audio and video processing streams into a unified dual sensory input system. By combining these two data sources and processing them through a coordinated architecture, the system achieves higher speech detection accuracy. The merging approach is implemented through integrated processing that shares computational resources and coordinates signal processing tasks.
Data Source
Figure 1A~1D
Figure 2~4
Figure 5
AI summary
A dual sensory input speech detection method includes receiving, at a first time, a first video image input of a conference participant of the video conference and a first audio input of the conference participant; communicating the first video image input to the video conference; identifying the first video image input as a first facial image of the conference participant; determining, based on the first facial image, the first video image input indicates the conference participant is in a speaking state; identifying the first audio input as a first speech sound; determining, while in the speaking state, the first speech sound originates from the conference participant; and communicating the first audio input to an audio output for the video conference.