Audio-Video Filtering with Visual Event Detection for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices struggle to efficiently filter sudden and unpredictable spurious noise, such as sneezing, coughing, or animal sounds, during video conferences, leading to latency and misalignment between audio and video streams.
Innovation Solution
Implementing visual event detector circuitry to analyze video frames for cues of impending spurious noise, triggering audio and video filters to mitigate these noises, and resuming normal output when the event ends, thus reducing latency and maintaining stream synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If audio filtering is applied reactively after noise detection, then noise can be filtered, but latency occurs and audio-video synchronization is lost
Solution Approach 1:
The system performs preliminary actions by detecting visual cues (hand-to-face motion, facial expressions) that precede spurious noise events. By identifying these visual indicators before the actual noise occurs, the system can proactively mute the audio stream in advance, preventing latency and maintaining synchronization between audio and video streams during video conferences.
2Device complexity
If traditional audio-only noise detection is used, then implementation is simple, but sudden spurious noise cannot be detected in time
Solution Approach 1:
The system merges visual event detection with audio filtering by integrating a visual event detector that analyzes video frames for cues indicating impending spurious noise. This multi-sensory approach combines computer vision technology with traditional audio processing, enabling the system to detect visual precursors (such as hand-to-face motions or facial expressions) before the actual noise occurs, thereby significantly improving detection accuracy and timing.
Data Source
AI summary
Apparatus, systems, and methods for audio and video filtering for electronic user devices are disclosed. An example apparatus includes at least one memory, instructions in the apparatus, and processor circuitry to execute instructions to detect a visual event based on image data, the visual event representative of an activity associated with a likelihood of noise, the image data associated with a video stream output by a camera associated with a user device, and in response to the detection of the visual event, apply an audio filter to a portion of an audio stream corresponding to the image data in the video stream.