Audio Stream Processing for Distributed Meeting Transcription
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Ad-hoc meetings often lack the infrastructure for recording and transcribing conversations, as they typically do not involve pre-set conferencing tools, and existing solutions are not designed for unplanned gatherings outside formal conference settings.
Innovation Solution
A computer-implemented method that uses multiple microphones to detect and localize speech sources, employing a speech unmixing model to select relevant audio channels and send them to a meeting server for generating a speaker-attributed transcript, which can be done using distributed devices such as smartphones and ambient capture devices with integrated microphones and cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple microphones are used to capture audio from ad-hoc meetings, then the quality of speech capture is improved, but the complexity of audio processing and channel selection increases
Solution Approach 1:
The system extracts only the two most relevant audio channels from the multiple microphone inputs by localizing speech sources and selecting channels with the highest speech energy. This extraction approach maintains high speech capture quality while reducing the processing burden by focusing only on the most important channels rather than processing all channels equally.
Solution Approach 2:
The system creates a simplified representation of the multi-channel audio environment by selecting two representative channels that capture the essential speech information. This copying approach allows the system to work with a reduced set of channels that replicate the most important speech capture characteristics without the full complexity of processing all microphone inputs.
2Measurement precision
If speech source localization is performed to determine direction of arrival, then speaker attribution accuracy is improved, but the computational requirements and processing time increase
Solution Approach 1:
The system performs speech source localization only when necessary for channel selection, rather than continuously processing all audio data. By applying localization selectively to determine the direction of arrival and identify the two most relevant channels, the system achieves accurate speaker attribution while minimizing unnecessary computational overhead and processing time.
3Measurement precision
If a speech unmixing model is used to select relevant channels, then the accuracy of speaker identification is improved, but the computational complexity and resource requirements increase
Solution Approach 1:
The speech unmixing model extracts only the two most relevant audio channels from the multiple microphone inputs by analyzing speech energy distribution and localization data. This extraction approach maintains high speaker identification accuracy by selecting channels with the most relevant speech information while reducing computational complexity by eliminating the need to process all channels through the full unmixing algorithm.
4Ease of operation
If distributed devices like smartphones are used for meeting recording, then accessibility and ease of deployment are improved, but the reliability and consistency of audio capture across different devices may worsen
Solution Approach 1:
The system adapts to different distributed devices by dynamically adjusting its audio processing parameters based on the specific characteristics of each device's microphone array. By changing parameters such as gain normalization, channel selection thresholds, and localization sensitivity, the system maintains reliable and consistent speaker attribution across diverse device types while preserving the ease of deployment advantage.
Data Source
AI summary
A computer implemented method includes receiving multiple channels of audio from three or more microphones detecting speech from a meeting of multiple users, localizing speech sources to determine an approximate direction of arrival of speech from a user, using a speech unmixing model to select two channels corresponding to a primary and a secondary microphone, and sending the two selected channels to a meeting server for generation of a speaker attributed meeting transcript.