Multistream Speech Recognition for Real-Time Speaker Attribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual conference systems lack real-time transcription and accurate attribution of speech to individual speakers, making it difficult for participants to follow conversations and presentations effectively.
Innovation Solution
Implementing multistream automatic speech recognition systems that transcribe multiple audio streams in real-time, allowing for precise attribution of speech to individual speakers during virtual conferences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional single-stream transcription is used, then system complexity is low, but speaker attribution accuracy deteriorates
Solution Approach 1:
The patent segments the audio input into multiple separate streams, each corresponding to a different speaker or audio source. By processing each stream independently through speech-to-text models, the system achieves accurate speaker attribution while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent introduces intermediary components including audio separation models, speech-to-text models, and text-to-ATC models that act as mediators between the raw audio input and the final transcript. These intermediaries enable accurate speaker attribution by processing and transforming audio data through multiple specialized stages.
2Ease of operation
If real-time transcription of multiple audio streams is implemented, then participant engagement improves, but processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing audio streams through separation models before transcription, and uses caching mechanisms to store previously processed transcripts. This allows real-time transcription to proceed more efficiently by reducing redundant processing and enabling faster retrieval of existing transcriptions.
Solution Approach 2:
The patent maintains continuous processing of audio streams through optimized pipeline architecture, where audio separation, speech-to-text conversion, and ATC generation occur in continuous flow. This ensures real-time transcription without interruptions, maintaining participant engagement while minimizing processing delays.
Data Source
AI summary
An example method includes hosting, by a conference provider, a virtual conference between a plurality of client devices exchanging audio streams; receiving, during the virtual conference, a first plurality of audio segments of a first audio stream from a first client device of the plurality of client devices; receiving, during the virtual conference, a second plurality of audio segments of a second audio stream from a second client device of the plurality of client devices; transcribing, by a transcription process, the first plurality of audio segments to create a first transcription; transcribing, by the transcription process, the second plurality of audio segments to create a second transcription; providing, during the virtual conference, the first transcription and the second transcription to the first and second client devices.