Multi-Speaker Transcript Generation Using Channel Change Symbols
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently generating coherent transcripts from multi-speaker audio streams, particularly when speech overlaps, and are computationally expensive, making real-time processing difficult.
Innovation Solution
A computerized method using a trained model, such as a Recurrent Neural Network Transducer (RNN-T), generates transcripts by inserting channel change (CC) symbols to separate overlapping speech, allowing for real-time processing and accurate separation of speakers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If post-conversation analysis is used to generate transcripts, then accuracy can be improved, but real-time processing capability is lost
Solution Approach 1:
The system performs preliminary processing by generating frame embeddings and inserting CC symbols during the transcription process itself, rather than performing separate post-conversation analysis. This allows the model to prepare and structure the transcript data in real-time as speech is being delivered, maintaining both accuracy and timeliness.
Solution Approach 2:
The patent introduces CC (channel change) symbols as an intermediary mechanism that enables the transcript generation model to handle overlapping speech in real-time. These symbols act as markers that allow the system to process multi-speaker audio streams without requiring computationally expensive post-processing, thus achieving real-time accuracy.
2Measurement precision
If complex models are used to separate overlapping speech, then separation accuracy is improved, but computational cost increases
Solution Approach 1:
The patent segments the speech separation task by inserting CC symbols at points where speakers change or overlap occurs. This segmentation allows the model to handle overlapping speech in discrete, manageable units rather than attempting to process the entire audio stream as a single complex task, reducing computational cost while maintaining accuracy.
Solution Approach 2:
The system changes the parameter representation by transforming audio data into frame embeddings and then into text with CC symbols. This parameter transformation allows the model to work with a more efficient representation that requires less computational power while still capturing the essential information needed for accurate speech separation.
Data Source
AI summary
The disclosure herein describes using a transcript generation model for generating a transcript from a multi-speaker audio stream. Audio data including overlapping speech of a plurality of speakers is obtained and a set of frame embeddings are generated from audio data frames of obtained audio data using an audio data encoder. A set of words and channel change (CC) symbols are generated from the set of frame embeddings using a transcript generation model. The CC symbols are included between pairs of adjacent words that are spoken by different people at the same time. The set of words and CC symbols are transformed into a plurality of transcript lines, wherein words of the set of words are sorted into transcript lines based on CC symbols, and a multi-speaker transcript is generated based on the plurality of transcript lines. The inclusion of CC symbols by the model enables efficient, accurate multi-speaker transcription.