Supervised Speaker Diarization Using UIS-RNN
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speaker diarization systems operate unsupervised, failing to leverage labeled training data for improved accuracy and real-time performance, leading to inaccurate speech recognition in multi-speaker environments.
Innovation Solution
A fully supervised probabilistic generative model using an unbounded interleaved-state recurrent neural network (UIS-RNN) that incorporates time-stamped speaker labels for training, enabling online diarization with improved speaker label prediction accuracy by learning from labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If unsupervised speaker diarization systems are used, then device complexity is reduced, but speaker label prediction accuracy deteriorates
Solution Approach 1:
The patent implements feedback by using predicted speaker labels from the probabilistic generative model to re-segment the audio signal and extract refined speaker embeddings. These refined embeddings are fed back into the model to generate improved speaker labels, creating an iterative feedback loop that progressively enhances prediction accuracy while maintaining reasonable system complexity
Solution Approach 2:
The patent applies preliminary action by performing an initial pass of speaker diarization to obtain preliminary speaker labels and segmentations. These preliminary results serve as the foundation for subsequent refinement steps, allowing the system to build upon initial predictions rather than starting from scratch, thereby improving overall accuracy
2Measurement precision
If fully supervised probabilistic generative models are used, then speaker label prediction accuracy is improved, but computational time increases
Solution Approach 1:
The patent applies segmentation by dividing the computational process into distinct phases: initial speaker embedding extraction, preliminary speaker label prediction, audio re-segmentation based on predicted labels, refined embedding extraction, and final speaker label refinement. This segmentation allows the computationally intensive fully supervised model to be applied selectively rather than continuously, reducing overall computational time while maintaining accuracy improvements
Solution Approach 2:
The patent implements partial action by applying the computationally expensive probabilistic generative model only to segments where speaker changes are detected or where refinement is most beneficial. Rather than processing the entire audio signal uniformly through the complex model, the system applies the model selectively to portions of the signal that require improved accuracy, thereby balancing computational cost with performance gains
3Productivity
If speaker diarization is performed in real-time, then productivity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent applies preliminary action by performing quick initial speaker embedding extraction and preliminary speaker label prediction using lightweight models. These preliminary results are generated rapidly to enable real-time responsiveness, while more computationally intensive refinement steps are applied selectively or with relaxed timing constraints to maintain overall accuracy without compromising real-time performance
Data Source
AI summary
A method includes receiving an utterance of speech and segmenting the utterance of speech into a plurality of segments. For each segment of the utterance of speech, the method also includes extracting a speaker=discriminative embedding from the segment and predicting a probability distribution over possible speakers for the segment using a probabilistic generative model configured to receive the extracted speaker-discriminative embedding as a feature input. The probabilistic generative model trained on a corpus of training speech utterances each segmented into a plurality of training segments. Each training segment including a corresponding speaker-discriminative embedding and a corresponding speaker label. The method also includes assigning a speaker label to each segment of the utterance of speech based on the probability distribution over possible speakers for the corresponding segment.