Speaker Separation Model Training for Long Conversations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional speaker separation technologies using Bayesian Information Criterion (BIC) struggle with accurately separating multiple speakers in lengthy conversations due to limitations in modeling speaker data distribution, leading to poor performance.
Innovation Solution
A speaker separation model training method that involves acquiring and processing audio data, extracting features, and training a neural network model using similarity functions and loss functions to enhance feature extraction and improve accuracy in two-speaker separation, especially for long conversations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional BIC with Single Gaussian model is used for speaker separation, then the method is simple to implement, but the separation accuracy deteriorates in lengthy conversations
Solution Approach 1:
The patent changes the mathematical model parameters from Single Gaussian to mixture models (Gaussian Mixture Model or Variational Autoencoder) to better capture the complex distribution of speaker data in lengthy conversations, thereby improving separation accuracy while maintaining reasonable implementation complexity
Solution Approach 2:
The patent performs pre-training of the separation model on large-scale data before fine-tuning on specific tasks. This preliminary action enables the model to learn robust speaker representation and distribution characteristics, improving its performance in lengthy conversations
2Measurement precision
If the network depth is increased to improve feature extraction capability, then the model can capture more complex patterns, but performance degradation increases
Solution Approach 1:
The patent implements pre-training on large-scale data before fine-tuning on specific separation tasks. This preliminary action allows the deep network to learn robust features without performance degradation, as the pre-trained weights provide a stable starting point that prevents overfitting and instability during task-specific training
Solution Approach 2:
The patent uses adversarial training where the separation model and verification model provide feedback to each other. The verification model detects whether separated speech comes from the same speaker, providing feedback that stabilizes the training of deep networks and prevents performance degradation
Data Source
AI summary
A speaker acquires audio data and performing processing. Audio features of the audio data are extracted. The audio features are inputted into a preset neural network model for training to obtain vector features. A first similarity value between a first vector feature and a second vector feature of a first speaker and a second similarity value between the first vector feature and a third vector feature of a second speaker are calculated. A loss function value is calculated and when is less than or equal to a preset loss function threshold, a training process of the speaker separation model is ended and parameters are updated. A two-speaker separation method, a terminal, and a storage medium are disclosed. Feature extraction capabilities of the model are enhanced, and accuracy of separation between speakers is improved, especially in long meetings and conversations.