Audio Separation Network Training with Pseudo Labels for Generalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based speech separation networks face poor generalization due to the lack of large-scale, diverse labeled training data, leading to overfitting and limited performance in mismatched scenarios, despite the high cost and impracticality of collecting such data.
Innovation Solution
A method for training an audio separation network using a first sample set with pseudo labels and perturbation data, enriching sample data and enhancing generalization capability through unsupervised learning by interpolating and adjusting network parameters based on loss convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large-scale, diverse labeled training data is collected to improve generalization capability, then the generalization capability is improved, but the time, manpower, and cost increase significantly
Solution Approach 1:
The patent uses unlabeled data as copies or substitutes for labeled data. By training the audio separation network on unlabeled data through self-supervised learning mechanisms, the system creates synthetic training examples that mimic the statistical properties of labeled data without requiring manual annotation, thus resolving the contradiction between data quality and collection cost
Solution Approach 2:
The audio separation network performs self-training by using its own predictions on unlabeled data to generate pseudo-labels. The system serves itself by automatically creating training signals from unlabeled data through consistency regularization and teacher-student framework, eliminating the need for external manual labeling resources
2Measurement precision
If complex networks with large number of parameters are used to improve separation performance, then the separation performance is improved, but overfitting occurs due to lack of labeled data
Solution Approach 1:
The patent changes the training paradigm from supervised learning requiring labeled data to self-supervised learning using unlabeled data. By modifying the loss function to use consistency regularization between teacher and student models on unlabeled data, the system enables complex networks to train effectively without overfitting, as the large amount of unlabeled data provides sufficient statistical constraints
Solution Approach 2:
The patent introduces a teacher model as an intermediary that generates pseudo-labels for unlabeled data. The teacher model's predictions serve as intermediate training signals that guide the student model without requiring ground truth labels, enabling complex networks to learn from unlabeled data and prevent overfitting
3Measurement precision
If speech training data is used for training, then the network performs well on matching speech, but the network fails suddenly on mismatched test speech
Solution Approach 1:
The patent trains the audio separation network on multi-purpose unlabeled data that can serve multiple speech types and scenarios. By using self-supervised learning on diverse unlabeled data rather than specialized labeled data, the network develops universal features that generalize across different speech conditions, speakers, and environments, making it versatile rather than specialized
Data Source
Figure 1~2A
Figure 2B
Figure 3
AI summary
A training method and device for an audio separation network, an audio separation method and device, and a medium. The training method comprises: obtaining a first separation sample set, the first separation sample set at least comprising two types of audio having pseudo labels; performing interpolation on the first separation sample set by using preset disturbance data so as to obtain a first sample set; separating the first sample set by using an unsupervised network so as to obtain a second separation sample set; determining the losses of second separation samples in the second separation sample set; and adjusting a network parameter of the unsupervised network by using the loss of the second separation samples of the second separation sample set, so as to enable the loss of a separation result output by the adjusted unsupervised network to satisfy a convergence condition.