Self-Supervised Speech Recognition Pseudo-Label Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating pseudo-labeled training data for automatic speech recognition (ASR) systems are inefficient, often requiring multiple stages of iterative training due to limited pseudo-label accuracy.
Innovation Solution
The proposed solution involves generating pseudo-labels by extracting intermediate outputs from an ASR model, clustering these outputs, and using cluster assignments or decoded word sequences to create pseudo-labeled training datasets, which can then be used to pre-train speech processing models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pseudo-labels are generated using traditional masking methods on speech tokens, then the method can be applied to speech signals, but the pseudo-label accuracy is limited and requires multiple stages of iterative training
Solution Approach 1:
The patent segments the speech signal processing into distinct components: extracting intermediate outputs from the ASR model, clustering these outputs to identify patterns, and generating pseudo-labels based on cluster assignments. This segmentation allows each component to be optimized independently, improving overall pseudo-label accuracy without requiring multiple full training iterations.
Solution Approach 2:
The patent performs preliminary clustering of intermediate outputs before generating pseudo-labels. By pre-organizing the intermediate outputs into clusters that represent linguistic patterns, the system establishes a foundation for more accurate pseudo-label generation in a single training stage, eliminating the need for iterative refinement.
2Productivity
If cluster assignments are used as pseudo-labels, then the pseudo-labels can be generated in a single stage, but the quality of pseudo-labels depends on clustering accuracy
Solution Approach 1:
The patent introduces clustering as an intermediary process between ASR model output and pseudo-label generation. The clustering algorithm acts as a mediator that transforms intermediate outputs into structured groupings, which then serve as the basis for cluster assignment pseudo-labels. This intermediary step enables single-stage training while maintaining pseudo-label quality through systematic pattern recognition.
3Reliability
If multiple stages of iterative training are performed to improve pseudo-label accuracy, then the ASR model performance improves, but the computational cost and training time increase significantly
Solution Approach 1:
The patent implements a self-service mechanism where the ASR model's own intermediate outputs are clustered and used to generate pseudo-labels for training. This self-supervised approach eliminates the need for external labeled data and multiple iterative training stages, achieving reliable ASR model performance with reduced computational cost by leveraging the model's internal representations.
Data Source
AI summary
Systems and methods are provided for generating a pseudo-labeled training dataset by at least one of: (1) extracting a set of intermediate outputs from an automatic speech recognition model based on applying the automatic speech recognition model to the set of unlabeled speech data, clustering the set of intermediate outputs into different clusters, and generating a first set of pseudo-labels comprising cluster assignments associated with the different clusters and which correspond to the unlabeled speech data, or (2) generating a set of decoded word sequences for the unlabeled speech data by applying the automatic speech recognition model to the set of unlabeled speech data, and generating a second set of pseudo-labels associated with the unlabeled speech data by applying the automatic speech recognition model to both (i) the set of decoded word sequences and (ii) the set of unlabeled speech data.