ASR Model Expansion via Unsupervised Task Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches for expanding automatic speech recognition (ASR) models rely on sufficient supervised training data, which is not available for many target tasks, leading to a need for zero-shot task expansion methods.
Innovation Solution
The method involves training a primary ASR model on transcribed source task speech utterances and using unsupervised learning to determine task vectors from un-transcribed source and target task speech utterances. These task vectors are then used to adapt the primary ASR model to recognize speech associated with the target task.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional supervised learning approaches are used for ASR model expansion, then model accuracy is improved, but the requirement for sufficient transcribed training data increases complexity and limits applicability
Solution Approach 1:
The patent introduces task vectors as an intermediary representation that bridges source task knowledge and target task requirements. Instead of directly training on transcribed target task data, the system computes task vectors from untranscribed audio and uses these vectors to adapt the model, eliminating the need for transcribed target task data while maintaining accuracy
Solution Approach 2:
The patent replaces the mechanical process of requiring transcribed data with an unsupervised learning mechanism. The system uses self-supervised learning to compute task vectors from raw audio waveforms, substituting the need for manual transcription with an automated unsupervised process that extracts task-relevant features directly from audio
2Quantity of substance
If unsupervised learning is used to train on un-transcribed data, then data requirements are reduced, but model convergence and performance may deteriorate
Solution Approach 1:
The patent segments the learning process into distinct stages: first training the base ASR model on transcribed source task data, then computing task vectors from untranscribed data, and finally using these vectors for adaptation. This segmentation allows the system to leverage both supervised and unsupervised data in appropriate phases, ensuring reliability while reducing overall data requirements
Solution Approach 2:
The patent changes the parameter space from requiring transcribed text labels to using task vectors derived from audio features. By transforming the training problem into a task vector computation and matching problem, the system maintains performance consistency while operating on untranscribed data, effectively changing how the model learns from audio
Data Source
AI summary
A method includes training, using an un-supervised learning technique, an auxiliary ASR model based on a first set of un-transcribed source task speech utterances to determine a first task vector, training, using the un-supervised learning technique, the auxiliary ASR model based on a second set of un-transcribed speech utterances to determine a second task vector, and training, using the un-supervised learning technique, the auxiliary ASR model based on un-transcribed target task speech utterances to determine a target task vector. The method also includes determining a first correlation between the first and target task vectors, determining a second correlation between the second and target task vectors, and adapting parameters of a trained primary ASR model based on the first and second source task vectors and the first and second correlations to teach the primary ASR model to learn how to recognize speech associated with the target task.