Speech Recognition Model Training with Text Injection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based automatic speech recognition (ASR) models face challenges in generalizing to unseen data, particularly in low-resource speech domains with insufficient transcribed speech training data, and struggle with recognizing sensitive information due to the need to redact or replace such data, which affects recognition accuracy.
Innovation Solution
The method involves training a speech recognition model using text-injection with unspoken textual utterances that include redacted information, where fake random data is inserted into redacted portions of transcriptions, and an alignment model generates alignment outputs to teach the model to recognize speech in target domains and sensitive information classes without directly using the sensitive information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensitive information is redacted or replaced in training data, then ethical handling of sensitive information is maintained, but recognition accuracy of sensitive information classes deteriorates
Solution Approach 1:
The patent introduces fake random data as an intermediary element that occupies the position of redacted sensitive information in the training transcript. This mediator allows the model to learn the structural and contextual patterns of sensitive information without directly exposing actual sensitive data, thereby maintaining ethical handling while preserving recognition capability through pattern learning from the placeholder data
Solution Approach 2:
The patent applies preliminary action by pre-processing the training data to identify and redact sensitive information before model training. By performing the redaction and fake data insertion in advance, the training process can proceed without direct exposure to sensitive information, while the model still learns to recognize the positions and patterns where sensitive information would appear
2Measurement precision
If extensive transcribed speech training data is used, then model accuracy is improved, but the need for extensive human speech training data increases
Solution Approach 1:
The patent uses copying by creating synthetic training transcripts that replicate the structure and patterns of real speech data without requiring actual transcribed speech recordings. The fake random data inserted into redacted portions creates artificial but structurally accurate training examples that can be generated in unlimited quantities, replacing the need for extensive human-annotated speech datasets
3Adaptability or versatility
If deep learning-based ASR models are trained on low-resource speech domains, then adaptability to target domains is improved, but generalization to unseen data deteriorates due to insufficient training data
Solution Approach 1:
The patent applies dimensionality change by transitioning from the audio dimension to the text dimension for training. Instead of training directly on low-resource speech audio data, the model is trained on text transcripts with fake random data inserted, creating a new training dimension that provides sufficient statistical patterns for generalization while adapting to the target domain's linguistic characteristics
Data Source
AI summary
A method includes receiving training data including transcribed speech utterances spoken in a general domain, modified speech utterances in a target domain, and unspoken textual utterances corresponding to the transcriptions of the modified speech utterances in the target domain. The modified speech utterances include utterances spoken in the target domain that have been modified to obfuscate one or more classes of sensitive information recited in the utterances. The method also includes generating a corresponding alignment output for each unspoken textual utterance of the received training data using an alignment model. The method also includes training a speech recognition model on the alignment outputs generated for the corresponding to the unspoken textual utterances, the un-transcribed speech utterances, and the transcribed speech utterances to teach the speech recognition model to learn to recognize speech in the target domain and phrases within the one or more classes of sensitive information.