Speech Hint Correction for Out-of-Vocabulary ASR Words
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automatic speech recognition (ASR) systems struggle with unseen words such as names, acronyms, and initialisms due to their absence in training datasets, leading to unsatisfactory results when using speech hints.
Innovation Solution
A two-stage process involving a predictor and corrector system, where the predictor generates alternative words based on speech hints and the corrector uses a neural model to select the most accurate output word by correlating audio encodings with these alternatives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech hints are added to end-to-end ASR models, then recognition accuracy for out-of-vocabulary words improves, but the model training complexity increases and general word error rate may degrade
Solution Approach 1:
The system segments the ASR pipeline into two independent parts: a pre-trained end-to-end ASR model for general words and a separate predictor-corrector module for out-of-vocabulary words. This allows the ASR model to maintain its original training simplicity while the correction module handles OOV words through speech hints, resolving the contradiction between improved OOV recognition and maintained training complexity.
Solution Approach 2:
The predictor-corrector module acts as an intermediary between the ASR model and the final transcript. It receives ASR outputs, applies speech hints through prediction and correction operations, and produces improved transcripts without modifying the original ASR model training process, thus improving OOV recognition while preserving training simplicity.
2Measurement precision
If speech hints are integrated into ASR processing, then out-of-vocabulary word recognition improves, but recall and precision tradeoff occurs
Solution Approach 1:
The system performs preliminary actions by pre-processing speech hints into alternative word candidates before the correction stage. The predictor generates multiple alternative words based on speech hints, and the corrector then selects the most appropriate one. This preliminary preparation enables the system to maintain high precision by filtering candidates before final selection, while also improving recall by providing multiple possibilities.
Solution Approach 2:
The corrector module uses feedback from correspondence scores generated by comparing audio encodings with alternative words to refine the final transcript selection. This feedback mechanism allows the system to adjust its corrections based on actual audio evidence, improving both recall (by considering multiple alternatives) and precision (by selecting the most accurate word based on score feedback).
3Quantity of substance
If traditional ASR models are used, then training data requirements are minimized, but accuracy on unseen words deteriorates
Solution Approach 1:
The ASR model maintains universality by continuing to use general training data for learning core speech recognition patterns. The predictor-corrector module adds multi-functionality by handling both OOV words through speech hints and maintaining compatibility with standard ASR outputs. This allows the system to minimize training data requirements for the main model while improving accuracy on unseen words through the additional correction functionality.
Data Source
AI summary
A method comprises: receiving an automatic speech recognition (ASR) text transcript generated by an ASR process that encoded input audio into audio encodings and converted the audio encodings to ASR words of the ASR text transcript that correspond to the audio encodings; receiving speech hints for non-standard words, and generating alternative words for an ASR word of the ASR words based on the speech hints; correlating an audio encoding of the audio encodings that corresponds to the ASR word against the ASR word and each of the alternative words, to produce correspondence scores; selecting an output word among the ASR word and the alternative words based on the correspondence scores; and providing the output word to a corrected transcript.