Speech Recognition Model Proper Noun Accuracy via Two-Pass MWER Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face challenges in accurately transcribing rare or uncommon words, such as proper nouns, due to their ambiguity and rarity in training data, leading to increased latency and error rates, especially in real-time mobile applications where low latency and high accuracy are crucial.
Innovation Solution
A two-pass speech recognition architecture is employed, combining a recurrent neural network transducer (RNN-T) for streaming capabilities with a listen-attend-spell (LAS) decoder, using a shared encoder and custom minimum word error rate (MWER) loss criteria to emphasize proper noun recognition by penalizing incorrect hypotheses and incorporating phonetically similar alternatives during training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional speech recognition models are used, then processing speed is maintained, but word error rate increases for proper nouns
Solution Approach 1:
The speech recognition model is divided into two distinct passes: a first pass using RNN-T for streaming transcription with low latency, and a second pass using LAS for improved accuracy on proper nouns. This segmentation allows each component to optimize for its specific function, resolving the contradiction between speed and precision.
Solution Approach 2:
The first pass RNN-T decoder performs preliminary transcription of the speech input, providing an initial hypothesis that is then refined by the second pass LAS decoder. This preliminary action enables the system to maintain low latency while preparing for subsequent accuracy improvement.
2Measurement precision
If more training data for rare words is collected, then recognition accuracy improves, but system complexity increases
Solution Approach 1:
The system changes the parameter of the loss function from standard cross-entropy to minimum word error rate (MWER) loss with penalties for proper nouns. This parameter change allows the model to focus on improving recognition accuracy for rare words without requiring additional training data or increasing system complexity.
Solution Approach 2:
The LAS decoder acts as an intermediary component that specifically handles proper noun recognition by applying penalty-based MWER loss. This intermediary approach isolates the complexity of handling rare words to a dedicated component rather than requiring the entire system to become more complex.
3Productivity
If streaming speech recognition is implemented, then real-time performance is achieved, but accuracy for uncommon words deteriorates
Solution Approach 1:
The system segments the speech recognition task into two passes: the first pass uses RNN-T for real-time streaming transcription, while the second pass uses LAS with MWER loss for improved accuracy on uncommon words. This segmentation resolves the contradiction by allowing streaming performance and accuracy to be optimized separately.
Solution Approach 2:
The two-pass architecture maintains continuous useful action by having the first pass provide immediate streaming results while the second pass continuously refines the transcription for proper nouns. This continuity ensures both real-time performance and improved accuracy are achieved without interruption.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A method (400) for training a speech recognition model (200) with a minimum word error rate loss function includes receiving a training example (302) including a proper noun and generating a plurality of hypotheses (222) corresponding to the training example. Each hypothesis of the plurality of hypotheses represents the proper noun and includes a corresponding probability that indicates a likelihood that the hypothesis represents the proper noun. The method also includes determining that the corresponding probability associated with one of the plurality of hypotheses satisfies a penalty criteria. The penalty criteria indicating that the corresponding probability satisfies a probability threshold, and the associated hypothesis incorrectly represents the proper noun. The method also includes applying a penalty (332) to the minimum word error rate loss function.