Neural Corrector for ASR Numeric Sequence Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional ASR systems face challenges in accurately transcribing numeric sequences due to data sparsity and out-of-vocabulary issues, particularly for long numeric sequences like phone numbers and postal codes, which are not adequately represented in training data, leading to incorrect transcriptions.
Innovation Solution
An enhanced ASR system incorporating a neural corrector/denormer and a recurrent neural network transducer (RNN-T) model, which includes a decoder and a neural corrector/denormer to improve transcription accuracy by using additional training data and a small-footprint neural network suitable for on-device environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ASR systems are used for numeric sequences, then the system is simple and fast, but transcription accuracy deteriorates due to data sparsity and out-of-vocabulary issues
Solution Approach 1:
The ASR system is segmented into distinct components: an end-to-end model for initial transcription and a neural corrector/denormer for post-processing. This segmentation allows each component to specialize - the E2E model handles general speech recognition while the neural corrector specifically addresses numeric sequence accuracy, resolving the contradiction between simplicity and accuracy by distributing complexity across specialized modules.
Solution Approach 2:
A neural corrector/denormer is introduced as an intermediary component between the E2E model output and final transcription. This intermediary processes intermediate transcriptions to correct numeric sequences, acting as a bridge that improves accuracy without requiring the entire system to be redesigned. The intermediary handles the complex numeric correction task while the main E2E model remains relatively simple.
2Measurement precision
If long numeric sequences are used in training data, then transcription accuracy for such sequences improves, but data sparsity increases making sufficient training data difficult to obtain
Solution Approach 1:
The system performs preliminary processing of numeric sequences through the neural corrector/denormer, which is pre-trained on numeric sequence data. This preliminary action allows the model to learn patterns from available training data and apply them to correct long numeric sequences during inference, effectively handling cases where sufficient training examples of long numeric sequences are not available.
Solution Approach 2:
The neural corrector/denormer changes the parameter space by operating on intermediate transcriptions rather than raw audio, and by focusing specifically on numeric sequence correction. This parameter change allows the system to leverage general speech recognition outputs while applying specialized correction for numeric sequences, reducing the amount of specialized training data needed.
3Measurement precision
If RNN-T E2E models are used for on-device ASR, then word error rate and latency are improved, but memory requirements increase
Solution Approach 1:
The model architecture is segmented into an E2E model and a separate neural corrector/denormer. The E2E model can be optimized for speed and basic accuracy, while the neural corrector handles the more computationally intensive numeric sequence correction. This segmentation allows the system to achieve high WER performance without requiring the entire system to run at full computational capacity simultaneously.
Solution Approach 2:
The neural corrector/denormer is applied selectively to correct numeric sequences rather than processing entire transcriptions uniformly. This partial action approach allows the system to focus computational resources on the specific problem of numeric sequence accuracy, achieving high WER performance while minimizing overall memory and computational requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method (500) includes receiving audio data (110) for an utterance (106) containing a numeric sequence, and decoding, using a sequence-to-sequence speech recognition model (200), the audio data for the utterance to generate, as output from the sequence-to-sequence speech recognition model, an intermediate transcription (115) of the utterance. The method also includes processing, using a neural corrector/denormer (300), the intermediate transcription to generate a final transcription (120) that represents the numeric sequence of the utterance in a written domain. The neural corrector/ denormer is trained on a set of training samples, where each training sample includes a speech recognition hypothesis for a training utterance and a ground-truth transcription (424) of the training utterance. The ground-truth transcription of the training utterance is in the written domain. The method also includes providing the final transcription representing the numeric sequence of the utterance in the written domain for output.