Streaming ASR Emission Latency Reduction via Sequence-Level Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current streaming ASR models face a challenge in reducing emission latency while maintaining accuracy, as approaches that manipulate per-token or per-frame probabilities to reduce latency often suffer from significant accuracy regressions.
Innovation Solution
A method is introduced to train a streaming speech recognition model that learns an alignment probability between acoustic frames and vocabulary tokens, including a blank token, using a tuning parameter to maximize the probability of emitting label tokens at a sequence level, thereby reducing emission latency without penalizing blank token emissions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If per-token or per-frame probabilities are manipulated to reduce emission latency, then latency is reduced, but accuracy suffers severe regression
Solution Approach 1:
The patent changes the parameter level from per-token or per-frame probability manipulation to sequence-level alignment probability manipulation. By operating at the sequence level with a tuning parameter applied to the overall alignment probability, the system reduces latency without the severe accuracy regression that occurs with granular probability manipulation. This parameter change in the level of operation resolves the contradiction between latency reduction and accuracy maintenance.
2Measurement precision
If sequence-level alignment probability is used with forward and backward probabilities, then accuracy is preserved while reducing latency, but computational complexity increases
Solution Approach 1:
The patent segments the alignment probability calculation into forward probabilities and backward probabilities that operate independently at different temporal directions. This segmentation allows the system to compute sequence-level alignment probabilities without requiring exhaustive search through all possible alignments, thereby reducing computational complexity while maintaining accuracy and enabling latency reduction through the tuning parameter.
Data Source
AI summary
A computer-implemented method of training a streaming speech recognition model that includes receiving, as input to the streaming speech recognition model, a sequence of acoustic frames. The streaming speech recognition model is configured to learn an alignment probability between the sequence of acoustic frames and an output sequence of vocabulary tokens. The vocabulary tokens include a plurality of label tokens and a blank token. At each output step, the method includes determining a first probability of emitting one of the label tokens and determining a second probability of emitting the blank token. The method also includes generating the alignment probability at a sequence level based on the first probability and the second probability. The method also includes applying a tuning parameter to the alignment probability at the sequence level to maximize the first probability of emitting one of the label tokens.