Encoder Frame Reduction in End-to-End ASR for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
End-to-end automatic speech recognition (ASR) models with large numbers of parameters and full-sequence processing suffer from high computational latency, which is unacceptable for low-latency speech recognition tasks such as short voice search queries.
Innovation Solution
An end-to-end model with a stack of multi-head attention blocks that applies an encoder reduction ratio to reduce the number of encoder output frames, using a combined pooling and multi-head self-attention layer to decrease the effective length of the output, coupled with a decoder to generate a sequence of output tokens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large ASR models with billions of parameters and full-sequence processing are used, then word error rates improve significantly, but computational latency increases substantially
Solution Approach 1:
The encoder output sequence is segmented into multiple groups, and only selected groups are processed by the decoder. This segmentation approach reduces the number of encoder output frames that need to be decoded, thereby reducing computational latency while maintaining recognition accuracy through strategic selection of representative frames
Solution Approach 2:
The patent extracts and processes only a subset of encoder output frames rather than all frames. By taking out selected representative frames for decoder processing, the system reduces computational load and latency while preserving essential information needed for accurate speech recognition
2Loss of time
If the number of encoder output frames is reduced, then computational latency decreases, but information completeness may be compromised
Solution Approach 1:
The encoder processes the complete input sequence and generates all encoder output frames in advance. This preliminary encoding ensures that all speech information is captured and stored in the encoder outputs before the decoder selectively processes a subset of these frames, preventing information loss
Solution Approach 2:
The encoder acts as an intermediary that captures and preserves complete speech information in its output frames. Even though the decoder only processes a subset of these frames, the encoder has already retained all necessary information, serving as a buffer that prevents information loss while enabling latency reduction
Data Source
AI summary
A method includes receiving a sequence of encoder input frames as input to an end-to-end model. The method also includes generating a sequence of encoder output frames based on the sequence of encoder input frames using an encoder of the end-to-end model. The encoder includes a stack of multi-head attention blocks arranged to apply an encoder reduction ratio on the sequence of encoder input frames. A number of encoder output frames generated as output from the encoder is reduced from a number of the encoder input frames received as input to the encoder by a factor proportional to the encoder reduction ratio applied by the stack of multi-head attention blocks. The method also includes decoding the sequence of encoder output frames into a sequence of output tokens using a decoder of the end-to-end model.