Cascaded Encoder ASR Model for Streaming and Non-Streaming Recognition

The model is segmented into two distinct encoder paths: a streaming encoder for real-time applications and a non-streaming encoder for offline applications. Each encoder is optimized for its specific application type, allowing the system to maintain high accuracy for both streaming and non-streaming tasks while using a single unified model architecture.

The streaming encoder is designed with local quality optimization, using causal convolution and attention mechanisms that are specifically tailored for real-time processing. This allows the encoder to maximize accuracy within the constraints of limited context availability, improving rare word recognition performance in streaming mode without requiring future context.

The non-streaming encoder employs dynamic bidirectional processing that can adaptively utilize future context when available. The model dynamically adjusts its processing based on whether complete audio input is available, allowing it to achieve high accuracy on rare words and long-tail proper nouns by leveraging contextual information from the entire audio sequence when operating in non-streaming mode.

The model is trained with multi-functionality in mind, using a unified architecture that can handle both common and rare words effectively. The training process incorporates techniques that enable the model to generalize better to long-tail proper nouns and rare words, allowing it to maintain high performance on these challenging cases even when trained on limited audio-text pairs, thus improving both training efficiency and rare word recognition accuracy.