Hybrid Language Model for Open-Vocabulary Speech Recognition

The patent segments the language model into two distinct components: a character-level LM for modeling local linguistic patterns and character sequences, and a word-level LM for modeling global linguistic constraints and word-level patterns. This segmentation allows each model to operate at its optimal level, with the character-level model handling open vocabulary sequences and the word-level model providing accuracy for known words, thereby resolving the contradiction between open vocabulary capability and recognition accuracy.

The patent merges character-level and word-level language models into a unified end-to-end speech recognition system. The character-level LM is trained on relatively small datasets to capture local patterns, while the word-level LM is integrated to provide global linguistic constraints. This combination enables the system to achieve word-level prediction accuracy without requiring the enormous amounts of transcribed speech data that would be needed for pure word-level models, as the character-level component handles sequences with limited training data.

The patent combines the acoustic model, character-level language model, and word-level language model into a single unified end-to-end neural network architecture. This merging allows all components to be jointly optimized through backpropagation, enabling the system to escape local optima that plague separately optimized factorized architectures. The unified architecture shares representations and gradients across all modules, ensuring consistent optimization toward a global optimum while maintaining the functional benefits of having distinct acoustic and linguistic components.