Back-off Language Model Compression via Trie Block Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing language models are inefficient in terms of storage space and processing speed, particularly in applications like voice recognition, due to their large size and complexity, which hinders quick lookup and concurrent usage of multiple models.
Innovation Solution
The development of a compressed language model representation using a trie structure and block encoding techniques, such as Huffman coding and random access encoding, allows for efficient storage and fast lookup of n-grams, reducing the model's size while maintaining quick access and enabling concurrent usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional language models are used to ensure comprehensive n-gram coverage, then language modeling accuracy is improved, but storage space requirements increase and processing speed decreases
Solution Approach 1:
The language model is divided into multiple segments or components including n-gram tables, back-off models, and interpolation weights. This segmentation allows efficient storage by organizing data in manageable units that can be compressed and loaded selectively, resolving the contradiction between comprehensive coverage and storage space requirements.
Solution Approach 2:
The patent implements a nested structure where back-off models are embedded within the main n-gram model. When higher-order n-grams are not found, the system automatically nests into lower-order back-off models, maintaining comprehensive language modeling capability while reducing storage space by sharing common structures across different model orders.
2Measurement precision
If traditional language models are used to maintain complete model data, then lookup accuracy is improved, but processing speed and loading time decrease
Solution Approach 1:
The language model data is pre-processed and organized into optimized data structures during model construction. N-grams are pre-sorted and indexed, and back-off paths are pre-computed, enabling fast lookup during inference without compromising accuracy. This preliminary organization resolves the contradiction between complete data maintenance and processing speed.
3Adaptability or versatility
If large language models are used to ensure comprehensive language coverage, then language modeling capability is improved, but memory usage increases and concurrent model usage becomes difficult
Solution Approach 1:
The back-off model structure provides universality by serving multiple functions: it acts as a fallback mechanism for missing n-grams, enables incremental model loading, and supports different n-gram orders within a single unified structure. This multi-functionality maintains comprehensive language modeling capability while reducing overall memory usage, enabling concurrent model usage.
4Measurement precision
If high-order n-grams are used to improve language model precision, then probability estimation accuracy is improved, but model size and complexity increase
Solution Approach 1:
The patent implements dynamic model complexity adjustment through interpolation weights that can be tuned based on available memory and performance requirements. The system dynamically balances between high-order n-gram precision and back-off model simplicity, allowing probability estimation accuracy to be maintained while controlling model complexity and size through configurable parameters.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to language models stored for digital language processing. In one aspect, a method includes the actions of generating a language model, including: receiving a collection of n-grams from a corpus, each n-gram of the collection having a corresponding first probability of occurring in the corpus, and generating a trie representing the collection of n-grams, the trie being represented using one or more arrays of integers, and compressing an array representation of the trie using block encoding; and using the language model to identify a second probability of a particular string of words occurring.