Stem Suffix N-gram Language Model for Portable Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing language models face challenges in accurately predicting words for highly inflected languages due to the large vocabulary sizes required, which can lead to inefficient training data collection, storage issues, and reduced predictive power on portable devices, with grammatically incorrect predictions being particularly problematic.
Innovation Solution
A structured stem and suffix n-gram language model is employed, which separates suffix types such as person and tense from the stem, using decoupled language models to train stem and suffix data, allowing for improved word prediction accuracy by considering the stem, person suffix, and tense suffix probabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extensive training data is collected to improve language model coverage and robustness, then prediction accuracy improves, but data storage requirements and model size increase beyond what can be deployed on portable devices
Solution Approach 1:
The patent segments the language model into multiple specialized n-gram models, each trained on specific linguistic features (e.g., person suffixes, tense suffixes, stems). This segmentation allows the system to achieve comprehensive coverage through combination of specialized models rather than requiring a single large model, reducing overall storage requirements while maintaining prediction accuracy for highly inflected languages
Solution Approach 2:
The patent extracts and separates specific linguistic features (suffixes like person and tense) from the complete language model. By isolating these features into dedicated n-gram models, the system can store and process only the necessary linguistic components rather than maintaining a complete large-vocabulary model, thereby reducing model size while preserving prediction capability
2Adaptability or versatility
If a large vocabulary language model is used to cover highly inflected languages, then word coverage improves, but training data collection becomes more burdensome and storage requirements increase
Solution Approach 1:
The patent divides the language modeling task into segmented components based on linguistic features (stems, person suffixes, tense suffixes). Each segment is handled by a specialized n-gram model trained on feature-specific data, allowing comprehensive word coverage through systematic combination of specialized models rather than requiring a single large-vocabulary model with extensive training data
3Quantity of substance
If the language model is pruned to reduce size for portable device deployment, then storage requirements decrease, but predictive power and accuracy are reduced
Solution Approach 1:
Instead of pruning a large model, the patent segments it into multiple small specialized n-gram models. Each model is optimized for a specific linguistic feature and can be independently stored and processed. The combination of these segmented models maintains predictive power for highly inflected languages while keeping individual model sizes small enough for portable device deployment
Solution Approach 2:
The patent changes the parameter organization from a single large vocabulary model to multiple models with specialized parameter sets (e.g., person suffix n-grams, tense suffix n-grams, stem n-grams). This parameter reorganization allows the system to maintain comprehensive predictive coverage while reducing the size of individual model components that must be stored in memory
4Device complexity
If traditional language models are used for highly inflected languages, then implementation is simpler, but grammatically incorrect predictions increase
Solution Approach 1:
The patent segments the language model into grammatically-aware components that separately model stems, person suffixes, and tense suffixes. This segmentation enables the system to enforce grammatical constraints by ensuring that predicted suffixes are compatible with the predicted stem, thereby reducing grammatically incorrect predictions while maintaining reasonable model complexity
Solution Approach 2:
The patent introduces grammatical feature models (person suffix n-gram, tense suffix n-gram) as intermediaries between the stem prediction and the final word formation. These intermediary models ensure that grammatical constraints are satisfied by filtering and validating suffix combinations based on their compatibility with the predicted stem, thereby improving grammatical correctness
Data Source
AI summary
Systems and methods are disclosed for predicting words using a structured stem and suffix n-gram language model. The systems and methods include determining, using a first n-gram word language model, a first probability of a stem based on a first portion of a previously-input word in the received input. Using a second n-gram language model, a second probability of a first suffix may be determined based at least on a second portion the previously-input word in the received input. Further, a third probability of a second suffix different from the first suffix may be determined using a third n-gram language model based at least on a third portion of the previously-input word in the received input. A fourth probability of a predicted word may be determined based on the first, second and third probabilities. One or more predicted words may be determined and provided as an output to the user.