Morpheme-Based Language Model for Portable Device Word Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional language models face challenges in handling word inflection, particularly in languages with complex grammatical structures, leading to increased vocabulary sizes and difficulties in training data acquisition, which is impractical for deployment on portable devices.
Innovation Solution
A morpheme-based language model is employed, decomposing words into prefixes, stems, and suffixes, allowing for concurrent prediction of these components based on context, reducing the need for extensive training data and improving prediction efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional language models partition words according to morphological information, then word prediction accuracy is improved, but device complexity increases due to separate histories required for each stem and suffix category
Solution Approach 1:
The patent segments words into morphological components (stems and suffixes) and processes them through separate embedding layers and historical contexts. This segmentation allows the model to capture morphological patterns while maintaining manageable complexity through modular architecture design.
Solution Approach 2:
The patent implements a unified language model architecture that handles both stem and suffix predictions simultaneously through shared computational resources. The model uses universal transformation matrices and combined loss functions to efficiently process multiple morphological categories without requiring entirely separate model instances.
2Adaptability or versatility
If vocabulary size is increased to accommodate word inflections, then language coverage is improved, but training data acquisition becomes impractical for portable devices
Solution Approach 1:
By segmenting the vocabulary into morphological components (stems and suffixes), the model reduces the effective vocabulary size needed for training. Instead of requiring training data for every possible inflected form, the model learns from smaller sets of root forms and applies morphological rules to generate predictions for derived forms.
Solution Approach 2:
The patent transforms the representation of words from whole-word embeddings to morpheme-based embeddings. This parameter change allows the model to generalize across inflected forms by learning from the underlying morphological structure, reducing the amount of training data required while maintaining comprehensive language coverage.
Data Source
AI summary
The present disclosure generally relates to systems and processes for morpheme-based word prediction. An example method includes receiving a current word; determining a context of the current word based on the current word and a context of a previous word; determining, using a morpheme-based language model, a likelihood of a prefix based on the context of the current word; determining, using the morpheme-based language model, a likelihood of a stem based on the context of the current word; determining, using the morpheme-based language model, a likelihood of a suffix based on the context of the current word; determining a next word based on the likelihood of the prefix, the likelihood of the stem, and the likelihood of the suffix; and providing an output including the next word.