Language Model Generation via Paraphrase Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition techniques face misrecognition issues due to the sparseness problem in n-gram models, where sequences of words not in learning example sentences are assigned a zero appearance probability, and smoothing methods can introduce sequences that never occur in practice, increasing misrecognition risk.
Innovation Solution
A language model is generated using not only learning example sentences but also n-grams learned from synonymous phrases, which allows for the inclusion of paraphrases to reduce misrecognition by improving the model's ability to predict sequences of words that do not exist in the training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If smoothing is applied to n-gram models to handle sparseness, then the model can predict sequences not in training data, but sequences that never occur in practice are included, increasing misrecognition risk
Solution Approach 1:
The patent applies preliminary action by generating paraphrases of training sentences before building the n-gram model. This expands the training data with semantically equivalent variations, allowing the model to learn from diverse表达方式 of the same meaning. When speech recognition encounters a sequence not exactly in training data, it can match against paraphrased versions, improving adaptability without relying on smoothing that introduces erroneous sequences.
Solution Approach 2:
The patent uses copying by creating multiple paraphrased versions of each training sentence. These copies contain the same semantic information but different wording, enabling the model to generalize better. Instead of using smoothing to estimate probabilities for unseen sequences, the system has actual training examples (copies) to reference, maintaining reliability while improving versatility.
2Device complexity
If n-gram models are trained only on learning example sentences, then the model structure is simple, but sequences of words not in learning example sentences are assigned zero appearance probability, causing misrecognition
Solution Approach 1:
The system performs preliminary action by automatically generating paraphrases of training sentences before model construction. This expands the effective training data without requiring manual creation of additional examples. The paraphrasing process is applied systematically to all training sentences, creating a richer dataset that helps the model recognize varied expressions while maintaining a relatively simple overall architecture.
Solution Approach 2:
The patent applies parameter changes by transforming the training data through paraphrasing operations. This changes the linguistic parameters (wording, structure) while preserving semantic meaning. The model learns from these parameter variations, improving its ability to handle unseen sequences without fundamentally changing the n-gram model structure itself.
Data Source
Figure 1~2
Figure 3~6
Figure 7~8
AI summary
A language model generating device according to the present invention includes: a paraphrase generating unit to generate, by using morphemes of a phrase included in learning example sentences that include a plurality of sentences and using synonyms for original expressions of the morphemes, a plurality of paraphrases that include a combination of an original expression of a morpheme and a synonym for an original expression of a morpheme and a combination of synonyms for original expressions of morphemes; and a language model generating unit to generate a language model that is based on an n-gram model from the plurality of paraphrases generated and the learning example sentences. Thus, the obtained language model can reduce speech misrecognition of a sequence of words that does not exist in the learning example sentences.