Classified Language Model Update System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generic language models trained using a generic corpus lack specificity, leading to misinterpretations in practical applications like speech recognition due to similar pronunciations and lack of relevant vocabulary, and updating these models is resource-intensive and time-consuming.
Innovation Solution
A system and method for updating language models by storing corpus data categorized by specific domains, allowing for the incremental updating of classified language models based on new data within those categories, reducing the need to rebuild the entire model and minimizing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a generic language model is used, then the model can handle various applications, but the model lacks specificity and leads to misinterpretations in practical applications
Solution Approach 1:
The patent divides the single generic language model into multiple classified language models, each trained on specific domain corpora (finance, science, geography, etc.). This segmentation allows each model to specialize in its domain, improving word recognition accuracy for that domain while maintaining overall system adaptability through the ability to select appropriate domain-specific models.
2Adaptability or versatility
If the entire language model is rebuilt by merging old and new corpus, then the model is updated with new data, but time consumption and computing resources become considerable
Solution Approach 1:
The patent segments the language model into multiple independent classified models, each with its own corpus. When updating, only the specific classified model relevant to the new corpus is updated, rather than rebuilding the entire model. This dramatically reduces computing time and resource consumption while maintaining the ability to update the model with new data.
Solution Approach 2:
The patent applies local quality by updating only the specific classified language model that corresponds to the domain of the new corpus, rather than uniformly updating the entire model. This allows targeted updates that minimize computational overhead while ensuring the model reflects the latest data in relevant domains.
3Adaptability or versatility
If the entire language model is rebuilt by merging old and new corpus, then the model is updated with new data, but computing resources become considerable
Solution Approach 1:
The patent segments the language model into multiple independent classified models, each with its own corpus. When updating, only the specific classified model relevant to the new corpus is updated, rather than rebuilding the entire model. This dramatically reduces computing time and resource consumption while maintaining the ability to update the model with new data.
Solution Approach 2:
The patent applies local quality by updating only the specific classified language model that corresponds to the domain of the new corpus, rather than uniformly updating the entire model. This allows targeted updates that minimize computational overhead while ensuring the model reflects the latest data in relevant domains.
Data Source
AI summary
A system for updating language models is provided. The system includes a data-storage module, a data-update module, and a model-building module. The data-storage module is used for storing multiple pieces of corpus data that corresponds to multiple categories. The data-update module is used for storing a piece of new corpus data into the data-storage module. The piece of new corpus data corresponds to one of the categories. The model-building module is used for building a plurality of classified language models, and for updating one of the classified language models based on the piece of new corpus data stored in the data-storage module. The classified language model updated corresponds to the category that corresponds to the piece of new corpus data.