Locale-Specific Language Model Data Extraction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for creating a language model for a new location are inefficient, as they require manual user input, rely on large data pools that don't address local nuances, and are unsuitable for low-memory devices, leading to incomplete or irrelevant updates.

Innovation Solution

A system that generates a student language model by comparing data from a teacher language model to the existing student model, updating only the necessary data based on user location and preferences, using algorithms to determine the required updates and reduce training time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a student language model is trained by incorporating all data from teacher language models, then the model completeness is improved, but the device memory requirement increases and training time increases

Engineering Contradiction:

Improvemodel completenessVSAvoiddata volume

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the necessary subset of data from teacher language models that is relevant to the target locale, rather than incorporating all data. This is achieved by comparing the teacher model data with existing student model data and identifying gaps specific to the target locale, thereby reducing the data volume while maintaining model completeness for the intended purpose.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality by tailoring the language model data to specific locale requirements. Instead of using generic or universal data from teacher models, the system selectively incorporates data that addresses the nuances and specific characteristics of the target locale, ensuring the model is optimized for local language usage patterns.

Inventive Principle:

Principle #3Local quality

2Reliability

If a student language model is trained by incorporating all data from teacher language models, then the model completeness is improved, but the training time increases

Engineering Contradiction:

Improvemodel completenessVSAvoidtraining time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-comparing the teacher language model data with the existing student model data before initiating training. This comparison identifies the specific subset of data that needs to be incorporated, allowing the training process to focus only on necessary updates rather than retraining with all data, thereby reducing training time while maintaining completeness.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent applies partial action by incorporating only the necessary portion of data from teacher models rather than all data. The system determines the minimal sufficient subset of data required to achieve model completeness for the target locale, avoiding the excessive action of processing unnecessary data and thereby reducing training time.

Inventive Principle:

Principle #16Partial or excessive action

3Manufacturing precision

If manual user input is used to create and train a language model, then the model accuracy for user preferences is improved, but the time required to train the model increases

Engineering Contradiction:

Improvemodel accuracyVSAvoidtraining time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-processing and organizing data from teacher language models before the student model training begins. This includes comparing data sets, identifying relevant subsets, and preparing the data for efficient incorporation, which reduces the actual training time while maintaining the accuracy benefits of user-preferred locale-specific data.

Inventive Principle:

Principle #10Preliminary action

4Adaptability or versatility

If large pools of data from teacher language models are used for training, then the model coverage is improved, but the adaptability to specific locale nuances is reduced

Engineering Contradiction:

Improvemodel coverageVSAvoidlocale specificity

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent applies local quality by selectively incorporating data that specifically addresses target locale nuances. The system compares teacher model data with existing student model data and identifies gaps related to specific locale characteristics, ensuring that the incorporated data enhances locale-specific accuracy rather than diluting it with generic data from broader coverage sources.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20240330609A1Systems and methods to support a new locale in a language model

Publication Date: 2024.10.03 ADEIA GUIDES INC

AI summary

Systems and methods are presented herein for generating a new language understanding model, based on a user request. A user may input a root language and a locale into an application for generating a student language model. The application may generate the student language model and may identify a teacher language model related to the student language model. The application may compare data from the identified teacher language model to the student language model. The application may determine a subset of data from the teacher language model is not contained in the student language model. If the application determines at least a subset of data from the teacher language model is not in the student language model, the application may add at least the subset of data from the teacher language model to the student language model.