Language Model Term Prioritization and Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing language models and ontologies struggle to efficiently prioritize the expansion and pruning of terminology and phrases within specific domains, leading to unbounded growth and increased computational overhead.
Innovation Solution
A system and method that automatically normalize terminology and phrases by receiving text from multiple platforms, determining term presence in a stored data model, identifying new contexts, and either adding or removing terms based on frequency and context, with human validation for critical decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If language models continuously expand to include new terms from multiple platforms, then terminology coverage and adaptability improve, but model size and computational overhead increase unboundedly
Solution Approach 1:
The system changes the parameter of term importance by calculating priority scores based on multiple factors including frequency of use, recency, source reliability, and domain relevance. This allows the language model to dynamically adjust which terms are retained during expansion, maintaining adaptability while controlling model size through priority-based selection rather than unbounded growth
Solution Approach 2:
The system applies different quality standards to different terms by evaluating each term's priority independently. High-priority terms (those with high frequency, recency, and domain relevance) receive full integration, while lower-priority terms are pruned or marked for deferred addition. This local differentiation allows the model to maintain high terminology coverage for important terms while limiting overall model size growth
2Loss of information
If language models include all encountered terms, then terminology completeness improves, but processing efficiency and speed deteriorate
Solution Approach 1:
The system changes the parameter of term retention by introducing priority thresholds. Terms above a certain priority threshold are retained in the language model to maintain completeness, while terms below the threshold are pruned. The priority is calculated based on frequency, recency, and domain relevance, allowing the system to maintain processing efficiency by excluding low-value terms while preserving terminology completeness for high-value terms
Solution Approach 2:
The system extracts and removes low-priority terms from the language model to improve processing efficiency. By identifying terms with low frequency, old recency, or low domain relevance, the system extracts these terms for pruning, thereby reducing model size and improving processing speed while maintaining completeness for important terms
3Measurement precision
If manual review is implemented for term addition, then term selection accuracy improves, but system automation and processing speed decrease
Solution Approach 1:
The system provides self-service by automatically calculating priority scores for candidate terms and performing pruning decisions based on predefined thresholds. The automated priority calculation considers frequency of use, recency, source reliability, and domain relevance, allowing the system to make accurate term selection decisions without continuous manual intervention. Manual review is only required for edge cases or high-stakes decisions, maintaining high automation while preserving accuracy
Data Source
AI summary
A system and method for updating computerized language models is provided that automatically adds or deletes terms from the language model to capture trending events or products, while maximizing computer efficiencies by deleting terms that are no longer trending and use of knowledge bases, machine learning model training and evaluation corpora, analysis tools and databases.