Dictionary Ambiguity Detection via Context Pattern Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing systems face challenges in identifying and managing ambiguous and spurious terms in dictionaries, leading to errors in information extraction and retrieval, as these terms can have different meanings or be irrelevant in different textual domains.
Innovation Solution
A system utilizing an AI platform with a dictionary manager, context manager, and director to apply seed terms to a text corpus, characterize linguistic properties, construct context patterns, and quantify matching content, identifying and selectively removing anomalous seed words to mitigate ambiguity and spurious terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a dictionary contains more seed terms to improve coverage, then the quantity of substance increases, but the reliability decreases due to inclusion of ambiguous and spurious terms
Solution Approach 1:
The system extracts and removes ambiguous and spurious terms from the dictionary by identifying them through context pattern analysis. The director component extracts context patterns from the corpus and uses them to evaluate seed terms, removing those that exhibit anomalous behavior or do not align with actual usage patterns in the corpus.
Solution Approach 2:
The system implements feedback by using context patterns extracted from the corpus to evaluate and refine the dictionary. The director component provides feedback on seed term performance by comparing extracted context patterns against dictionary definitions, allowing iterative improvement of the dictionary's reliability while maintaining coverage.
2Measurement precision
If manual dictionary management is performed to improve reliability, then the measurement precision improves, but the productivity decreases due to time-consuming processes
Solution Approach 1:
The system enables self-service by automatically evaluating seed terms against context patterns extracted from the corpus. The director component autonomously identifies ambiguous and spurious terms through automated analysis of context patterns, eliminating the need for manual evaluation while maintaining high precision in term assessment.
Solution Approach 2:
The system replaces manual dictionary management with automated computational processes. The context manager and director components use algorithmic analysis of context patterns to evaluate seed terms, substituting human judgment with automated machine learning-based assessment that achieves comparable or superior precision without the time cost of manual processes.
Data Source
AI summary
Embodiments relate to a system, program product, and method for dictionary membership management directed at identifying ambiguity in semantic resources. A dictionary of seed terms is applied to a text corpus and matching items in the corpus are identified. The linguistic properties for each matching item are characterized and a context pattern of each matching item is constructed. Each context pattern is applied to the dictionary and matching content between the seed terms and the context pattern is identified and quantified. Lexicon items from the dictionary that have anomalous behavior reflected in the quantification are identified. One or more seed words identified as having anomalous behavior are selectively removed from the dictionary.