Entity Linking via Attention-Based Entropy Minimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search algorithms struggle to disambiguate entities with ambiguous meanings in large-scale computer networks, leading to incorrect search results, particularly for proper nouns and common names shared across different contexts, such as Perth in Scotland and Australia.
Innovation Solution
An unsupervised machine learning technique is implemented to refine entity linking models using an attention mechanism that minimizes entropy in the output distribution, allowing the system to distinguish between similar entities without labeled data, effectively disambiguating entities in search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If search algorithms expand results based on frequently appearing terms, then search result coverage is improved, but measurement precision deteriorates due to ambiguous entity meanings
Solution Approach 1:
The patent changes the parameter of entity identification from simple term matching to probability-based entity scores. By computing entity scores that reflect the likelihood of each entity being the intended target, the system can distinguish between ambiguous entities (e.g., Perth in Scotland vs. Perth in Australia) while still expanding search results based on frequently appearing terms. This resolves the contradiction by transforming the identification mechanism to maintain both coverage and precision.
Solution Approach 2:
The patent replaces the mechanical term-matching system with a machine learning-based probability scoring system. Instead of relying on rigid keyword matching that cannot handle ambiguity, the system uses trained models to compute entity scores and disambiguate between multiple possible entities. This substitution enables the system to maintain high result coverage while accurately identifying the intended entity despite ambiguous terminology.
2Measurement precision
If entity linking uses labeled data for training, then measurement precision is improved, but device complexity increases due to data annotation requirements
Solution Approach 1:
The patent implements self-service by training entity linking models on unlabelled data through unsupervised learning techniques. The system automatically learns entity disambiguation patterns without requiring manual annotation of training data, eliminating the need for complex data annotation infrastructure while achieving high entity linking accuracy. The model learns from the structure and context in the data itself, making the system self-sufficient.
Solution Approach 2:
The patent inverts the conventional approach by using unlabelled data instead of labeled data for training. Rather than requiring annotated examples to teach the system, the model learns from raw unlabelled text, inverting the traditional supervised learning paradigm. This inversion eliminates the bottleneck of data annotation while maintaining or improving entity linking performance through unsupervised pattern discovery.
Data Source
AI summary
In an example, for each of one or more terms in a text document, one or more entities to which the term potentially maps are identified. The text document includes at least one ambiguous term. One or more features are extracted from the text document. An attention model is applied to the text document based on the extracted one or more features, resulting in an attention weight being applied to each of the one or more terms in the text document. The one or more terms are encoded based on the attention weights. Each of one or more ambiguous terms is classified based on the encoded terms, the classification assigning a value to each different entity that each ambiguous term potentially maps to. A minimum entropy loss function is evaluated using the classification, and results are back-propagated to the attention model.