Entity Linking in Noisy Text via Local Context Windows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In noisy text environments, such as unstructured conversations, automatically disambiguating text elements to link them with appropriate entities is challenging due to the lack of clear global context, making it difficult to distinguish between multiple possible entities.
Innovation Solution
A system that determines tokens from a set of words, identifies candidates for these tokens using a candidate dictionary, ranks them based on semantic similarity, and decides whether to select the top-ranked candidate or abstain, utilizing a combination of semantic clusters, popularity, anchor text probabilities, morphology, and string similarity features, with a machine learning algorithm to enhance the disambiguation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automatic entity linking is applied to noisy text environments, then the ability to provide structure and enable downstream applications is improved, but the accuracy of disambiguation deteriorates due to lack of clear global context
Solution Approach 1:
The patent segments the text into multiple local contexts (windows) rather than treating it as a single global context. Each local context window is processed independently to identify entities, allowing the system to handle noisy text by focusing on localized semantic regions where context is more reliable for disambiguation.
Solution Approach 2:
The patent introduces contextual windows as intermediary structures between the raw noisy text and the entity linking process. These windows serve as mediators that capture local semantic information, enabling accurate disambiguation without requiring clear global context throughout the entire text.
2Adaptability or versatility
If multiple possible entities are considered for each text element, then the completeness of entity coverage is improved, but the difficulty of automatic disambiguation increases in noisy environments
Solution Approach 1:
The patent applies different processing qualities to different parts of the text by creating local context windows with varying sizes and focusing levels. Each window is optimized for its specific local context, allowing the system to handle multiple entities appropriately while reducing overall disambiguation complexity through localized processing.
Data Source
AI summary
A system for disambiguation of noisy text includes an interface and processor. The interface is configured to receive a set of words. The processor is configured to determine a token from the set of words, wherein the token includes a subset of the set of words; determine candidates for the token; determine a ranking for the candidates based at least in part on a semantic similarity between the token and each of the candidates; determine whether to select a top ranked candidate or to abstain; and in response to a select determination to select the top ranked candidate, indicate the top ranked candidate is selected.