Entity Linking via Contextual Embeddings and MinHash Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional entity linking methods are inefficient in capturing text context and require pre-computation of alias word frequencies, leading to dependency on a typical text corpus, and struggle with polysemy in representing surface entity mentions.
Innovation Solution
A computer-implemented method using a contextual text representation model, such as BERT, to generate contextual entity representation vectors combined with knowledge graph embedding vectors, applied to a fully connected layer with a softmax function for classification, and employing MinHash Locality Sensitive Hashing for alias indexing and similarity scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional entity linking methods use pre-trained word representations and recurrent neural networks, then entity mentions can be embedded, but the methods are inefficient in capturing text context
Solution Approach 1:
The patent transforms the entity linking approach by changing the representation parameters from pre-trained static word embeddings to dynamic contextual embeddings generated by BERT. This parameter change enables the model to capture rich text context while maintaining computational efficiency through the transformer architecture's parallel processing capabilities.
Solution Approach 2:
The patent replaces the mechanical sequential processing of recurrent neural networks with the parallel attention mechanism of transformers. This substitution eliminates the sequential bottleneck while enhancing context capture through self-attention, directly resolving the contradiction between context retention and processing efficiency.
2Reliability
If conventional methods aggregate word representations of associated texts in the knowledge graph, then entities can be represented, but they require pre-computation of alias word frequencies and dependency on typical text corpus
Solution Approach 1:
The patent implements self-service by enabling the BERT model to generate contextual embeddings directly from the input text without requiring pre-computed alias word frequencies or dependency on external text corpora. The model serves itself by leveraging its pre-trained language understanding to immediately represent entities in context, eliminating complex pre-processing pipelines.
Solution Approach 2:
The patent applies preliminary action in the form of pre-training BERT on large corpora before deployment. This pre-training equips the model with general language understanding capabilities, so that during entity linking, it can immediately generate accurate contextual representations without requiring task-specific pre-computation of alias frequencies or corpus dependencies.
3Measurement precision
If conventional approaches use classifier with coreference and coherence features, then entity disambiguation can be performed, but they struggle with polysemy in representing surface entity mentions
Solution Approach 1:
The patent applies dynamics by transitioning from static word embeddings to dynamic contextual embeddings that adapt to each specific usage context. The BERT model generates unique representations for each entity mention based on its surrounding text, enabling the system to dynamically resolve polysemy by capturing the specific meaning intended in each context rather than relying on fixed feature engineering.
Data Source
AI summary
Provided is a method for performing entity linking between a surface entity mention in a surface text and entities of a knowledge graph, including supplying the surface text to a contextual text representation model, pooling contextual representations of the tokens of a surface entity mention in the surface text with contextual representations of the other tokens within the surface text to provide a contextual entity representation vector representing the surface entity mention; supplying an identifier of a candidate knowledge graph entity to a knowledge graph embedding model, to provide an entity node embedding vector and combining the contextual entity representation vector with the entity node embedding vector to generate an input vector applied to a fully connected layer which provides an unnormalized output transformed by a softmax function into a normalized output processed to classify whether the surface entity mention corresponds to the candidate knowledge graph entity.