Unified CNN-RNN Architecture for Entity Linking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity linking systems face limitations in capturing global interdependence among target entities within a document, as they either overlook topical coherence or rely on discrete features that lead to data sparseness and noise, while previous neural network models are local and cannot effectively model global relationships.
Innovation Solution
A unified architecture using convolutional neural networks (CNNs) to learn local representations and recurrent neural networks (RNNs) to capture topical coherence among entity mentions and candidate entities, enabling simultaneous modeling of local and global features for entity linking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If local approach with independent ranking is used, then discrete features can be exploited for entity linking, but topical coherence among target entities is overlooked
Solution Approach 1:
The patent merges the local approach (independent ranking with discrete features) and global approach (simultaneous disambiguation with topical coherence) into a unified neural network framework. The CNN-RNN architecture processes both local mention-entity pairs and global document-level entity sequences together, allowing the model to learn both local relatedness and global topical coherence jointly from the same data, thereby resolving the contradiction between exploiting discrete features and capturing topical coherence
Solution Approach 2:
The patent creates a composite feature representation by combining discrete hand-designed features with continuous neural network-generated features. The system integrates traditional discrete features (lexical tags, dependency paths, KB infoboxes) with continuous vector representations learned by CNN and RNN models, forming a hybrid feature space that preserves the advantages of both discrete feature exploitation and global coherence capture
2Loss of information
If global models with simultaneous disambiguation are used, then topical coherence is captured, but discrete features still suffer from data sparseness
Solution Approach 1:
The patent transforms the feature representation parameters from discrete categorical values to continuous vector embeddings. By converting discrete features (words, entities, attributes) into continuous dense vectors through neural network embeddings, the system enables effective learning even with limited data. This parameter transformation allows the model to generalize better and overcome data sparseness while maintaining global coherence modeling
Solution Approach 2:
The patent replaces the traditional mechanical feature engineering approach (hand-designed discrete features) with a neural network-based automatic feature learning system. Instead of manually crafting discrete features that suffer from sparseness, the system uses CNN and RNN models to automatically learn continuous feature representations from data, thereby eliminating the data sparseness problem while preserving global coherence capture
3Quantity of substance
If neural network models are used for local features, then data sparseness is alleviated, but global interdependence among target entities cannot be captured
Solution Approach 1:
The patent segments the entity linking task into two distinct but integrated components: local mention-to-entity matching (handled by CNN) and global entity sequence modeling (handled by RNN). This segmentation allows each neural network component to specialize - CNN focuses on local feature representation with continuous embeddings that alleviate data sparseness, while RNN focuses on capturing global interdependence among entities in the document, thereby resolving the contradiction between local feature learning and global coherence
Data Source
AI summary
A system, method and computer program product for disambiguating one or more entity mentions in one or more documents. The method facilitates the simultaneous linking entity mentions in a document based on convolution neural networks and recurrent neural networks that model both the local and global features for entity linking. The framework uses the capacity of convolution neural networks to induce the underlying representations for local contexts and the advantage of recurrent neural networks to adaptively compress variable length sequences of predictions for global constraints. The RNN functions to accumulate information about the previous entity mentions and/or target entities, and provide them as the global constraints for the linking process of a current entity mention.