Joint NER and Entity Linking Model for Noisy Social Media Text

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current natural language processing systems face challenges in accurately identifying and linking entities in text, particularly in social media data, due to poor recall and errors propagating from separate named entity recognition (NER) and entity linking (EL) models, which are not optimized for noisy and informal text formats like Twitter.

Innovation Solution

A joint unified probabilistic model is introduced that jointly models NER and EL, allowing predictions from one task to influence the other, using a cross-product prediction space and optimizing a rank-based maximum-margin objective function via stochastic gradient descent, to improve generalization and accuracy in entity identification and disambiguation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If separate NER and EL models are used, then the system structure is simple and easier to implement, but the recall and accuracy deteriorate due to error propagation between stages

Engineering Contradiction:
Improveease of implementationVSAvoidrecall and accuracy
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent merges the previously separate NER and EL models into a single unified joint model. This joint model simultaneously performs both named entity recognition and entity linking tasks, allowing the system to leverage dependencies between the two tasks. By combining the models, errors no longer propagate sequentially from NER to EL, thereby improving recall and accuracy while maintaining a manageable system structure through shared parameters and coordinated training.

Inventive Principle:
Principle #5Merging (Combining)

2Reliability

If a joint unified model is used, then recall and accuracy improve by leveraging task dependencies, but the model complexity and training difficulty increase

Engineering Contradiction:
Improverecall and accuracyVSAvoidmodel complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The joint unified model achieves multi-functionality by simultaneously performing both NER and EL tasks within a single model framework. The model uses shared parameters and features that serve both entity recognition and entity linking purposes. This universal approach allows the model to leverage dependencies between tasks, improving recall and accuracy while managing complexity through parameter sharing and coordinated optimization rather than fully independent sub-models.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Ease of manufacture

If traditional pipeline approach is used, then the system is easier to train with separate models, but errors propagate from NER to EL stage reducing overall performance

Engineering Contradiction:
Improveease of trainingVSAvoiderror propagation
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent combines separate training processes into a unified joint training framework. Instead of training NER and EL models separately in a pipeline fashion, the joint model is trained simultaneously on both tasks using coordinated optimization. This unified training approach allows gradient updates to reflect dependencies between NER and EL objectives, preventing error propagation that occurs when models are trained and applied sequentially in traditional pipeline architectures.

Inventive Principle:
Principle #5Merging (Combining)

Data Source

PatentUS10410139B2Named entity recognition and entity linking joint training
Publication Date: 2019.09.10 ORACLE INT CORP
  • US10410139B2 patent drawing
  • US10410139B2 patent drawing
  • US10410139B2 patent drawing

AI summary

A system that performs natural language processing receives a text corpus that includes a plurality of documents and receives a knowledge base. The system generates a set of document n-grams from the text corpus and considers all n-grams as candidate mentions. The system, for each candidate mention, queries the knowledge base and in response retrieves results. From the results retrieved by the queries, the system generates a search space and generates a joint model from the search space.