Graph-Based Entity Resolution With Joint LM–GNN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph-based entity resolution methods either rely on out-of-the-box language models as feature generators or use supervised contrastive methods that only improve graph encoders, lacking a synergistic improvement loop between language models and graph encoders.
Innovation Solution
A framework that jointly trains a language model and a graph neural network using an adjusted variational inference methodology, enabling mutual improvement between models and addressing both attribute and structural information in graph-structured data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If out-of-the-box language models are used as feature generators, then implementation simplicity is maintained, but model performance and accuracy are limited
Solution Approach 1:
The patent merges language model encoding and graph encoder processing into a unified variational inference framework. The language model and graph encoder are jointly trained through shared latent variables and coordinated optimization, allowing them to work together synergistically rather than sequentially. This integration resolves the contradiction by achieving high accuracy through combined models while managing complexity through unified training architecture.
Solution Approach 2:
The variational inference framework establishes feedback loops where the language model and graph encoder mutually refine each other's representations during joint training. The evidence lower bound optimization creates iterative feedback between the two components, allowing each to benefit from the other's improvements. This feedback mechanism enables the system to achieve superior accuracy compared to using out-of-the-box language models alone.
2Measurement precision
If supervised contrastive methods are used to improve language models, then language model performance is enhanced, but only graph encoders benefit while language models do not improve further
Solution Approach 1:
The patent combines language model encoding and graph encoder processing into a unified variational inference framework. The language model and graph encoder are jointly trained through shared latent variables and coordinated optimization, allowing them to work together synergistically rather than sequentially. This integration resolves the contradiction by achieving high accuracy through combined models while managing complexity through unified training architecture.
Solution Approach 2:
The variational inference framework establishes feedback loops where the language model and graph encoder mutually refine each other's representations during joint training. The evidence lower bound optimization creates iterative feedback between the two components, allowing each to benefit from the other's improvements. This feedback mechanism enables the system to achieve superior accuracy compared to using out-of-the-box language models alone.
3Ease of manufacture
If separate training approaches are used for language models and graph encoders, then training simplicity is maintained, but synergistic improvement between models is lost
Solution Approach 1:
The patent merges language model encoding and graph encoder processing into a unified variational inference framework. The language model and graph encoder are jointly trained through shared latent variables and coordinated optimization, allowing them to work together synergistically rather than sequentially. This integration resolves the contradiction by achieving high accuracy through combined models while managing complexity through unified training architecture.
Solution Approach 2:
The patent transforms the training approach by changing the optimization parameters from separate loss functions to a unified evidence lower bound objective. This parameter change enables joint optimization of both language model and graph encoder simultaneously, capturing synergistic effects that separate training cannot achieve. The variational inference framework manages this complexity through structured parameter sharing and coordinated updates.
Data Source
AI summary
The present disclosure relates to entity resolution between graphs of entities and their relations. A language model (LM) and a graph neural network (GNN) may be iteratively trained. A plurality of first node embeddings for a plurality of nodes in a graph may be generated using the LM. A plurality of second node embeddings for the plurality of nodes based at least in part on the plurality of first node embeddings and the graph may be generated using the GNN. A first node and a second node of the plurality of nodes that both represent a particular entity may be identified based at least in part on a similarity between one of the plurality of second node embeddings associated with the first node and one of the plurality of second node embeddings associated with the second node