Interactive Entity Resolution Using Multi-Pass Blocking and Entity Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Entity resolution across different datasets and sources is challenging due to data size, quality inconsistencies, and computational complexity, leading to false positives and negatives, which hinders accurate matching and connection of entities.
Innovation Solution
A machine-learning based entity resolution system that creates an entity graph and index by clustering and partitioning data using multi-pass blocking, distributed clustering, and edge pruning, followed by probabilistic fuzzy matching to resolve entity queries efficiently and accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional entity resolution methods are used on large datasets, then complete entity matching is attempted, but computational resources required become huge and processing time increases
Solution Approach 1:
The patent segments the entity resolution process into multiple passes: blocking phase divides data into candidate pairs using blocking keys, filtering phase applies quick filters to eliminate non-matches, and scoring phase performs detailed similarity assessment only on remaining candidates. This segmentation reduces computational complexity from O(n²) to manageable levels while maintaining matching accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-computing blocking keys, creating blocking indexes, and pre-filtering candidate pairs before the main matching process. This preliminary organization of data structures enables rapid retrieval and comparison during entity resolution, significantly improving processing efficiency.
2Reliability
If traditional entity resolution methods are used on large datasets, then complete entity matching is attempted, but processing time increases
Solution Approach 1:
The patent segments the entity resolution process into multiple passes: blocking phase divides data into candidate pairs using blocking keys, filtering phase applies quick filters to eliminate non-matches, and scoring phase performs detailed similarity assessment only on remaining candidates. This segmentation reduces computational complexity from O(n²) to manageable levels while maintaining matching accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-computing blocking keys, creating blocking indexes, and pre-filtering candidate pairs before the main matching process. This preliminary organization of data structures enables rapid retrieval and comparison during entity resolution, significantly improving processing efficiency.
3Adaptability or versatility
If entity resolution is performed on inconsistent data from multiple sources, then comprehensive entity connection is achieved, but false positives and false negatives increase
Solution Approach 1:
The patent applies local quality by using different blocking strategies and similarity metrics tailored to specific data types and sources. Each data field (name, email, phone, address) has customized blocking keys and comparison rules, allowing the system to adapt to local characteristics of different data sources while maintaining overall matching accuracy.
Solution Approach 2:
The patent implements feedback mechanisms where matching results are continuously evaluated and used to refine blocking strategies and similarity thresholds. The system learns from false positives and negatives by adjusting parameters based on feedback from quality assessment, improving reliability over time.
Data Source
AI summary
An entity resolution method and system may obtain historical entity data from one or more databases to generate an entity matrix based on a clustering of the historical entity data. The entity resolution method and system may partition the entity matrix into one or more disjointed groups of records based on multi-pass blocking of the attributes associated with the one or more entities and cluster the one or more disjointed groups of records based on one or more similarity metrics between each of the one or more disjointed groups of records. The entity resolution method and system may generate an entity graph for the one or more entities based on the clustering and may create an entity index for an entity based on the entity graph. Ultimately, the entity resolution method and system resolves an entity query of a requestor as matching the entity.