Single Identifier Platform Using Graph-Based Entity Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face challenges in efficiently and effectively resolving entities across varying information sources due to inconsistent and incomplete identifying information, leading to difficulties in matching records that correspond to the same entity.
Innovation Solution
A database system employing a scoring model to determine record pair probabilities, generating a graph data structure with nodes and edges, performing connected component analysis, and applying optimal weighted clustering to assign unique identifiers to entities, while pruning edges below a threshold score to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If database systems use predefined rules or heuristics to match records based on identifying information, then record matching can be performed, but accuracy deteriorates when identifying information is inconsistent or incomplete across different sources
Solution Approach 1:
The system dynamically adjusts the weight parameters of different identifying attributes (name, address, phone number, etc.) based on their reliability and completeness in the context of each record pairing. This allows the matching algorithm to adapt to varying information quality across different data sources, improving accuracy without sacrificing versatility.
Solution Approach 2:
The patent implements a dynamic scoring model that continuously evaluates and reweights matching criteria based on the specific characteristics of each record pair. Rather than using static predefined rules, the system adapts its matching parameters in real-time to handle inconsistent and incomplete identifying information from multiple sources effectively.
2Measurement precision
If database systems apply strict matching rules to ensure accuracy, then record matching precision improves, but processing time and computational complexity increase
Solution Approach 1:
The patent segments the entity resolution process into distinct phases: initial filtering using lightweight criteria, intermediate scoring of candidate pairs, and final verification. This segmentation allows the system to quickly eliminate non-matching records while applying more computationally intensive accuracy checks only to promising candidates, thereby maintaining precision without excessive processing time.
Solution Approach 2:
The system applies partial matching actions by first performing quick preliminary assessments on all record pairs using minimal criteria, then applying more rigorous matching procedures only to a subset of candidates that pass the initial filter. This approach achieves high accuracy for final matches while avoiding the time cost of applying strict rules to every possible pair.
3Reliability
If database systems cluster all records together to ensure completeness, then no potential matches are missed, but false positives increase and cluster quality deteriorates
Solution Approach 1:
The patent applies local quality by assigning different quality thresholds and matching criteria to different regions or stages of the clustering process. High-quality matches are identified using strict local criteria within candidate groups, while the overall clustering maintains completeness through a hierarchical approach that ensures no potential matches are initially excluded.
Solution Approach 2:
The system dynamically changes clustering parameters such as threshold scores and connection criteria based on the density and quality of candidate record pairs. In high-quality dense regions, stricter parameters are applied to reduce false positives, while in sparser regions, more lenient parameters maintain completeness, thereby balancing accuracy and efficiency.
Data Source
AI summary
The present disclosure recites systems and methods for generating record clusters. For example, a computer-implemented method comprises receiving a plurality of records from data sources and providing at least a subset of the records to a scoring model that determines scores for various pairings of the records, a score for a given pair of the records representing a probability that the given pair of records contain data elements about the same entity. The method further comprises generating a graph data structure that includes a plurality of nodes, individual nodes representing a different record from the records. The method also comprises assigning a different unique identifier to individual clusters of the final clusters and responding to a request for data regarding a given entity by providing aggregated data elements from those records of the records associated with a cluster of the final clusters having an identifier that represents the given entity.