Adaptive Match Keys for Customer Data Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Companies face significant challenges in managing customer data due to frequent changes, inconsistencies, and inaccuracies, leading to resource-intensive database matching processes that consume substantial system resources.
Innovation Solution
The implementation of a system that generates adaptive match keys based on estimating counts, using a graph of nodes and edges to identify cliques and estimate clique counts, allowing for efficient matching and data management with near-perfect recall while reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional shallow matching process is used to identify matching records, then system resource consumption is reduced, but matching accuracy and recall rate deteriorate
Solution Approach 1:
The patent applies dynamics by making the match key generation process adaptive rather than static. The system dynamically generates match keys based on estimated clique counts from graph analysis, allowing the matching process to adjust its depth and resource consumption based on the actual data characteristics and expected match frequency, thereby resolving the contradiction between resource efficiency and matching accuracy
Solution Approach 2:
The patent applies preliminary action by performing graph-based clique estimation before the actual matching process. This preliminary analysis identifies promising match key candidates and their expected effectiveness, allowing the system to prioritize which records to match deeply and which to match shallowly, thus improving accuracy without proportionally increasing resource consumption
2Reliability
If intensive multiple-value matching process is applied to all records, then matching accuracy is improved, but system resource consumption increases significantly
Solution Approach 1:
The patent applies segmentation by dividing the matching process into two distinct stages: shallow matching for initial record identification and intensive multiple-value matching for precise match confirmation. This segmentation allows the system to apply computational intensity only where necessary, improving overall matching accuracy while controlling resource consumption by avoiding unnecessary deep matching of non-candidate records
Solution Approach 2:
The patent applies local quality by applying different matching intensities to different subsets of records based on their likelihood of being matches. Records identified as high-probability candidates through graph-based clique estimation receive intensive multiple-value matching, while other records receive only shallow matching, thereby optimizing the balance between accuracy and resource usage across different parts of the data set
3Speed
If match keys are generated without considering clique counts, then processing speed is improved, but matching precision and recall deteriorate
Solution Approach 1:
The patent applies preliminary action by performing graph-based clique count estimation before generating match keys. This preliminary step identifies which attribute combinations are most promising for matching based on their co-occurrence patterns in the data, allowing the system to generate match keys with higher precision while maintaining processing efficiency by focusing only on the most relevant attribute combinations
4Measurement precision
If graph-based clique estimation is performed for all possible combinations, then matching precision is improved, but device complexity and processing time increase
Solution Approach 1:
The patent applies taking out by extracting only the most relevant cliques from the graph based on their count thresholds and significance. Rather than analyzing all possible attribute combinations, the system identifies and extracts the top-k most promising match key candidates based on their clique counts, thereby improving matching precision while avoiding the complexity and processing burden of exhaustive analysis
Data Source
AI summary
A system creates a graph of nodes connected by edges, the nodes including: i) a first node associated with a first value and a count of the first value, and ii) a second node associated with a second value and a count of the second value, the edges including an edge that connects the first and second nodes and is associated with a count of instances of the first value being stored with the second value. The system includes each node and each associated with clique count less than clique threshold in keys sets and deletes each node and each edge associated with clique count less than clique threshold. The system identifies triplet nodes connected by triplet edges. If estimated clique count for triplet values represented by triplet nodes is less than clique threshold, the system includes triplet values in keys set and identify triplet of nodes as analyzed.