Geographic Subgraph Clustering for Record Matching Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems for matching records in databases are inefficient due to the brute force algorithm's high computational complexity, which becomes impractical for large datasets, and previous methods fail to effectively identify duplicate records across intersecting sets.
Innovation Solution
The system uses clustering by generating subgraphs based on criteria and comparing them to produce an intersecting set, where records are matched using methods like geographical proximity to identify duplicates, reducing the number of comparisons required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a brute force algorithm is used to compare each record with every other record, then all possible duplicate records can be identified, but the processing time increases proportionally to n2 making it impractical for large databases
Solution Approach 1:
The patent divides the database into multiple subsets based on geographic proximity (e.g., by zip code, city, or geographic region). Each subset contains records that are geographically close to each other. This segmentation reduces the comparison scope from all n records to smaller subsets, dramatically reducing processing time while maintaining reliability by ensuring that duplicate records within the same geographic area are still detected.
Solution Approach 2:
The patent introduces a geographic dimension as an additional filtering criterion beyond traditional record fields. By adding this spatial dimension, the system creates geographic clusters or subsets that group records by location. This dimensional addition enables the system to reduce the search space efficiently while preserving the ability to detect duplicates, as duplicate records are likely to share geographic characteristics.
2Loss of time
If the database is divided into subsets based on a single attribute (e.g., first digit of zip code), then processing time is reduced, but records in different subsets that are actually duplicates are not compared
Solution Approach 1:
Instead of using a single attribute for segmentation, the patent employs multiple geographic attributes hierarchically (e.g., country → state/province → city → zip code → neighborhood). This multi-level segmentation creates progressively finer subsets, ensuring that duplicate records remain within the same subset at each level while maintaining manageable subset sizes for efficient processing.
Solution Approach 2:
The patent makes the segmentation process multi-functional by using the same geographic segmentation logic for both reducing processing time and ensuring complete duplicate detection. The geographic subsets serve dual purposes: they limit the comparison scope to improve efficiency while simultaneously preserving duplicate records together to maintain detection completeness. This universal application of geographic segmentation resolves the contradiction between speed and accuracy.
Data Source
AI summary
Systems and methods are provided for cluster matching of contact objects. First and second sub graphs can be generated that include respective first and second subsets of the plurality of contact objects. The sub graphs can be intersected to generate an intersecting set of contact object, which can then be analyzed using one or more processes to determine whether matching objects exist in the intersecting set.