Geographic Proximity Matching for Database Record Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems are inefficient in identifying duplicate records across databases due to the sheer number of comparisons required, as they do not effectively utilize geographical proximity to match records, leading to incomplete matching processes.
Innovation Solution
A method and system that utilize geographical proximity by comparing location attributes of contact objects in databases to determine distance using a matching engine, associating records within a threshold distance with a group identification number, and optimizing the process by calculating centroids and adjusting threshold distances to ensure comprehensive matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a brute force algorithm compares each record to every other record, then all possible duplicate matches can be found, but the processing time becomes prohibitively long for large databases
Solution Approach 1:
The patent segments the database into multiple geographic clusters based on location attributes. Each cluster contains records within a specific geographic region, allowing the system to compare records only within their respective clusters rather than across the entire database. This segmentation dramatically reduces the number of comparisons needed while maintaining the ability to detect duplicates through subsequent verification steps.
Solution Approach 2:
The patent introduces geographic location as an intermediary filtering mechanism. By using location attributes and distance calculations as a preliminary filter, the system identifies candidate duplicate pairs that are geographically proximate. This intermediary step reduces the search space before applying more expensive comparison operations, balancing speed and accuracy.
2Productivity
If records are clustered by a single attribute (e.g., first digit of zip code), then processing time is reduced, but matches between records in different clusters are missed
Solution Approach 1:
The patent transitions from one-dimensional clustering (single attribute like zip code prefix) to multi-dimensional geographic clustering using latitude and longitude coordinates. This allows records to be grouped based on actual geographic proximity rather than arbitrary attribute boundaries, enabling the system to capture duplicates that span traditional clustering boundaries while maintaining processing efficiency.
Solution Approach 2:
The patent changes the clustering parameters from discrete attribute values (e.g., zip code digits) to continuous geographic coordinates with distance-based thresholds. By using floating-point distance calculations and configurable radius parameters, the system adapts to varying geographic distributions and maintains both speed and completeness across different data sets.
3Productivity
If geographic proximity is used to pre-filter potential matches, then processing efficiency increases, but matches between records with non-identical location attributes may be missed
Solution Approach 1:
The patent performs preliminary geographic filtering to identify candidate duplicate pairs based on location proximity. These candidates are then subjected to more rigorous verification using additional record attributes. This two-stage approach allows the system to efficiently narrow down the search space while maintaining accuracy through subsequent verification steps that catch false negatives from the preliminary filter.
Data Source
AI summary
Contact objects in one or more databases can be matched using various systems and methods to determine geographical proximity between the objects. Location attributes associated with first and second objects can be compared to determine a distance between the locations associated with the objects. The objects can then be grouped if the distance is less than a threshold distance.