Duplicate Record Detection via Field Clustering

The patent segments the duplicate detection process into multiple stages: first grouping records by clustering algorithm (which divides the large dataset into smaller clusters), then performing detailed comparison only within clusters. This segmentation reduces the overall computational complexity from O(n²) brute force to approximately O(n log n) or better, while maintaining detection completeness.

The patent applies preliminary clustering action before detailed duplicate comparison. By pre-grouping records into clusters based on similarity metrics (using algorithms like k-means or hierarchical clustering), the system performs preliminary sorting and organization that eliminates the need for exhaustive pairwise comparisons, thereby reducing computational intensity while preserving detection accuracy.

The patent merges the treatment of dependent fields by considering them together during clustering. Instead of processing fields independently, the system combines multiple fields (name, address, phone, etc.) into a unified clustering process that accounts for their interdependencies, thereby improving detection accuracy while maintaining reasonable processing simplicity through automated multi-field evaluation.

The patent changes the approach from independent field processing to integrated record processing by modifying the clustering parameters to consider field relationships. The system adjusts clustering parameters and similarity metrics to reflect real-world dependencies between fields, enabling accurate duplicate detection without complex manual processing.