Focused Multi-Blocking for Record Link Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational time required for comparing large numbers of customer records to identify linked records grows exponentially, making the process slow and resource-intensive, as each comparison is computationally expensive and requires significant processing power.
Innovation Solution
The implementation of focused multi-blocking techniques, where records are systematically selected and grouped using algorithms like double metaphone and zip code to assign block keys, reducing the number of comparisons needed by applying multiple focused blocks, allowing for efficient identification of linked records while minimizing computational time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all records are compared pairwise to identify linked records, then the link identification rate is maximized, but the computational time and processing resources increase exponentially
Solution Approach 1:
The patent divides the large set of records into smaller blocks using blocking keys (such as hashed values of name fields, address fields, or other identifying attributes). Records are only compared within their own block rather than against all other records, dramatically reducing the number of comparisons needed while maintaining high link identification rates through multiple blocking passes.
Solution Approach 2:
The patent performs preliminary blocking and grouping of records before the actual comparison process. By pre-organizing records into blocks based on blocking keys, the system eliminates the need for exhaustive pairwise comparisons, reducing computational time from exponential to linear or near-linear complexity.
2Quantity of substance
If the number of records increases, then the coverage and completeness of customer data improves, but the number of comparisons grows exponentially requiring more computing power
Solution Approach 1:
The patent segments the growing record set into manageable blocks using blocking keys derived from record attributes. This segmentation allows the system to handle increasing quantities of records by distributing them across multiple blocks, where comparisons are performed locally within each block rather than globally across all records, keeping computational resource requirements manageable.
Solution Approach 2:
The patent introduces blocking keys as intermediary structures that mediate between the large set of records and the comparison process. These blocking keys act as indices or mediators that quickly identify which records should be compared, avoiding the need for direct exhaustive comparison and reducing the computational complexity associated with large record quantities.
3Reliability
If multiple blocking passes are performed to increase link identification rates, then more linked records are found, but the processing time increases
Solution Approach 1:
The patent employs dynamic blocking where blocking keys are generated or refined based on the results of previous blocking passes. The system adapts the blocking strategy across multiple passes, using information from earlier passes to improve subsequent blocking, thereby increasing link identification rates while minimizing the time penalty through intelligent adaptation rather than brute-force repetition.
Data Source
AI summary
Techniques for comparing customer records to identify linked customer records pertaining to a single customer entity are provided. The techniques include identifying a target group of electronic customer records having data fields containing data pertaining to a customer, identifying one or more focused blockers identifying a data value for an electronic customer record data field, and analyzing the target group of electronic customer records to identify a focused group of electronic customer records containing the focused blocker data value. The techniques also include comparing pairs of electronic customer records from the focused group of electronic customer records to identify linked records which pertain to a single customer entity.