Capped Linear Combination for Customer Record Linking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Comparing customer records to identify linked records is computationally expensive and time-consuming, especially as the number of records grows exponentially, requiring significant processing power and often resulting in inefficient comparisons due to the need for brute force analysis.
Innovation Solution
Implementing a computer system that uses capped linear combinations and hierarchical blocking techniques to systematically select and compare records, reducing unnecessary comparisons by grouping records based on data attributes and applying multiple blockers to increase the likelihood of identifying linked records while minimizing computational time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If brute force comparison of all customer records is performed, then identification accuracy of linked records is improved, but computational time and processing power requirements increase exponentially
Solution Approach 1:
The patent segments the customer record comparison process into multiple hierarchical stages: blocking (grouping records by common attributes), pre-comparison filtering (applying capped linear combinations to distance values), and detailed comparison. This segmentation divides the exponentially complex full comparison task into manageable segments that can be processed efficiently at each stage, maintaining accuracy while reducing overall computational time.
Solution Approach 2:
The patent applies preliminary actions by performing blocking and pre-comparison filtering before conducting detailed record comparisons. Records are first grouped by blocking rules, then pre-compared using capped linear combinations to eliminate obvious non-matches. This preliminary action reduces the set of records requiring full detailed comparison, thereby reducing computational time while preserving identification accuracy for the remaining candidates.
2Quantity of substance
If the number of customer records increases, then comprehensive customer data coverage is improved, but the number of comparisons required grows exponentially
Solution Approach 1:
The patent segments the large dataset into smaller blocks using blocking rules that group records by common attributes (e.g., name, address, phone number). This segmentation allows the system to process large quantities of records by handling smaller blocks independently, preventing the exponential growth of comparisons while maintaining comprehensive data coverage across all blocks.
Solution Approach 2:
The patent applies partial action by not performing full detailed comparisons on all record pairs. Instead, it uses blocking to identify candidate pairs and capped linear combinations to filter further, performing exhaustive detailed comparison only on a small subset of high-probability matches. This partial action maintains productivity even as data coverage increases.
3Measurement precision
If detailed comparison of all record attributes is performed, then link identification accuracy is improved, but processing power requirements increase
Solution Approach 1:
The patent segments the attribute comparison process into hierarchical stages: blocking (comparing key attributes for grouping), pre-comparison (comparing selected attributes using capped linear combinations), and detailed comparison (comparing all attributes). This segmentation allows detailed comparison to be applied only to a small subset of candidate pairs identified at earlier stages, maintaining link identification accuracy while reducing overall processing power requirements.
Solution Approach 2:
The patent applies local quality by applying different comparison intensities to different sets of records. High-intensity detailed comparison is applied locally to candidate pairs that pass blocking and pre-comparison filters, while low-intensity or no comparison is applied to the majority of record pairs. This local quality approach maintains accuracy for identified links while reducing total processing power consumption.
Data Source
AI summary
Techniques for comparing customer records to identify linked customer records are provided. The techniques may include a computer system identifying a pair of customer records and comparing the customer records to determine if the customer records are linked records. The computer system may analyze a first set of data values from first corresponding data fields from the pair of records to determine a first distance between the first set of data values, analyze a second set of data values from second corresponding data fields from the pair of records to determine a second distance between the second set of data values, and combine the first distance and the second distance into a combined distance value which is representative of a distance between the pair of customer records by using a combination function which provides a maximum value cap for the first distance and a weight for the first distance.