Blocking Component for Entity Resolution Record Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity resolution methods face computational inefficiencies when matching records across large tables due to the high number of comparisons required, especially in fraud detection where records with inconsistent identifying information need to be identified efficiently.
Innovation Solution
A computer-implemented method involving a configuration component to select blocking variables, a coding component to generate match codes, and a blocking component to group records into blocks based on these codes, reducing the number of comparisons needed by matching records only within their respective blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deterministic or probabilistic entity resolution is performed across all records in large tables, then accurate entity resolution is achieved, but computational expense increases significantly
Solution Approach 1:
The patent divides the large table of records into multiple blocks using blocking variables (e.g., first few characters of name, date of birth ranges, geographic regions). Each block contains a subset of records that are potential matches for each other. This segmentation reduces the number of record comparisons from O(n²) across all records to O(k²) within each block, where k is the average block size, significantly reducing computational expense while maintaining entity resolution accuracy through subsequent matching algorithms applied within each block.
2Measurement precision
If records are compared across the entire data set to identify matches, then comprehensive entity resolution is achieved, but processing time increases
Solution Approach 1:
The patent performs preliminary blocking of records into groups based on blocking variables before executing the matching algorithm. This preliminary action organizes records so that the matching component only needs to compare records within the same block, rather than comparing all records against all other records. This preprocessing step significantly reduces processing time while ensuring matching completeness within each block through deterministic or probabilistic matching algorithms.
3Reliability
If all records are processed for matching without blocking, then accurate fraud detection is achieved, but scalability to large data sets is limited
Solution Approach 1:
The patent segments the data processing task by creating multiple blocks of records based on blocking variables such as name prefixes, date of birth ranges, and geographic regions. Each block is processed independently by the matching component, enabling parallel processing and significantly improving scalability to large data sets. This segmentation maintains fraud detection accuracy by ensuring that potential matches are captured within their respective blocks while reducing the overall computational burden.
Solution Approach 2:
The patent introduces blocking variables as an additional dimension for organizing and processing records. Instead of processing records in a single dimension (all records against all records), the system adds a blocking dimension that groups records into manageable subsets. This dimensional change enables efficient processing of large data sets while maintaining the ability to detect fraudulent activities through subsequent matching within each block.
Data Source
AI summary
Techniques to block records for matching are described. Some embodiments are particularly directed to techniques to block records for matching entities with inconsistent identifying information. In one embodiment, for example, an apparatus may comprise a configuration component, a coding component, a blocking component, and a matching component. The configuration component may be operative to receive a data set comprising a plurality of records and operative to receive a set of blocking variables, the blocking variables present as variables in each of the plurality of records. The coding component operative to generate match codes based on the blocking variables. The blocking component operative on the processor circuit to produce a plurality of blocks of records from the data set based on the match codes. The matching component operative to match records within each of the plurality of blocks by performing deterministic or probabilistic entity resolution based on similar variables of the records within each of the blocks. Other embodiments are described and claimed.