Record Matching System Using Segmented Exact and Fuzzy Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large datasets containing electronic records of entities, such as individuals and businesses, often lack clear links due to variations in data entry, including misspellings and differences in data formats, making it difficult to accurately match and correlate records across disparate systems.
Innovation Solution
A system that performs both exact and fuzzy matching of records using similarity rules and ontologies to standardize data formats, ensuring that only necessary fuzzy matching is performed to optimize processing time and reduce false positives, while generating user-friendly interfaces for validation by users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fuzzy matching is performed extensively to improve record matching accuracy, then matching completeness is improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent segments the matching process into two distinct phases: exact matching (comparing records with identical key fields like name, address, SSN) and fuzzy matching (comparing records with similar but not identical fields). This segmentation allows the system to perform comprehensive fuzzy matching only on subsets of records that pass initial exact matching filters, thereby maintaining high matching accuracy while significantly reducing overall processing time and computational resources.
Solution Approach 2:
The patent applies preliminary exact matching actions before performing fuzzy matching. By first identifying records with exact matches on key identifiers and then applying fuzzy matching only to records that are close matches, the system prepares the data in advance to minimize the scope of computationally intensive fuzzy matching operations, thus resolving the contradiction between matching completeness and processing efficiency.
2Reliability
If fuzzy matching is performed between all records to ensure complete matching, then matching completeness is improved, but false positives increase
Solution Approach 1:
The patent applies different matching quality standards to different subsets of records. Exact matching with strict equality checks is applied to records with identical key fields, while fuzzy matching with similarity thresholds is applied to records with potential variations. This local differentiation of matching strictness ensures high completeness without generating excessive false positives, as each record pair is evaluated with the appropriate level of stringency.
Solution Approach 2:
The patent changes the matching parameters dynamically based on the characteristics of the records being compared. For exact matching, it uses strict equality parameters; for fuzzy matching, it uses similarity threshold parameters that allow for variations in spelling, formatting, and data quality. This parameter adaptation allows the system to maintain high matching completeness while controlling false positive rates through context-appropriate matching criteria.
3Reliability
If comprehensive fuzzy matching is performed on all records, then data correlation completeness is improved, but system complexity and resource consumption increase
Solution Approach 1:
The patent divides the data correlation system into modular components: an exact matching module that handles records with identical identifiers, a fuzzy matching module that handles records with variations, and a result integration module that combines both results. This segmentation reduces system complexity by allowing each module to be independently optimized and managed, while still achieving comprehensive data correlation through their coordinated operation.
Data Source
AI summary
Methods, systems and apparatus, including computer programs encoded on computer storage media for a data record resolution and correlation system. One of the methods includes obtaining data sets each including records associated with entities, the records each specifying information for a plurality of aspects of an entity, the information being encoded in the records, for each of the data sets, according to a respective format. One or more bundles are determined from the data sets, with each bundle indicating records determined to be associated with a same entity and determined using similarity rules that compare aspects specified in the records. Information describing a particular bundle is presented to a reviewing user, and the reviewing user can specify one or more records included in the particular bundle that are properly matched. The particular bundle can be updated based on the reviewing user's specification.