Entity Matching via Demographic Segmentation and Probabilistic Confidence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity matching systems face challenges in linking entities that have undergone name changes, such as due to marriage or divorce, as they rely on unreliable identifiers like social security numbers, which are also subject to data entry errors and privacy concerns.
Innovation Solution
A method and system for entity matching that parses demographic fields from a request, performs look-up operations in an index to create a candidate list of entities, and uses probabilistic comparisons to determine a unique identifier, mitigating the need for complete demographic data and reducing reliance on sensitive information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If social security numbers or governmental identification numbers are used for entity matching, then entity identification confidence is improved, but reliability deteriorates due to data entry errors and decreased availability
Solution Approach 1:
The system changes the parameters used for entity matching from relying on single identifiers (social security numbers) to using multiple demographic parameters (name, address, date of birth) in combination. This allows the system to maintain identification confidence while reducing reliance on any single error-prone parameter.
Solution Approach 2:
The system creates a composite matching approach by combining multiple demographic data elements (first name, last name, address, date of birth) into a unified matching algorithm. This composite approach compensates for errors in individual fields, similar to how composite materials compensate for weaknesses in individual components.
2Measurement precision
If multiple demographic fields are processed to create candidate lists, then entity matching accuracy is improved, but data processing complexity increases
Solution Approach 1:
The matching process is segmented into distinct phases: first generating a candidate list based on name matching, then filtering candidates using address and date of birth verification. This segmentation allows complex multi-field processing to be broken into manageable, sequential steps that reduce overall system complexity.
Solution Approach 2:
The system performs preliminary filtering by first creating a candidate list based on name matching before applying more complex verification rules. This preliminary action reduces the dataset size early in the process, making subsequent complex processing more manageable and efficient.
3Reliability
If expanded demographic data processing is performed, then entity matching reliability is improved, but processing time increases
Solution Approach 1:
The system performs preliminary name-based filtering to create a candidate list before applying more time-consuming verification rules. This preliminary action significantly reduces the dataset size early in the process, allowing expanded demographic data processing to be applied to a smaller set of candidates, thereby reducing overall processing time.
Solution Approach 2:
The processing is segmented into fast initial filtering (name matching) followed by more intensive verification (address and date of birth checking). This segmentation allows the system to quickly eliminate non-matching records before investing more processing time in detailed verification of potential matches.
Data Source
AI summary
Expanded data processing for entity matching is provided. A request for determining a matching entity is received. The received request is parsed to determine demographic field data associated with the request. A first look up operation is performed in an indexed demographic data for a first demographic field associated with the request. A first list of entities matching with the first demographic field data is received in response to the first look up operation. A second look up operation is performed in the indexed demographic data to determine a set of demographic attributes associated with each entity on the first list. The demographic attributes correspond to a second demographic field associated with the request. A matching confidence for whether the entity is associated with a given entity in the first list is determined based on a comparison between the demographic field data and the demographic attributes.