Database Entity Resolution via Multi-Criteria Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intelligence analysts and other computer users face challenges in determining unique real-world individuals in large databases due to duplicate names, misspellings, and missing information, especially when importing data from external sources, leading to duplication of records and data inconsistencies.
Innovation Solution
A database system that performs entity resolution by receiving user input to select and match imported data entities with existing data entities using specified criteria, consolidating data, and storing matching criteria for future use, allowing for automatic matching and consolidation of data without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual verification of each data entry is performed to ensure uniqueness, then data accuracy is improved, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system performs self-verification through automated entity resolution. The database automatically compares imported data against existing records, identifies duplicates based on defined criteria, and consolidates entries without requiring manual verification by analysts, thus achieving both accuracy and efficiency
Solution Approach 2:
The patent replaces manual mechanical verification processes with automated computational algorithms. The system uses processing logic to automatically match entities, compare data fields, and resolve duplicates, substituting human manual checking with machine-based automated systems
2Productivity
If automated entity resolution is implemented to reduce manual work, then productivity is improved, but system complexity and difficulty of detecting and measuring increase
Solution Approach 1:
The entity resolution process is segmented into distinct manageable steps: data import, entity identification, matching against criteria, consolidation, and verification. This segmentation allows the complex automated system to operate through modular processes that are easier to implement, maintain, and debug
Solution Approach 2:
The system handles complexity by allowing flexible parameter configuration for matching criteria (such as name similarity thresholds, field weightings, and tolerance levels). By adjusting these parameters, the system can adapt to different data scenarios without requiring fundamental system redesign, thus managing complexity through configurability
3Reliability
If strict matching criteria are applied to ensure data consistency, then data quality is improved, but adaptability to handle variations in spelling and formatting decreases
Solution Approach 1:
The matching criteria are applied locally to different data fields with appropriate specificity. Instead of using a single rigid matching rule for all fields, the system applies field-specific criteria (such as exact matching for SSN, fuzzy matching for names, and pattern matching for addresses), allowing each field to be evaluated according to its particular characteristics and tolerance levels
Solution Approach 2:
The matching criteria are dynamic rather than static. The system can adjust matching stringency, tolerance levels, and comparison methods based on the specific data being processed, the confidence threshold required, and operational needs. This dynamic adaptation allows the system to maintain high data consistency while accommodating variations in spelling, formatting, and data quality
Data Source
AI summary
A method of matching a plurality of imported data entities to a plurality of existing data entities in a database is disclosed. The method comprises receiving imported data comprising imported data entities each having one or more properties each having one or more values; and causing presentation of a graphical user interface (GUI) that displays values for the one or more properties over the imported data and allows selection of a property having a certain value. The method further comprises receiving first input specifying first matching criteria that is based at least in part on a first subset of one or more properties of the imported data entities; and receiving second input specifying second matching criteria that is different from the first matching criteria and that is based at least in part on a second subset of the one or more properties of the imported data entities, wherein the second subset of the one or more properties is different than the first subset of one or more properties. The method also comprises receiving a first selection of a first of the one or more properties having a first value and a second selection of a second of the one or more properties having a second value through the GUI; and generating a first subset of the imported data entities based on the first selection and a second subset of the imported data entities based on the second selection. Finally, the method comprises matching the first subset of the imported data entities to a first subset of the existing data entities using the first matching criteria and using a first matching technique; and matching the second subset of the imported data entities to a second subset of the existing data entities using the second matching criteria and using a second matching technique, where the method is performed using one or more processors.