Entity Resolution Using Multi-Attribute Rule Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual data cleansing and entity resolution in large datasets are inefficient and prone to inaccuracies, especially when dealing with millions or billions of data entries containing inconsistencies and duplications.
Innovation Solution
A computer-implemented method and system that applies multiple rules with different attribute matching levels to identify and link data entries associated with a single entity, using a server with components like a rule generator, searcher, synthesizer, and matching confidence generator to deduplicate and integrate data entries efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review is used for data cleansing, then accuracy can be maintained, but efficiency and time consumption deteriorate significantly when handling large datasets
Solution Approach 1:
The patent segments the data cleansing task into multiple rule-based matching levels (e.g., exact match, partial match, fuzzy match) applied to different attribute groups. This allows automated processing to handle routine cases efficiently while maintaining accuracy through hierarchical rule application, resolving the contradiction between manual accuracy and automated efficiency.
Solution Approach 2:
The system changes parameters by applying multiple rules with different matching strictness levels to the same data entries. By varying the matching criteria (from strict exact match to more lenient fuzzy match), the system achieves both high accuracy through multiple validation layers and high efficiency through automated rule-based processing.
2Productivity
If automated rule-based matching is applied, then processing efficiency improves, but measurement precision may deteriorate due to over-matches and under-matches
Solution Approach 1:
The patent divides entity matching into multiple attribute groups (e.g., identifying attributes vs. contact attributes) and applies different matching rules to each group. This segmentation allows the system to process data efficiently through automated rules while maintaining precision by applying appropriate matching strictness to each attribute type.
Solution Approach 2:
The system applies multiple rules with different matching levels, including some that may produce partial matches or over-matches. By synthesizing results from multiple rule applications, the system achieves high overall accuracy while maintaining fast automated processing, as the excessive rule application is computationally efficient and can be parallelized.
3Measurement precision
If multiple rules with different attribute matching levels are applied, then entity resolution accuracy improves, but computational complexity increases
Solution Approach 1:
The patent segments attributes into different groups (identifying attributes like name and entity type vs. contact attributes like phone and email) and applies different matching rules to each group. This segmentation reduces computational complexity by avoiding unnecessary complex comparisons on all attributes while maintaining high accuracy through targeted rule application.
Solution Approach 2:
The system applies different matching strictness levels to different attribute groups based on their importance and reliability. Identifying attributes receive stricter matching rules while contact attributes receive more lenient rules. This local quality approach optimizes the balance between accuracy and computational complexity by allocating processing resources strategically.
Data Source
AI summary
Disclosed are a method, a system, and a non-transitory computer readable medium for identifying data entries associated with an entity. A first rule is applied on a plurality of data entries associated with a plurality of entities to obtain a first set of data entries from the plurality of data entries. Each data entry includes corresponding attributes. The first rule is to search for data entries with a first group of attributes. A second rule is applied on the plurality of data entries associated with a plurality of entities to obtain a second set of data entries from the plurality of data entries. The second rule is to search for data entries with a second group of attributes. A third set of data entries associated with a single entity is obtained based on the first set of data entries and the second set of data entries.