Entity Normalization via Bucketed Grouping and Adaptive Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for identifying duplicate objects in large collections are ineffective when objects have different types or include incomplete/inaccurate information, as they assume homogeneity and are not adaptable to diverse data sets.
Innovation Solution
A system and method that groups objects using a selected 'grouper' and compares them within buckets using a selected 'matcher', allowing for efficient identification of duplicates across different types and with incomplete/inaccurate information by utilizing a collection of groupers and matchers designed for such scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional approaches match specific facts (ISBN, SSN) to identify duplicate objects, then identification accuracy is improved for homogeneous object types, but the method becomes ineffective when objects have different types or incomplete information
Solution Approach 1:
The patent creates a universal duplicate identification system that can handle multiple object types (books, products, movies, people) through a single framework. The system uses type-agnostic grouping mechanisms and multiple matching strategies that adapt to different object characteristics, making the solution universally applicable rather than type-specific
Solution Approach 2:
The system dynamically changes matching parameters based on object type and data quality. Different matchers are selected and configured according to the specific object type being processed, allowing the system to optimize accuracy for each type while maintaining a unified architecture
2Reliability
If pair-wise matching is performed across all objects to identify duplicates, then comprehensive duplicate detection is achieved, but computational cost increases significantly
Solution Approach 1:
The patent segments the object collection into multiple buckets based on grouping criteria (object type, key attributes, or other characteristics). This segmentation reduces the problem from comparing all objects pairwise to comparing only objects within the same bucket, dramatically reducing computational complexity while maintaining detection completeness
Solution Approach 2:
The system introduces intermediate grouping structures (buckets) as mediators between the full object set and the final duplicate identification. These intermediate structures organize objects in a way that enables efficient pairwise comparison within groups while ensuring that potential duplicates are not missed
3Productivity
If traditional matching methods are applied to objects with incomplete or inaccurate information, then processing speed is maintained, but identification accuracy deteriorates
Solution Approach 1:
The system dynamically adapts its matching behavior based on data quality indicators. When incomplete or inaccurate information is detected, the system adjusts its matching strategy by selecting appropriate matchers that can handle such conditions, balancing speed and accuracy based on the specific data characteristics
Data Source
AI summary
A system and method identifying duplicate objects from a plurality of objects. The system and method groups similar objects into buckets based on a selected grouper, matches objects within the same bucket based on a selected matcher, and identifies the matching objects as duplicate objects.