Fuzzy Entity Matching via ML Blurring Operators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current fuzzy matching mechanisms in database systems require multiple field-specific matchers, leading to complexity, errors, and high maintenance costs, as they are imperfect and prone to false positives and negatives, especially when dealing with intricate field types like contact person records.
Innovation Solution
A single machine learning-based fuzzy matching mechanism using a statistical classifier that applies blurring operators specific to each field type, allowing for continuous updates based on prediction feedback to improve accuracy and precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple field-specific matchers are used for fuzzy matching, then matching accuracy for different field types is improved, but system complexity and maintenance costs increase
Solution Approach 1:
The patent applies a single universal fuzzy matching mechanism that can handle multiple field types (names, addresses, phone numbers, etc.) through a unified statistical classifier. This replaces the traditional approach of creating separate field-specific matchers, thereby reducing system complexity while maintaining the ability to accurately match different field types through a common framework that learns from diverse data patterns.
Solution Approach 2:
The system dynamically adjusts matching parameters and thresholds based on feedback from prediction results. The statistical classifier continuously learns from actual matching outcomes, modifying its internal parameters to optimize performance across different field types. This adaptive parameter adjustment allows a single matcher to achieve high accuracy without requiring manual configuration for each field type.
2Measurement precision
If field-specific matchers are used for each field type, then matching precision for specific fields is improved, but false positives and negatives increase due to imperfections
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors matching results and uses this information to retrain and refine the statistical classifier. By analyzing false positives and negatives from actual usage, the system automatically adjusts its matching criteria and improves its accuracy over time. This closed-loop feedback system ensures that the single universal matcher becomes increasingly reliable across all field types without accumulating the errors that plagues multiple specialized matchers.
3Adaptability or versatility
If multiple field matchers are maintained, then coverage for different field types is improved, but maintenance costs and errors increase
Solution Approach 1:
The patent creates a single universal fuzzy matching mechanism that can handle multiple field types through a unified statistical classifier. This replaces the traditional approach of creating separate field-specific matchers, thereby reducing maintenance costs and errors while maintaining comprehensive coverage across different field types such as names, addresses, phone numbers, and more.
Solution Approach 2:
The system performs self-maintenance through automated continuous learning from feedback data. The statistical classifier automatically updates its parameters and matching strategies based on actual usage patterns and correction feedback, eliminating the need for manual maintenance and reducing operational errors. This self-service capability allows the system to adapt to changing data patterns without requiring human intervention for each field type.
Data Source
AI summary
A method and system of matching field values of a field type are described. Blurring operations are applied on a first and second values to obtain blurred values. A first maximum score is determined from first scores for blurred values, where each one of the first scores is indicative of a confidence that a match of the first and the second values occurs with knowledge of a first blurred value. A second maximum score is determined from second scores for the blurred values, where each one of the second scores is indicative of a confidence that a non-match of the first and the second values occurs with knowledge of the first blurred value. Responsive to determining that the first maximum score is greater than the second maximum score, an indication that the first value matches the second value is output.