Database Record Matching Using Confidence Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems face challenges in automatically resolving conflicts during record merges, particularly when new records overlap with existing information, leading to burdensome manual resolution for users.
Innovation Solution
A method and apparatus that utilize artificial intelligence and fuzzy logic to determine matches with minor mistakes by comparing tax identification, company name, address, city, and phone fields, assigning confidence scores, and verifying fields through a field verification service to automatically combine records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional database software flags overlaps for manual resolution, then conflict detection is achieved, but user burden increases significantly due to large numbers of conflicts requiring manual resolution
Solution Approach 1:
The system performs self-service by automatically resolving conflicts using confidence scores and verification services without requiring user intervention. The database server independently determines matches, verifies fields, and combines records, eliminating the need for users to manually review and resolve each conflict.
Solution Approach 2:
The manual mechanical process of user review and conflict resolution is replaced with an automated electronic system using confidence score calculations, field verification services, and algorithmic decision-making to determine record matches and resolve conflicts automatically.
2Extent of automation
If Oracle rule sets are used to manage conflicts, then some automation is achieved, but records are not automatically combined where conflicting information exists
Solution Approach 1:
The system automatically combines records by having the database server perform self-service operations: calculating confidence scores, verifying fields through verification services, and executing record combination without requiring user approval or intervention at each step.
Solution Approach 2:
The system performs preliminary actions by pre-calculating confidence scores for multiple fields (tax identification, company name, address, city, phone) and pre-verifying fields through verification services before finalizing the record combination, enabling automatic resolution without user involvement.
3Measurement precision
If exact matching is required for all fields, then data accuracy is maintained, but matches with minor mistakes are missed
Solution Approach 1:
The system applies different quality standards to different fields by calculating separate confidence scores for each field (tax identification, company name, address, city, phone) and combining them. This allows minor discrepancies in less critical fields while maintaining high accuracy requirements for critical fields like tax identification.
Solution Approach 2:
The system changes the matching parameter from binary exact-match to a continuous confidence score that can accommodate minor variations. By using confidence scores and verification services, the system can tolerate minor mistakes while still maintaining high data quality standards through the averaging mechanism and threshold comparisons.
Data Source
AI summary
A novel technique for matching, merging, and combining a new database record with a master database record is described herein. The technique uses specific fields to compare in a unique exact match and fuzzy match combination to determine if a non-exact matching record pair is indeed a matched pair. Once the match is established, the records are merged or combined to utilize the best information from the two records.