Bulk Record Matching Update Logic for Database Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current record matching systems in relational databases require re-performing matching operations to update existing Single Best Records (SBRs) with new data, which is inefficient and time-consuming.
Innovation Solution
A mechanism is introduced that uses an update manager logic and SBR generation logic to update existing SBRs in the master index database by matching new records with stored SBRs without re-performing the original matching operations, utilizing an update manager logic to associate and update SBRs based on match field values, and creating new SBRs for unmatched records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If re-performing matching operations is used to update existing SBRs with new data, then data completeness is improved, but processing time and system efficiency deteriorate
Solution Approach 1:
The system performs preliminary actions by maintaining a master index database with pre-computed SBRs and their associated match weights. When new records arrive, the system compares them against existing SBRs using pre-established matching criteria and weight thresholds, rather than re-performing complete matching operations. This preliminary structuring of data and matching rules enables efficient updates without full re-matching.
Solution Approach 2:
The matching process is segmented into distinct phases: (1) comparison of new records against existing SBRs in the master index, (2) calculation of match weights for specific fields, (3) evaluation against thresholds to determine matches, and (4) selective updates only for matched records. This segmentation allows the system to process only relevant portions of data rather than re-matching everything.
2Measurement precision
If re-performing matching operations is used to update existing SBRs, then data accuracy is improved, but system productivity deteriorates
Solution Approach 1:
The system applies partial action by performing matching operations only on the extent necessary - comparing new records against existing SBRs and updating only those records that meet the matching criteria. The match weight calculation focuses on specific fields rather than exhaustive comparison, and updates are applied selectively based on threshold evaluations, thereby maintaining accuracy while improving productivity.
3Adaptability or versatility
If complete block record matching is performed by each slave computing device, then matching independence is improved, but processing time for updates deteriorates
Solution Approach 1:
The master computing device performs preliminary actions by maintaining the master index database with existing SBRs and distributing only new records to slave devices. Slave devices independently compare these new records against the pre-existing SBRs using pre-established matching criteria, achieving independence without requiring complete block matching. This preliminary preparation enables faster update processing.
Data Source
AI summary
Systems, methods, and other embodiments associated with bulk matching with update functionality are described. In one embodiment, a method includes receiving a first new record for matching with other records. The method includes querying a database that stores respective single best records (SBR) for respective sets of matching records to return SBR information. The first new record is matched to the other records by comparing the first new record to the SBR information to identify an SBR that matches the first new record. The first new record is associated with the identified SBR in the database.