Graph-Based Record Matching for Fragmented Database Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database search methods often return incomplete results due to variations in identifier representation and mistyped data, leading to inaccurate matching and fragmented files, as they fail to accurately determine if new data already exists in the database, resulting in duplicate records.
Innovation Solution
A record-matching system that generates identifier scores and compound scores to determine matching decisions, using a graph-based approach to identify connected components and merge candidate records, thereby updating the database to eliminate duplicates and detect fragmented files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional exact matching methods are used for database searches, then the search process is simple and fast, but the matching accuracy is low and fragmented files are generated
Solution Approach 1:
The patent introduces an intermediary record-matching system between the query and the database records. This intermediary layer performs fuzzy matching, generates matching scores, and determines matching decisions, thereby improving accuracy without requiring changes to the underlying database structure or query mechanism.
Solution Approach 2:
The patent replaces the mechanical exact-matching mechanism with a probabilistic scoring system. Instead of simple equality comparison, the system generates matching scores based on multiple criteria and uses these scores to determine matches, enabling handling of variations and errors in identifier representation.
2Reliability
If traditional database search methods are used, then the database structure remains simple, but duplicate records are created and storage efficiency decreases
Solution Approach 1:
The patent performs preliminary matching before inserting new records into the database. By querying existing records and determining matching decisions in advance, the system prevents duplicate records from being created, thereby maintaining data integrity and avoiding unnecessary storage of redundant data.
Solution Approach 2:
The system implements a feedback mechanism where matching results from the record-matching system are used to control whether new records are inserted into the database. If a match is determined, the insertion is prevented; if no match is found, the record is inserted. This feedback loop ensures data integrity while optimizing storage efficiency.
Data Source
AI summary
In some aspects, a record-matching computing system for detecting fragmented records is provided. The record-matching system is configured to identify a list of candidate records for merging from a set of data records. The record-matching system determines a matching decision for each pair of candidate records in the list and generates a graph. The graph includes nodes representing respective candidate records and edges connecting the nodes. Each edge represents a match between a pair of nodes connected by the edge according to the matching decisions. The record-matching system detects a connected component in the graph from which a qualified connected component is identified based on the minimum connectivity of the qualified connected component. The record-matching system updates the set of data records stored by merging candidate records represented by the nodes in the qualified connected component.