Graphical Match Policy for Database Duplicate Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storing and maintaining duplicate records in databases wastes valuable computing resources, bandwidth, and storage space, and can cause confusion when returning results, as existing systems require extensive processing and are often complex to manage.
Innovation Solution
A matching system that uses graphical match policies with candidate and comparison filters to identify and consolidate duplicate records, employing visual interfaces for users to configure match rules and prioritize resource efficiency by filtering records before comparison.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If duplicate records are stored in the database, then more data is available for querying, but memory and disk storage space are consumed
Solution Approach 1:
The patent extracts and removes duplicate records from the database through automated detection and consolidation processes. The system identifies redundant data entries and eliminates them, keeping only unique records, thereby reducing storage space consumption while preserving the essential data volume needed for querying.
Solution Approach 2:
The patent applies the principle of discarding duplicate records that consume storage space while recovering the value of having clean, unique data. The system detects duplicates, removes them from storage, and maintains data integrity through consolidation, effectively discarding redundant information while preserving useful data.
2Quantity of substance
If duplicate records are stored in the database, then more data is available for querying, but processing resources are consumed
Solution Approach 1:
The patent extracts duplicate records from the database through automated detection mechanisms that identify redundant entries. By removing these duplicates before querying operations, the system reduces the number of records that need to be processed, thereby lowering CPU usage and processing resource consumption while maintaining access to all unique data.
Solution Approach 2:
The patent performs preliminary detection and removal of duplicate records before querying operations occur. By pre-processing the database to eliminate duplicates in advance, the system reduces the processing burden during actual query operations, improving efficiency without sacrificing data availability.
3Quantity of substance
If duplicate records are stored in the database, then more data is available for querying, but disk storage space is consumed
Solution Approach 1:
The patent discards duplicate records that waste storage space while recovering the benefit of having optimized database size. The system identifies redundant entries, removes them from disk storage, and consolidates data, effectively eliminating storage space waste associated with duplicate records.
4Reliability
If duplicate records are returned by queries, then complete results are provided, but user confusion is caused
Solution Approach 1:
The patent extracts and removes duplicate records from query results through automated detection and consolidation. By eliminating redundant entries before returning results to users, the system provides complete and accurate data while improving ease of operation by presenting clear, non-repetitive results that are easier for users to understand and interpret.
5Measurement precision
If extensive processing is performed to identify duplicates, then duplicate detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent implements self-service mechanisms where the database system automatically detects and removes duplicates through built-in procedures and triggers. This automated approach maintains high detection accuracy while reducing system complexity by eliminating the need for complex external detection systems, allowing the database to manage its own data quality.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for providing a graphical match policy for identifying duplicative data. An embodiment operates by receiving a selection of a match rule for identifying duplicate records within a database, the match rule comprising a candidate filter and a comparison filter. One or more candidate attributes of the candidate filter and one or more comparison attributes of the comparison filter are determined. A first subset of the records within the database that satisfy the candidate filter are identified. A second subset of the records from the first subset of records that satisfy the comparison filter are identified. The second subset of records that satisfy both the candidate filter and the comparison filter are returned.