Multi-Engine Duplicate Detection via Unified External Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for identifying and removing duplicate customer data in complex data systems lack confidence and efficiency, often requiring multiple passes and additional analysis to accurately determine similar data sets, leading to hesitation and confusion in business decision-making.
Innovation Solution
A method and system that utilize a plurality of matching engines to create suspect tables and determine weights and scores, with external logic criteria for comparative assessment and ordered priority, allowing for the identification of suspect candidates in a single pass with heightened confidence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple passes and additional analysis are conducted to increase confidence in duplicate identification, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent combines multiple matching engines into a unified framework where results from different engines are aggregated and evaluated together. By merging the outputs of multiple matching engines and using a single external logic to assess all results, the system achieves high confidence duplicate identification in one pass rather than requiring multiple sequential passes, thus resolving the contradiction between precision and time loss.
Solution Approach 2:
The external logic serves multiple functions simultaneously: it evaluates results from multiple matching engines, applies business rules, determines suspect candidate status, and produces final decisions. This multi-functional approach eliminates the need for separate analysis passes, improving both confidence in identification and processing efficiency.
2Measurement precision
If additional matching engines and analysis tools are used to improve accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The external logic acts as an intermediary layer that receives results from multiple matching engines and consolidates them into a unified assessment. This mediator approach allows the system to incorporate multiple engines for improved accuracy while managing complexity through a single coordinating logic that handles all evaluations systematically.
Solution Approach 2:
The system segments the duplicate detection process into distinct components: multiple matching engines for data matching, an external logic for evaluation and decision-making, and a suspect table for storing results. This segmentation allows each component to be optimized independently while working together, improving accuracy without overwhelming complexity.
3Device complexity
If traditional single matching engine approach is used, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent merges multiple matching engines into a single coordinated evaluation framework where all engine results are assessed by a common external logic. This combination approach maintains relative simplicity while significantly improving measurement precision through the aggregated insights from multiple engines, resolving the contradiction between simplicity and precision.
Data Source
AI summary
The present invention in various implementations provides for a method and system for removing suspect duplicate data in a database having a plurality of datasets for a suspect processing transaction, using a plurality of matching engines, comparing results of matching engines in a logically predetermined comparative assessment, and thereafter providing an ordered priority of results of matching engines to identify suspect candidates.