Token Set Data Matching for Faster Record Reconciliation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data matching and reconciliation processes are inefficient, computationally expensive, and expose sensitive information due to the complexity of handling multiple incomplete records and inconsistent data, leading to integrity issues and potential data breaches.
Innovation Solution
A data matching process that tokenizes records into 'entity tokens' or 'token sets', allowing for accelerated matching by aggregating related token record pairs, using one-way hashing and encryption to protect sensitive information, and utilizing a Consolidation Platform for analysis and consolidation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional data matching techniques are used to reconcile multiple incomplete records, then data completeness can be improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent segments the data matching process into distinct phases: tokenization of individual records, creation of token pairs from record pairs, aggregation of token pairs into token sets, and matching of token sets. This segmentation allows each phase to be optimized independently, reducing overall processing time while maintaining data completeness.
Solution Approach 2:
The patent applies preliminary tokenization to transform raw data into standardized token representations before matching. This preliminary action creates a consistent foundation that accelerates subsequent matching operations and enables efficient aggregation of related tokens into sets, significantly reducing the time required for reconciliation.
2Loss of information
If traditional data matching techniques are used to reconcile multiple incomplete records, then data completeness can be improved, but computational complexity increases
Solution Approach 1:
The patent merges multiple token pairs that share common tokens into unified token sets. By combining related token pairs based on overlapping tokens, the system reduces the total number of comparisons needed and simplifies the matching process, thereby reducing computational complexity while achieving complete data reconciliation.
Solution Approach 2:
The patent creates token representations as simplified copies of the original data. These tokens serve as lightweight proxies that capture essential information without requiring processing of the full complex records, significantly reducing computational complexity during matching operations.
3Reliability
If manual record combination is used to reconcile incomplete records, then data integrity can be maintained, but processing efficiency decreases
Solution Approach 1:
The patent introduces tokens as intermediary representations between the original data and the matching process. These tokens maintain data integrity by preserving essential information in a standardized format, while enabling automated processing that dramatically improves productivity compared to manual combination methods.
4Measurement precision
If detailed attribute-level matching is performed to ensure accurate data reconciliation, then measurement precision improves, but processing time increases
Solution Approach 1:
The patent segments matching into hierarchical levels: first matching token sets at a coarse level, then drilling down to attribute-level comparisons only for promising candidates. This segmentation maintains high matching accuracy while reducing overall processing time by avoiding exhaustive detailed comparisons of all record pairs.
Solution Approach 2:
The patent applies partial detailed matching by performing full attribute-level comparison only on token set pairs that show initial similarity. For most records, a lighter-weight token-level comparison suffices, providing sufficient accuracy while dramatically reducing processing time through selective application of detailed analysis.
Data Source
AI summary
The present disclosure is directed to systems and methods for performant data matching. Entities maintain large amounts of data and desire to reconcile duplicative records. One way to solve this problem is through data matching. However, standard data matching at the record level can be laborious and inefficient. To remedy these inefficiencies in data matching, the present disclosure describes a system where the token records are tokenized a second time into token sets based on the token records satisfying at least one token set rule. A token set rule may be based on the common presence of multiple tokens in a token record. If multiple token records have the required tokens from the set rule, then those token records can be hashed and rolled-up into the token set (i.e., tokenized a second time into the token set). The token set allows for more efficient data matching.