Cross-Database Entry Matching Using Ranked Token Association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of linking or associating data entries in different databases, such as an expense report and a travel itinerary, where entries are entered at different times and formats can vary, leading to inefficiencies in data matching and association.
Innovation Solution
A system and method that converts character fields into tokens, weights them based on statistical significance, and compares these tokens to determine matches, allowing for the association of data entries across databases by modifying entries to align formats and using a match-detection engine to link corresponding entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data entries from different databases are linked using traditional matching methods, then the association accuracy may be sufficient for simple cases, but the system becomes inefficient and unable to handle large volumes of unassociated entries entered at different times
Solution Approach 1:
The patent segments the data association process into distinct phases: tokenization of character fields into discrete units, ranking of tokens by significance, and multi-stage matching processes. This segmentation allows the system to efficiently process large volumes of data while maintaining accuracy by focusing computational resources on the most discriminative features first.
Solution Approach 2:
The patent transforms the data representation by converting character fields into ranked tokens, changing the parameter space from raw text to structured token sequences. This parameter transformation enables more efficient comparison and matching operations while preserving the semantic information needed for accurate association.
2Loss of information
If the system processes and associates all data entries across databases, then the data integration improves, but the computational resources and processing time increase significantly
Solution Approach 1:
The patent performs preliminary tokenization and ranking of data entries before the actual matching process. By pre-processing the data into standardized token formats and pre-ranking tokens by their discriminative value, the system reduces the computational burden during the association phase, enabling faster processing while maintaining complete data integration.
Solution Approach 2:
The patent implements a multi-stage matching process that focuses first on the most discriminative tokens and progressively considers less significant tokens only when needed. This partial action approach allows the system to achieve high-confidence matches quickly while still having the capability to process additional tokens if complete integration is required, thus balancing processing time with integration completeness.
Data Source
AI summary
In one embodiment, a first entry in a first database is modified to include data from a highest-ranked one of one or more available data tables that correspond to the first entry. Each of one or more characters fields of the modified first entry are converted into a respective one or more first-entry tokens, and each of one or more character fields of each of a plurality of second entries in a second database is converted into a respective one or more second-entry tokens. The first-entry tokens are compared to the second-entry tokens, and, in response to the comparison, it is determined whether the first entry matches one of the second entries. In response to determining that the first entry matches one of the second entries, the first entry and the matching second entry are associated with one another in one or both the first and second databases.