String Matching via Hash-Based Blocking Groups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational complexity of matching strings in entity resolution tasks, such as data integration and record linkage, due to the need for extensive string comparisons, limits efficiency and scalability, especially when dealing with large datasets and diverse data sources.
Innovation Solution
A computer-implemented method using a string hash function to divide input character strings into n-grams, calculate hash values, and aggregate them into integer values for grouping, significantly reducing computational complexity by pre-sorting strings into blocking groups, allowing for efficient matching using integer comparisons rather than string comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional string comparison methods are used for matching, then matching accuracy is maintained, but computational complexity increases significantly
Solution Approach 1:
The patent introduces hash values as an intermediary representation of strings. Instead of directly comparing strings, the system computes hash values from strings and compares these compact integer representations. This intermediary step reduces the computational burden while maintaining matching accuracy, as hash values preserve the essential characteristics needed for comparison.
Solution Approach 2:
The patent creates a simplified copy of the string data in the form of hash values. Rather than manipulating and comparing the full string data, the system works with these compact hash value copies, which contain the necessary information for matching but require significantly less computational resources to process.
2Reliability
If all string pairs are compared exhaustively, then complete matching is achieved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by computing hash values for all strings before performing comparisons. This pre-processing step organizes the data into a more efficient structure, allowing the system to quickly identify potential matches without exhaustively comparing every string pair, thus reducing processing time while maintaining completeness.
Solution Approach 2:
The patent segments the comparison task by dividing it into two stages: first comparing hash values (which is computationally efficient), and then performing full string comparison only on pairs with matching or similar hash values. This segmentation reduces the total number of expensive full-string comparisons needed while ensuring no matches are missed.
3Loss of information
If string length increases, then more information is captured, but comparison complexity increases
Solution Approach 1:
The patent extracts the essential information from long strings by computing hash values that capture the key characteristics needed for matching. This extraction process separates the essential matching information from the redundant or less relevant parts of the strings, allowing efficient comparison while preserving the necessary information content.
Data Source
AI summary
A method for matching first elements with second elements. Each of the first elements and second elements is a character string. The method comprises: calculating a first integer hash value for each of the first elements using a string hash function, wherein the first integer hash value is an output integer calculated from using each of the first elements as an input character string of the function; calculating second integer hash values for each of the second elements using the function; grouping each of the first elements into at least one group of a set of blocking groups using its first integer hash value; grouping each of the second elements into at the least one group of the set of blocking groups using its second integer hash value; and matching first elements with second elements within each group of the set of blocking groups using a string comparison function.