Context-Weighted N-Gram Similarity for Merchant Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing string similarity metrics do not adequately account for the contextual importance of specific words within strings, leading to inaccurate similarity scores, as seen in comparing company names like "JOE RESTAURANT" and "JON RESTAURANT".
Innovation Solution
A system and method that generate similarity measures by using context-weighted words, where the system processes transaction data to identify contexts such as industry and location, calculates word frequencies, assigns weights based on frequency, and applies these weights to string similarity algorithms to improve matching accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional string similarity metrics are used to compare company names, then the matching process is simple and fast, but the similarity score is artificially inflated and inaccurate due to frequent common words like 'RESTAURANT'
Solution Approach 1:
The patent applies local quality by assigning different weights to different words within the same string based on their contextual importance. Specifically, common words like 'RESTAURANT' that appear frequently in the food industry are weighted less, while unique identifying words like 'JOE' or 'JON' are weighted more heavily. This localized differentiation of word importance within the string resolves the contradiction by maintaining simple overall system structure while improving measurement precision through selective word weighting.
Solution Approach 2:
The patent changes the parameter of word weight from a uniform value to a context-dependent variable. By calculating word frequencies within specific contexts (industry, location, timing) and dynamically adjusting weights based on these frequencies, the system transforms the static similarity calculation into a dynamic one that adapts to contextual information, thereby improving accuracy without requiring fundamentally complex system architecture.
2Measurement precision
If context-weighted word frequencies are calculated and applied to string similarity algorithms, then matching accuracy improves by emphasizing infrequent words, but the processing time and computational complexity increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing word frequencies within specific contexts before performing similarity comparisons. The system builds a contextual vocabulary that captures word frequencies across different industries, locations, and time periods in advance. This pre-computation allows the system to quickly retrieve and apply pre-determined weights during transaction processing, significantly reducing real-time processing time while maintaining high matching accuracy.
Solution Approach 2:
The patent extracts only the essential contextual information needed for weighting - specifically word frequencies within defined contexts - and separates this from the full transaction data. By extracting and storing only these critical frequency statistics, the system avoids processing entire transaction histories during matching operations, thereby reducing computational overhead and processing time while preserving the ability to perform accurate context-weighted comparisons.
Data Source
AI summary
An embodiment of the present invention is directed to generating an n-gram-based string similarity score where the n-grams sourcing from frequent words, given the context of interest, may be under-weighted and the n-grams sourcing from infrequent words, given the context of interest, may be over-weighted. An embodiment of the present invention is directed to defining a weight for each 2-gram depending on whether or not the word from which it sourced was frequent based on the relevant context.