Transaction Data Matching via ML Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In modern computing, systems face challenges in matching transactional data from multiple sources due to differences in coding and timing, leading to redundant data storage, particularly in transactional data feeds like credit card transactions.
Innovation Solution
A computer-implemented method using a machine learning model to determine similarities between transactional data and second data, identifying matches above a threshold, and replacing values in the second data with corresponding values from the identified transactional data record, thereby reducing redundancy and storing related data in a single database record.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If data from multiple sources is stored as distinct records, then data redundancy is reduced, but data matching accuracy deteriorates due to coding differences
Solution Approach 1:
The system transforms the data matching problem by changing parameters from exact string matching to similarity scoring based on multiple factors (edit distance, substring matching, positional analysis). This allows the system to accommodate coding differences while still identifying related records, resolving the contradiction between reducing redundancy and maintaining matching accuracy
Solution Approach 2:
The patent introduces an intermediary matching service that acts as a mediator between data sources with different coding schemes. This service computes similarity scores and determines matches without requiring changes to the source systems, enabling accurate matching despite coding variations while preventing redundant storage
2Productivity
If traditional exact matching methods are used, then data processing speed is maintained, but matching accuracy deteriorates due to coded data variations
Solution Approach 1:
The system performs partial matching by computing similarity scores based on selected fields rather than requiring complete exact matches across all fields. This approach processes data efficiently while accommodating variations in coding, achieving both speed and accuracy by focusing on key discriminative fields
Solution Approach 2:
The matching mechanism changes from binary exact matching to a multi-parameter similarity assessment that considers edit distance, common substrings, and positional relationships. This parameter transformation enables the system to handle coded variations while maintaining processing efficiency through configurable similarity thresholds
3Speed
If similar data is stored redundantly across multiple records, then data access speed is improved, but data storage efficiency deteriorates
Solution Approach 1:
The system merges similar transactional data into single consolidated records by identifying matches through similarity computation. Related data from multiple sources is combined into one record with standardized fields, eliminating redundancy while maintaining fast access through efficient indexing and retrieval of the unified record
Data Source
AI summary
Embodiments of the present disclosure pertain to matching transactional data. In one embodiment, the present disclosure includes a computer implemented method comprising receiving transactional data for a first user and second data for the first user, selecting transactional data records for the first user from a data store of transactional data for a plurality of users, determining a plurality of similarities between fields of the transactional data and second data, determining a likelihood of a match between a transactional data field and a second data field based on the plurality of similarities using a machine learning model, and in accordance therewith, identifying one record in the transactional data records for the first user that generates said likelihood of the match above a first threshold, and replacing values second data fields with corresponding values in the one record.