Merchant Data Cleansing via Probabilistic Matching Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing payment network systems face challenges in accurately aggregating and cleaning merchant data due to noisy and ambiguous information, leading to inefficiencies in data analytics and fraud detection.
Innovation Solution
A computer-implemented method using probabilistic matching models and data models like phone-to-city, payment aggregator, and merchant name normalization models to identify and correct ambiguous merchant data fields in electronic payment transaction records, ensuring accurate matching and aggregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rule-based systems are used for merchant data aggregation, then manual control and interpretability are maintained, but data accuracy and automation efficiency deteriorate due to noisy and ambiguous merchant information
Solution Approach 1:
The patent replaces rule-based mechanical systems with machine learning models (probabilistic matching models, phone-to-city models, payment aggregator models) that automatically learn patterns from data. This substitution enables the system to handle noisy and ambiguous merchant information more effectively, improving data accuracy while reducing the need for manual rule configuration and interpretation.
Solution Approach 2:
The system implements self-service through automated machine learning models that independently process and clean merchant data without requiring manual intervention. The probabilistic matching models and specialized data models automatically identify and correct issues in merchant information, enabling the system to serve itself in data cleansing tasks while maintaining high accuracy.
2Productivity
If manual processes are used for data cleansing, then control over data quality is maintained, but processing speed and productivity deteriorate
Solution Approach 1:
The patent replaces manual data cleansing processes with automated machine learning-based systems. The probabilistic matching models and specialized models (phone-to-city, payment aggregator) automatically process transaction records at high speed while maintaining data quality through learned patterns and probabilistic reasoning, eliminating the speed limitation of manual processes.
Solution Approach 2:
The system implements feedback mechanisms where the machine learning models continuously learn from processed data and improve their performance. The probabilistic matching models use feedback from matching outcomes to refine their predictions, and the system incorporates human-in-the-loop feedback for edge cases, ensuring both high processing speed and maintained data quality control.
3Reliability
If third-party data and hand-written rules are used for merchant aggregation, then coverage is extended, but false positives increase due to optimization problems with unconventional merchant names
Solution Approach 1:
The patent replaces hand-written rules with machine learning models that are trained on diverse merchant data including unconventional names with emoticons, pictures, and special characters. The probabilistic matching models and specialized models learn to handle these variations automatically, improving reliability by reducing false positives while maintaining adaptability to unconventional merchant names through pattern recognition rather than rigid rule matching.
Data Source
AI summary
Embodiments provide methods and systems for merchant data cleansing in payment network. Method performed by server system includes accessing electronic payment transaction records from transaction database. Each electronic payment transaction record includes merchant data fields. Method includes determining set of electronic payment transaction records with ambiguous merchant data fields having matching probability scores less than predetermined threshold value computed by probabilistic matching model and identifying at least one issue for non-matching of each of set of electronic payment transaction records. Method includes determining data model based on at least one issue of each of set of electronic payment transaction records. Data model is one of: phone-to-city model, payment aggregator model, and merchant name normalization model. Method includes updating set of electronic payment transaction records with unambiguous merchant data fields corresponding to ambiguous merchant data fields by applying data model to each of set of electronic payment transaction records.