Extraction-Based Transaction Categorization With Self-Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transaction categorization technologies fail to accurately categorize transactions for large groups of users due to inconsistent data formats among entities, frequent format changes, and the need for extensive manual labor in training data labeling, especially for new entities lacking historical data.
Innovation Solution
A method using a machine learning model to classify transaction fields, apply mapping rules, and score candidate mappings to automatically categorize transactions without requiring comprehensive training data sets for each entity, adapting to format changes by detecting and correcting errors through scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional supervised learning techniques are used to train a machine learning model to categorize transactions involving each entity, then the model can achieve accurate categorization for that entity, but the process requires significant manual labor, user feedback, data processing, and computational overhead to label training data for thousands of entities
Solution Approach 1:
The system performs self-training by automatically generating training data from unlabeled transaction data. The machine learning model iteratively improves its own categorization capabilities without requiring manual labeling of training data for each entity, thus eliminating the significant manual labor and computational overhead while maintaining categorization accuracy.
Solution Approach 2:
The system performs preliminary categorization using the machine learning model on unlabeled transaction data before any manual review. This preliminary action generates candidate training data that can be automatically validated and used to train the model, reducing the need for extensive manual data preparation and labeling efforts.
2Measurement precision
If a machine learning model is trained on historical transaction data for an entity, then it can accurately categorize transactions for that entity, but it fails to accurately categorize transactions when the entity changes its data format or when dealing with newer entities lacking historical data
Solution Approach 1:
The system dynamically adapts to format changes by continuously learning from new transaction data. The machine learning model is designed to handle varying data formats and structures, automatically adjusting its categorization logic when presented with new formats from existing entities or data from newer entities, thus maintaining accuracy despite format changes.
Solution Approach 2:
The system employs a universal machine learning model that can handle multiple entity types and data formats simultaneously. Rather than requiring separate trained models for each entity, the universal model learns patterns across diverse entities and adapts to new formats and entities without retraining, enabling it to categorize transactions for both established and newer entities accurately.
3Measurement precision
If manual labeling of training data is performed for each entity to ensure accurate categorization, then the model can learn entity-specific patterns, but the process becomes impractical for thousands of entities due to the significant amount of manual labor and computational overhead required
Solution Approach 1:
The system eliminates manual labeling by enabling the machine learning model to learn entity-specific patterns automatically from unlabeled transaction data. The model self-trains by identifying patterns in the data it processes, thus achieving entity-specific categorization accuracy without the manual labor and computational overhead that would limit processing throughput for thousands of entities.
Solution Approach 2:
The system extracts entity-specific patterns and characteristics directly from the transaction data itself, rather than requiring manually labeled training data. By extracting these patterns automatically during the categorization process, the system maintains entity-specific accuracy while dramatically increasing processing throughput for large numbers of entities.
Data Source
AI summary
Aspects of the present disclosure relate to automated transaction categorization. Embodiments include receiving data associated with transactions involving multiple users and an entity; extracting fields from the data by creating an embedding representation of the data; processing the extracted fields through multiple layers of a trained neural network model to assign a class to the fields; generating candidate mappings for the transactions by applying multiple sets of mapping rules to the extracted fields; generating a score for each candidate mapping by applying scoring rules to the candidate mappings; selecting a set of mapping rules for categorizing transactions involving the entity based on the generated score for a corresponding candidate mapping of the candidate mappings; and creating mappings of transactions associated with a particular user based on applying the selected set of mapping rules to each transaction associated with the particular user and the entity.