Random Forest Rule Generation to Prevent Merchant Rule Overfitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing decision tree algorithms for creating transaction rules overfit on individual entities with limited historical data, limiting their applicability to similar entities and requiring manual expert intervention or ad-hoc rules with low accuracy.
Innovation Solution
A random forest machine learning algorithm is used to identify and generate rules by calculating feature importance, iteratively varying hyperparameters, and generating rules with quantiles to ensure applicability across similar entities, reducing overfitting and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If decision trees are used to generate rules for individual merchants, then individualized rules can be created, but the rules become too specific and overfit the data, limiting applicability to similar entities
Solution Approach 1:
The patent segments the rule generation process by introducing intermediary representations (decision rules, pattern rules, and generalized rules) that bridge individual merchant data and similar entity groups. Instead of directly creating overly-specific rules, the system breaks down the process into hierarchical levels of abstraction, allowing rules to be tailored to individual merchants while maintaining compatibility with broader patterns observed across similar entities.
Solution Approach 2:
The system dynamically adjusts rule parameters based on data availability and entity similarity. When historical data is limited for individual merchants, the system modifies rule parameters by incorporating patterns from similar entities, effectively changing the degree of individualization versus generalization. This parameter adjustment prevents overfitting while maintaining adaptability to individual merchant needs.
2Reliability
If individualized rules are created for each merchant, then fraud detection accuracy improves, but the complexity of rule creation and scaling to thousands of merchants increases significantly
Solution Approach 1:
The patent implements a universal rule generation framework that serves multiple functions simultaneously: it can generate rules for individual merchants, generalize patterns across groups of similar merchants, and scale to thousands of entities. The system's multi-functionality allows a single platform to handle diverse rule creation needs without proportionally increasing complexity, as the same core algorithms serve both individual and collective rule generation.
Solution Approach 2:
The system efficiently copies and adapts successful rule patterns from one merchant or group of merchants to similar entities. Instead of creating rules from scratch for each merchant, the system identifies effective patterns in historical data and replicates them across similar merchants, significantly reducing the complexity of rule creation while maintaining fraud detection accuracy across large numbers of entities.
3Measurement precision
If domain experts manually create individualized rules by analyzing historical data, then high accuracy rules can be generated, but the process is time-consuming and difficult to scale
Solution Approach 1:
The patent implements self-service rule generation where the system automatically analyzes historical data and creates optimized rules without requiring manual domain expert intervention for each merchant. The automated system performs data analysis, pattern recognition, and rule generation tasks that would otherwise require expert time, thereby maintaining high rule accuracy while dramatically reducing the time and resources needed for rule creation across thousands of merchants.
Solution Approach 2:
The system replaces the mechanical process of manual domain expert analysis with automated machine learning algorithms. Instead of experts manually examining historical data and deriving patterns, the system uses computational algorithms to automatically identify patterns and generate rules, substituting human cognitive work with automated processing that maintains or improves accuracy while reducing time investment.
4Ease of operation
If ad-hoc rules are created by users with minimal historical data, then rule creation is simple, but the accuracy rate decreases
Solution Approach 1:
The patent introduces an intermediary automated system that mediates between user simplicity and data accuracy. Users can easily initiate rule creation with minimal input, and the intermediary system automatically bridges the gap by analyzing available historical data and generating accurate rules. This intermediary layer maintains ease of operation for users while ensuring high accuracy rates through automated data-driven rule generation, eliminating the need for users to manually create ad-hoc rules.
Data Source
AI summary
In an example embodiment, a random forest machine learning algorithm is used to create and/or identify rules to apply to an individual entity in a computer system that has a plurality of entities, each with a number of rules. More precisely, rule predicates are used as features of a random forest model built to predict a particular outcome (e.g., a transaction that is fraudulent). Hyperparameters of the random forest model are varied and iterated. A classifier is used to calculate feature importance for all features in the training data. Feature importance may be calculated using permutation feature importance. The N “most important” features are then found from this set. The N “most important” features are then used to find rules above a certain precision and recall rate. These rules may then be backtested and the best rules can be used to generate additional rules.