ML Rule Legibility via Segmentation and Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Human-written fraud detection rules are often slow to identify fraud patterns and lack transparency, while machine learning-based solutions generate rules that are difficult for humans to understand, complicating fraud detection in financial transactions.
Innovation Solution
A machine learning classifier is created to identify candidate classification rules that may be impossible for humans to identify, and a selection process is used to present only effective and understandable rules to users, combining the advantages of both human-written and machine learning-based approaches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If human analysts write fraud detection rules, then the rules are easy to understand and have identifiable purpose, but fraud patterns are identified later than desirable and may be overlooked entirely
Solution Approach 1:
The system segments the rule generation process into two distinct phases: (1) a machine learning component that rapidly generates candidate rules from transaction data, and (2) a selection component that filters these candidates based on legibility criteria. This segmentation allows each component to optimize for its strength - speed for ML, understandability for selection - while collectively solving both aspects of the contradiction.
Solution Approach 2:
The patent introduces an intermediary selection process that acts as a mediator between the machine learning rule generator and the final fraud detection system. This intermediary filters and evaluates ML-generated rules, selecting only those that meet legibility thresholds while discarding overly complex patterns. This intermediary enables the system to leverage ML speed without sacrificing human understandability.
2Productivity
If machine learning techniques are used to generate fraud detection rules, then fraud patterns can be identified earlier and more effectively, but the generated rules are difficult for humans to understand
Solution Approach 1:
The system implements feedback loops where ML-generated rules are evaluated against legibility criteria, and this evaluation feedback is used to refine the selection process. The feedback mechanism allows the system to learn which ML-generated rules are both effective at detecting fraud and understandable by humans, progressively improving the balance between speed and legibility.
Solution Approach 2:
The patent changes the parameters of rule generation by adjusting the complexity thresholds and legibility criteria. By dynamically modifying these parameters, the system can control the trade-off between generating highly effective ML rules and maintaining human understandability, allowing flexible optimization based on specific operational needs.
3Reliability
If complex machine learning models are used to maximize fraud detection accuracy, then detection effectiveness improves, but the models become black boxes that cannot be understood by humans
Solution Approach 1:
The system extracts only the essential, understandable components from complex ML models by selecting and presenting simplified rule representations to human analysts. Instead of exposing the full complexity of the ML model, the system extracts interpretable rule patterns that capture the core fraud detection logic, maintaining accuracy while eliminating the black box problem.
Solution Approach 2:
The patent creates a composite approach combining ML-generated rules with human-understandable rule formats. This composite system leverages the pattern recognition power of ML while presenting results in a structured, interpretable format that humans can understand and validate, effectively merging the strengths of both approaches.
Data Source
AI summary
Pure machine learning classification approaches can result in a “black box” solution where it is impossible to understand why a classifier reached a decision. This disclosure describes generating new classification rules leveraging machine learning techniques. New rules may have to meet evaluation criteria. Legibility of those rules can be improved for understanding. A machine learning classifier can be created that is used to identify possible candidate classification rules (e.g. from a group of decision trees such as a random forest classifier). Classification rules generated with the assistance of machine learning may allow for identification of transaction fraud or other classifications that a human analyst would be unable to identify. A selection process can identify which possible candidate rules are effective. The legibility of those rules can then be improved so that they can be more easily understood by humans.