Online Fraud Detection Model Update Trigger via Drift Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current online fraud detection Machine Learning (ML) models either do not update at all or implement passive updates, leading to performance decline due to statistical changes in incoming data points, and suffer from high computational costs and inefficiencies in handling data drift.
Innovation Solution
A computerized method and system that utilize meta-learning techniques to detect data drift, assign weights, make update decisions, and evaluate performance, allowing the online ML model to adapt and optimize its performance over time by determining when an update is required based on incoming data points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If passive update is implemented by continuously updating the online fraud detection model with each incoming data point, then the model can adapt to statistical changes in data, but the computational cost becomes excessively high and resource-intensive
Solution Approach 1:
A drift detection model is introduced as an intermediary component between the incoming data stream and the online fraud detection model. This mediator analyzes data drift characteristics and determines when updates are actually needed, preventing unnecessary continuous updates while ensuring the model adapts when statistical changes occur. The drift detection model acts as a filter that reduces the frequency of updates to only when necessary.
Solution Approach 2:
The system implements a feedback mechanism where the drift detection model continuously monitors incoming data points and provides feedback about data distribution changes. Based on this feedback, the system intelligently decides whether to trigger an update of the online fraud detection model. This feedback loop enables adaptive update scheduling that balances model performance with computational efficiency.
2Adaptability or versatility
If passive update is implemented to continuously adapt the model, then the model can respond to data changes, but the system's responsiveness and scalability are slowed down
Solution Approach 1:
The drift detection model serves as a mediator that preprocesses incoming data and determines update necessity before triggering model updates. This intermediary layer filters out unnecessary update triggers, allowing the system to maintain high responsiveness for data processing while only performing computationally intensive model updates when actually needed based on detected data drift.
3Measurement precision
If update decisions are made based on all features, then comprehensive data drift detection is achieved, but the complexity of the decision-making process increases
Solution Approach 1:
The system assigns different weights to different features based on their importance and relevance to fraud detection. Instead of treating all features equally, the drift detection model focuses on weighted features that have higher impact on model performance. This local quality approach concentrates computational effort on the most critical features, reducing decision-making complexity while maintaining detection accuracy.
Solution Approach 2:
The system dynamically adjusts the parameters of the drift detection model, including feature weights and detection thresholds, based on the specific characteristics of the data stream and fraud patterns. These parameter changes allow the system to adapt the complexity of monitoring to the actual needs, simplifying the decision process when data is stable and increasing monitoring precision when drift is detected.
Data Source
AI summary
A computer-implemented method for determining when an update of an online ML model is required. The computer-implemented method includes: (i) receiving a batch of financial transactions data; (ii) selecting a set of features from the one or more features; (iii) detecting a drift and a drift type in each feature in the selected set of features, by operating a drift detection model thereon; (iv) generating a batch-representation-vector of drift type for each feature in the selected set of features; (v) receiving a predicted-decision of update-needed by forwarding the generated batch-representation-vector to a trained MetaBDMM model, the predicted-decision of update-needed is one of: update-needed; and update-not-needed, and (vi) forwarding the predicted-decision of update-needed to the online ML model. When the predicted-decision of update-needed is update-needed, the online ML model performs an update and when the predicted-decision of update-needed is update-not-needed the online ML model doesn't perform the update.