Machine Learning Model Updating via Candidate Variable Subgroups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models deteriorate over time due to static training, leading to performance issues in handling evolving transactional data, which can result in financial losses, reputation damage, and regulatory fines, and are labor-intensive to generate and update.
Innovation Solution
Automatically generate and update machine learning models by intelligently creating candidate models using subgroups of decision variables, evaluating their performance against existing models, and updating when the candidate models perform better, with features including machine learning algorithms, hyperparameters, and thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are trained statically, then the training process is simple and labor-efficient, but the model performance deteriorates over time as transactional data evolves
Solution Approach 1:
The system automatically generates candidate models by selecting subgroups of decision variables from existing models, evaluates their performance, and updates the production model without human intervention. This self-service mechanism resolves the contradiction by enabling continuous performance improvement while maintaining high update efficiency through automation.
Solution Approach 2:
The invention changes parameters by selecting different subgroups of decision variables (algorithms, hyperparameters, features, thresholds) to create candidate models. This parameter variation allows the system to explore different model configurations and identify improvements while maintaining an automated, efficient update process.
2Reliability
If machine learning models are manually generated and updated, then the process can be carefully controlled and reviewed, but it becomes labor-intensive and time-consuming
Solution Approach 1:
The system performs self-service by automatically generating candidate models, evaluating their performance metrics, and updating the production model without requiring manual intervention. This eliminates the time-consuming aspects of manual model development while maintaining detection accuracy through automated performance evaluation and selection.
Solution Approach 2:
The invention implements feedback by automatically evaluating candidate models against performance metrics and using these results to determine whether to update the production model. This closed-loop feedback mechanism ensures high detection accuracy while reducing model update time through automated decision-making based on performance data.
3Adaptability or versatility
If existing machine learning models are used without updates, then the system is stable and easy to maintain, but it fails to detect evolving fraudulent and suspicious activities effectively
Solution Approach 1:
The system automatically adapts to evolving transactional data by self-service generation of candidate models with updated decision variables, performance evaluation, and automated deployment. This self-service approach enhances detection capability for evolving fraud patterns while avoiding the complexity of manual model management and retraining processes.
Data Source
AI summary
A system and method for automatically training a machine learning model may include a computing device; a memory; and a processor, the processor configured to: use of one or more subgroups of decision variables of a first machine learning model to train one or more candidate models; evaluate performance metric of one or more candidate models against the first machine learning model: when the performance metric of one or more candidate models is higher than the performance metric of the first machine learning model, update the first machine learning model to a second machine learning model selected from one or more candidate models.