Threshold-Guided AI Model Retraining Across Three Drift Types
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems lack an automated system to simultaneously consider data drift, concept drift, and model drift for effective retraining of AI models, leading to inadequate retraining mechanisms that fail to address the underlying causes of model degradation.
Innovation Solution
A system and method for adaptive retraining of AI models that computes drift magnitude scores, aggregates data, concept, and model drift scores, and determines retraining based on overall drift scores, using combined analysis of these drifts to select appropriate retraining data and perform retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated analysis of drifts is implemented, then retraining effectiveness is improved, but device complexity increases
Solution Approach 1:
The system segments the complex drift analysis into three distinct types: data drift detection (comparing input data distributions), concept drift detection (comparing target variable relationships), and model drift detection (comparing model behavior changes). Each drift type is analyzed separately using specific algorithms, making the overall complex system manageable and effective
Solution Approach 2:
The system introduces drift magnitude scores as intermediary metrics that quantify the extent of each drift type. These scores serve as mediators between the raw drift detection algorithms and the final retraining decision, providing a structured way to evaluate and compare different drift types before determining whether retraining is necessary
2Reliability
If multiple drift types are analyzed simultaneously, then model performance is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary drift detection by continuously monitoring data and model behavior in the background before performance degradation becomes critical. Drift magnitude scores are computed proactively, allowing the system to identify drift patterns early and initiate retraining before model performance significantly deteriorates
Solution Approach 2:
The system implements a feedback mechanism where drift detection results feed into automated retraining decisions. The drift magnitude scores provide quantitative feedback that triggers retraining workflows, creating a closed-loop system that continuously monitors and responds to drift conditions without requiring manual intervention
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for adaptive retraining of an artificial intelligence model. A computer system computes drift magnitude scores for respective drift functions. A computer system computes an aggregated data drift score for a data drift, an aggregated concept drift score for a concept drift, and an aggregated model drift score for a model drift. A computer system computes an overall drift score, based on the aggregated data drift score, the aggregated concept drift score, the aggregated model drift score, a predetermined data drift threshold, a predetermined concept drift threshold, and a predetermined model drift threshold. A computer system determines whether retraining of the artificial intelligence model is required, based on the overall drift score. A computer system performs the retraining of the artificial intelligence model, in response to determining the retraining of the artificial intelligence model is required.