Drift-Triggered Model Retraining with Interpolated Active Learning Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face performance degradation due to drifts, necessitating retraining, but current retraining methods like periodic retraining and online learning are inefficient or resource-intensive, especially when data availability is insufficient.
Innovation Solution
A method for retraining machine learning models by detecting drifts, identifying relevant features and responses, determining a time window, and interpolating data as needed to ensure sufficient data for retraining, using techniques such as dynamic time wrapping and clustering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic retraining is performed, then model performance is maintained, but retraining may occur when not needed or data may be insufficient
Solution Approach 1:
The system implements drift detection that continuously monitors model predictions and compares them with actual data to detect performance degradation. This feedback mechanism triggers retraining only when drift is detected, avoiding unnecessary periodic retraining while ensuring model performance is maintained when needed.
Solution Approach 2:
The system performs preliminary drift detection before retraining is actually needed. By continuously monitoring for drift conditions and preparing data in advance, the system can initiate retraining immediately when drift is detected, rather than waiting for scheduled intervals.
2Reliability
If online learning is used to update models continuously, then model performance is maintained, but computational resources are excessively consumed
Solution Approach 1:
Instead of continuous online learning updates, the system performs retraining periodically but only when drift is detected. This transforms continuous computational effort into discrete, event-driven retraining operations, significantly reducing overall resource consumption while maintaining model performance.
Solution Approach 2:
The system uses its own operational data and drift detection capabilities to determine when retraining is needed, rather than relying on external scheduling or continuous monitoring systems. This self-service approach optimizes resource usage by making retraining decisions based on actual model performance needs.
3Reliability
If retraining is performed when data is insufficient, then model performance may degrade, but collecting more data delays retraining
Solution Approach 1:
The system performs preliminary drift detection and data assessment before retraining is actually needed. By continuously monitoring drift conditions and pre-processing data, the system ensures that when drift is detected, sufficient data is already available or can be quickly assembled, eliminating delays between drift detection and retraining.
Solution Approach 2:
The system introduces an intermediary drift detection and data assessment layer between data collection and model retraining. This intermediary evaluates whether sufficient data is available and prepares appropriate datasets, acting as a bridge that ensures retraining proceeds timely with adequate data without requiring extensive additional data collection.
4Productivity
If data selection is made without interpolation, then retraining is faster, but data insufficiency leads to poor model performance
Solution Approach 1:
The system introduces data interpolation as an intermediary step between data extraction and model retraining. When drift detection identifies a need for retraining but sufficient data is not immediately available, the interpolation process generates synthetic or estimated data to complete the dataset, enabling timely retraining without compromising data sufficiency.
Solution Approach 2:
The system changes the parameters and characteristics of available data through interpolation techniques. By transforming and generating additional data points based on existing patterns, the system expands the available dataset to meet retraining requirements without requiring extensive additional data collection, thus maintaining both speed and performance.
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for retraining a model in case of a drift in machine learning. A computer detects a drift in machine learning. A computer identifies in a database features and a response of a machine learning model. A computer determines a time window of the drift. A computer extracts, from the database, data of the features and the response in the time window. A computer determines whether extracted data is sufficient for retraining the machine learning model. A computer, in response to determining that the extracted data is not sufficient for retraining the machine learning model, interpolates one or more of the features for a predetermined future time horizon. A computer interpolates a response corresponding to one or more interpolated features. A computer retrains the machine learning model, using the one or more interpolated features and an interpolated response corresponding thereto.