Incremental Machine Learning Training via Drift Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models in AI systems face performance degradation due to data drift, where new data patterns differ from the original training dataset, leading to increased resource consumption and difficulty in determining when retraining is necessary, especially in identity management systems.
Innovation Solution
Incremental training of machine learning models by evaluating datasets for drift using drift detection models, such as Probably Approximately Correct (PAC) learning or graph embeddings, to determine if updates are needed, thereby retaining previous knowledge and adapting to new data patterns without complete retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are continuously retrained on new data, then model performance is maintained, but computational resources and time are excessively consumed
Solution Approach 1:
The patent applies partial action by implementing incremental training that processes only a subset of new data rather than complete retraining. The system evaluates whether incremental updates are sufficient, and only performs partial retraining when necessary, thereby reducing computational resource consumption while maintaining model performance.
Solution Approach 2:
The system performs preliminary evaluation of data drift using drift detection models before committing to full retraining. By assessing whether the new data significantly deviates from training data patterns, the system can avoid unnecessary computational resources while ensuring model performance is maintained when drift is detected.
2Adaptability or versatility
If complete retraining is performed frequently, then model adapts to new data patterns, but training time increases significantly
Solution Approach 1:
The system implements periodic evaluation of data drift using drift detection models at scheduled intervals or triggered by specific events. This periodic monitoring allows the system to adapt to new data patterns only when necessary, reducing training time while maintaining adaptability through timely incremental updates rather than continuous retraining.
Solution Approach 2:
The training frequency and extent are made dynamic based on detected data drift levels. The system adjusts whether to perform incremental training or complete retraining based on real-time assessment of data pattern changes, optimizing the balance between adaptability and training time consumption.
3Productivity
If all new data is stored for model training, then comprehensive training is possible, but storage requirements increase
Solution Approach 1:
The system extracts only the necessary portions of new data that exhibit significant drift from the training data patterns. By using drift detection to identify and extract only the relevant data subsets for incremental training, the system reduces storage requirements while maintaining training comprehensiveness for adapting to new patterns.
4Ease of operation
If drift detection models are implemented, then training necessity is determined, but system complexity increases
Solution Approach 1:
The system implements self-service through automated drift detection and evaluation mechanisms that automatically determine when and whether incremental training is necessary. The drift detection models autonomously assess data pattern changes and trigger appropriate training actions, simplifying operation while managing complexity through automation rather than manual intervention.
Data Source
AI summary
Systems and methods for embodiments of incremental training of machine learning model in artificial intelligence systems are disclosed. Specifically, embodiments of incremental training of machine learning models using drift detection models are disclosed, including embodiments that utilize drift detection models to determine drift based on identity graphs in artificial intelligence identity management systems.