Drift Detection in Machine Learning Models via Segmented Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting drift in machine learning models are inadequate, as they fail to accurately identify the source of drift and do not effectively handle different types of drift such as abrupt, gradual, incremental, or reoccurring changes in data properties over time.
Innovation Solution
The system retrieves current datasets from supervised and unsupervised machine learning models, segregates them based on drift detection models, applies these models to generate predictive results, and determines errors by comparing them to reference values. It detects drift when errors exceed a threshold and corrects the models accordingly, using various drift detection methods such as Fast Hoeffding Drift Detection Method, Paired Learner, Shuffling and Resampling, and Kernel Based Distribution Discrepancy Test.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional drift detection methods are used, then model performance monitoring is possible, but the ability to accurately identify the source of drift deteriorates
Solution Approach 1:
The patent segments the drift detection process into multiple specialized drift detection models, each designed to detect specific types of drift (data drift, concept drift, feature drift). This segmentation allows the system to not only monitor model performance but also precisely identify the source and type of drift by comparing outputs from different specialized models.
2Ease of operation
If basic drift detection algorithms are used, then drift detection is simple, but the ability to handle different types of drift (abrupt, gradual, incremental, reoccurring) deteriorates
Solution Approach 1:
The patent creates a universal drift detection system that handles multiple types of drift (abrupt, gradual, incremental, reoccurring) through a unified architecture. The system uses multiple drift detection models that can be selectively applied based on the detected drift type, making the system both simple to operate and highly adaptable to various drift scenarios.
3Measurement precision
If feature drift monitoring is performed without considering feature importance, then all feature changes are detected, but the ability to identify meaningful drift sources deteriorates
Solution Approach 1:
The patent applies local quality by differentiating the treatment of features based on their importance to the model. The system prioritizes monitoring and analysis of high-importance features while reducing focus on low-importance features. This approach ensures comprehensive drift detection coverage while preventing information overload by focusing resources on features that truly impact model performance.
Data Source
AI summary
Present disclosure relates to management of artificial intelligence systems by identifying root cause of reduced performance and/or failure in computing systems, and particularly relates to systems and methods for detecting a drift in supervised and unsupervised machine learning (ML) models. The system retrieves current dataset corresponding to output of supervised ML models and unsupervised ML models. Further, the system segregates the current dataset based on requirement of a drift detection model and applies a plurality of drift detection models to the segregated dataset to generate predictive results corresponding to the current dataset. Furthermore, the system determines errors in predictive results by comparing predictive results to reference values associated with current dataset. Additionally, the system detects the drift in supervised ML models and unsupervised ML models based on determined errors being above a threshold value. The supervised ML models and unsupervised ML models are corrected based on detected drift.