Adaptive Image Classification Using Drift-Aware Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-based image classification models deployed at customer sites face challenges in maintaining prediction accuracy due to data changes, limited training data, and computational resources, leading to overfitting and reduced generalization capabilities.
Innovation Solution
Implementing multiple classification models that detect data drift through disagreement measurements and adaptive training datasets selected non-uniformly based on time intervals, allowing for autonomous learning without external data exposure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional AI-based image classification models are deployed at customer sites, then prediction accuracy can be achieved initially, but prediction accuracy deteriorates over time due to data changes and limited training data
Solution Approach 1:
The system transitions from static models to dynamic adaptive learning. Multiple classification models are continuously trained on adaptive training datasets that are updated over time based on production data, allowing the model to adapt to changing conditions and maintain prediction accuracy long-term
Solution Approach 2:
The system implements feedback loops where production data is continuously monitored, disagreement measurements are calculated between multiple models, and training datasets are adaptively updated based on this feedback to improve model performance over time
2Reliability
If more training data is used to improve model accuracy, then prediction accuracy improves, but computational resources and complexity increase
Solution Approach 1:
Instead of uniformly treating all training data equally, the system applies local quality by weighting different time intervals differently. Recent production data is assigned higher weights while older data is assigned lower weights, allowing the system to focus computational resources on the most relevant and currently applicable data patterns
Solution Approach 2:
The system uses partial action by selectively training on a subset of production data rather than all available data. Through adaptive sampling based on time intervals and disagreement measurements, the system identifies and trains on only the most informative portions of the data, reducing computational burden while maintaining accuracy
3Device complexity
If classification models are trained on limited production data, then computational resources are conserved, but generalization capabilities are reduced
Solution Approach 1:
The system performs preliminary action by continuously maintaining a historical archive of production data organized by time intervals. Even though not all data is used for training, the archived data serves as a reference that enables the model to generalize to unseen patterns by comparing current data against historical distributions
Solution Approach 2:
The system changes the parameter of data weighting over time. By dynamically adjusting the weights assigned to different time intervals based on disagreement measurements and production conditions, the system optimizes which historical data is most relevant for generalization, improving adaptability without requiring equal treatment of all data
4Reliability
If multiple classification models are used to improve prediction accuracy, then prediction accuracy improves, but disagreement measurements and data drift detection complexity increase
Solution Approach 1:
The system introduces an intermediary measurement mechanism - the disagreement measurement - that quantifies the differences between multiple classification models. This intermediary metric simplifies the complex task of detecting data drift by transforming it into a measurable disagreement score that can be threshold-based detection
Data Source
AI summary
A method, a computerized apparatus and a computer program product for adaptive learning for image classification. The method comprises applying a set of classification models on a calibration dataset and on a production dataset, and calculating disagreement measurements over the predictions thereof on each dataset. Based on similarity measurement between the disagreement measurement of the calibration and the production datasets, being below a predetermined threshold, a data drift is indicated in the production dataset. The method further comprises determining a training dataset for training a classification model for the production dataset. The training dataset is selected over a plurality of sets of images ordered according to time intervals in which images therein are obtained. The selection is performed based on weights determined for the plurality of sets.