Anomaly Detection Model Retraining for Data Trend Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection algorithms using trained models face issues with over-detection and non-detection when data trends change, leading to repeated incorrect determinations.
Innovation Solution
A model training apparatus that includes a storage unit, detector, and retraining unit to store data and model parameters, determine anomalies, and retrain the model using corrected labels and trends to prevent repeated incorrect detections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a trained model is continuously used for anomaly detection, then detection speed is maintained, but detection accuracy deteriorates due to data trend changes
Solution Approach 1:
The system implements periodic retraining of the anomaly detection model at predetermined intervals. The retraining unit periodically updates the model using accumulated detection results and test data, ensuring the model adapts to changing data trends while maintaining continuous detection capability between retraining cycles.
Solution Approach 2:
The system uses detection results from the detector as feedback for model retraining. The retraining unit accumulates detection results and test data, then uses this feedback information to retrain and update the model parameters, creating a closed-loop system that continuously improves detection accuracy based on actual performance.
2Measurement precision
If the model is retrained frequently to improve accuracy, then detection accuracy improves, but computational resource consumption increases
Solution Approach 1:
Instead of continuous retraining, the system performs retraining periodically at predetermined intervals or when specific conditions are met. This reduces computational resource consumption compared to continuous retraining while still maintaining detection accuracy through periodic model updates.
Solution Approach 2:
The system uses accumulated detection results and test data over multiple periods for retraining, rather than processing all possible data continuously. This partial action approach achieves good detection accuracy while reducing the computational burden of processing complete datasets.
Data Source
AI summary
An apparatus for training a model includes a storage unit configured to store a parameter of the model trained by using a training data set, and the training data set, a detector configured to use the model to determine whether an anomaly is present in a test data set and store a determined result and the test data set in the storage unit, and a retraining unit configured to retrain the model by using the determined result, the test data set, and the training data set.


