Dynamic Density Estimation for Incremental Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning techniques for anomaly detection suffer from poor performance due to lack of training data variety, low detection speed, and inability to continuously improve, particularly in industrial applications like testing manufactured articles.
Innovation Solution
A method involving a cold-start stage for initializing support vector sets using a pre-trained CNN, followed by an incremental training stage that updates these sets with self-generated anomalous samples, utilizing a convolutional neural network (CNN) and support vector sets to enhance anomaly detection performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine-learning models are trained with only normal data and a small amount of abnormal data, then the model can be initialized quickly, but the anomaly-detection performance is poor due to lacking training data variety
Solution Approach 1:
The system performs self-training by automatically generating synthetic abnormal samples from detected anomalies and using them to retrain the model. The model serves itself by converting its detection outputs into training data, eliminating the need for extensive manual abnormal data collection and achieving continuous performance improvement
Solution Approach 2:
The system establishes a feedback loop where detection results are fed back into the training process. Anomalies detected during inference are used to update the model through incremental retraining, creating a closed-loop system that continuously improves performance based on real-world detection outcomes
2Productivity
If traditional machine-learning techniques are implemented with poor memory efficiency, then the system uses less memory, but the detection speed is low
Solution Approach 1:
The training process is segmented into cold-start stage and incremental retraining sessions. The model is pre-trained on normal data first, then incrementally updated with synthetic abnormal samples. This segmentation allows efficient memory usage during pre-training while enabling speed improvements through subsequent incremental updates without requiring full retraining
3Reliability
If the model is trained offline with fixed training data, then the training process is simple, but the system cannot continuously improve anomaly-detection performance
Solution Approach 1:
The system transitions from static offline training to dynamic incremental retraining. The model adapts continuously by incorporating newly detected anomalies into subsequent training sessions, making the system dynamic and capable of evolving its performance over time based on accumulating detection experience
Solution Approach 2:
The system performs preliminary pre-training on normal data to establish a baseline model quickly. This preliminary action enables the model to start detecting anomalies immediately, while subsequent incremental retraining with synthetic abnormal samples progressively improves performance without requiring complete retraining from scratch
4Reliability
If synthetic abnormal samples are generated and used for retraining, then training data variety is improved, but computation effort increases
Solution Approach 1:
The system generates synthetic abnormal samples by copying and transforming features from detected anomalies. Instead of creating entirely new complex data, the system replicates and modifies existing anomaly patterns, significantly reducing the computational effort required for data generation while still achieving diverse training samples
Solution Approach 2:
The system transforms normal samples into synthetic abnormal samples by modifying parameters such as feature values and distributions. This parameter-based transformation approach is computationally more efficient than generating entirely new samples from scratch, while still producing diverse training data that improves model performance
Data Source
AI summary
In detecting anomaly in samples, convolutional neural network (CNN) and machine-learning classifier modelled with support vectors are used. The CNN and classifier are initially trained with normal samples, and incrementally trained in retraining sessions each with self-generated anomalous samples identified in inference preceding a retraining session under consideration, thereby continually improving the anomaly-detection performance without a need to seek anomalous samples for initializing the CNN and classifier. The support vectors are selected as feature k-centers of output feature map of the CNN. Dynamic density estimation is used to determine which feature k-centers in existing support vector set are retainable in updating the support vector set in the retraining session. As such, not all feature k-centers need to be recomputed to give the support vectors in the updated support vector set. Computation effort in updating the support vector set is reduced in comparison to generating this set from scratch.