Model Relearning Assessment for IoT Anomaly Detection Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing abnormality detection systems for IoT devices face challenges in selecting appropriate schemes to track changes in normal states, particularly when over-detection feedback (FB) fails to function well due to significant changes or large numbers of new normal patterns, requiring deep machine learning knowledge that is not accessible to all users.
Innovation Solution
A determination device that includes a determination unit to assess the necessity of relearning or over-detection feedback based on information from initial and over-detection models, and a notification unit to inform users of the appropriate scheme, using criteria such as over-detection data ratios, model loss functions, and anomaly scores to guide model updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If over-detection feedback is used to track small changes from a learned model, then the system can maintain detection accuracy with minimal intervention, but the scheme becomes ineffective when significant changes occur or large numbers of new normal patterns are introduced
Solution Approach 1:
The patent implements a feedback mechanism where the determination device continuously monitors the learned model's performance and automatically triggers relearning when deterioration is detected. This closed-loop feedback system resolves the contradiction by maintaining the simplicity of over-detection feedback for small changes while automatically adapting to significant changes through automated relearning decisions.
Solution Approach 2:
The determination device performs self-assessment of the learned model's state and autonomously determines when relearning is necessary. This self-service approach eliminates the need for user intervention and deep machine learning knowledge, allowing the system to automatically adapt to both small and significant changes without manual configuration.
2Reliability
If relearning is performed frequently to track changes in normal state, then the model remains up-to-date with current patterns, but computational resources and time are consumed
Solution Approach 1:
Instead of performing full relearning continuously, the system applies partial action by only triggering relearning when the determination device detects actual model deterioration. This avoids unnecessary relearning operations and their associated time costs while maintaining model accuracy when needed.
Solution Approach 2:
The patent replaces manual monitoring and decision-making mechanisms with an automated determination device that uses algorithms to assess model state and trigger relearning. This substitution eliminates the need for user intervention and provides consistent, objective decisions about when relearning is necessary, reducing both time loss and improving reliability.
3Ease of manufacture
If users manually determine when to perform relearning or use over-detection feedback, then they can control resource usage, but deep machine learning knowledge is required to appropriately select schemes
Solution Approach 1:
The determination device provides self-service functionality by automatically assessing the learned model's state and determining the appropriate scheme (relearning or over-detection feedback) without user intervention. This eliminates the need for users to possess deep machine learning knowledge while maintaining control over resource usage through automated decision-making.
Solution Approach 2:
The determination device acts as an intermediary between the learned model and the user, translating complex model state assessments into simple recommendations or automated actions. This intermediary layer shields users from the complexity of machine learning concepts while providing appropriate control mechanisms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A determination unit (332) determines necessity of relearning of an initial learning model based on at least one of information regarding the initial learning model generated by learning initial learning data known to be normal, information regarding over-detection data over-detected by an abnormality detection system that uses the initial learning model, and information regarding an over-detection model generated based on the over-detection data. A notification unit (333) notifies of determination result by determination unit (332).