Anomaly Score Learning Model for Biased IoT Detection Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing abnormality detection systems for IoT devices face increased processing times due to biased data distributions, particularly when learning with imbalanced data sets, leading to erroneous detections.
Innovation Solution
A training device that selectively learns and generates models from unlearned data with anomaly scores above a threshold, iteratively refining the model to reduce biased data and shorten learning time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If learning is performed using all learning data in a single stage, then learning time is short, but learning accuracy deteriorates due to biased data distribution
Solution Approach 1:
The learning process is segmented into multiple stages: first learning common patterns from all data, then selectively learning rare patterns from unlearned data with high anomaly scores. This segmentation resolves the contradiction by allowing different learning objectives in different stages, achieving both speed and accuracy.
Solution Approach 2:
The system performs preliminary learning on common data patterns before addressing rare patterns. By first establishing a baseline model from all data, then iteratively refining with selected unlearned data, the system prepares the learning process in advance to handle both common and rare cases effectively.
2Manufacturing precision
If two-stage learning is used to handle biased data distribution, then learning accuracy improves, but processing time increases
Solution Approach 1:
The system extracts only the necessary unlearned data (those with high anomaly scores) for the second learning stage, rather than relearning all data. This extraction approach maintains learning accuracy by focusing on rare patterns while significantly reducing processing time compared to complete two-stage learning.
Solution Approach 2:
Instead of performing complete two-stage learning on all data, the system applies partial action by selectively processing only unlearned data with high anomaly scores. This partial approach achieves the necessary learning accuracy improvement without the full time cost of traditional two-stage methods.
3Adaptability or versatility
If all normal data is used for learning, then model coverage is comprehensive, but detection accuracy for rare patterns deteriorates
Solution Approach 1:
The learning process is segmented into common pattern learning (from all data) and rare pattern learning (from selected unlearned data). This segmentation allows the model to maintain comprehensive coverage while achieving high detection accuracy for rare patterns through focused iterative learning.
Solution Approach 2:
The system performs preliminary learning on all data to establish comprehensive coverage, then iteratively refines the model by focusing on unlearned data with high anomaly scores. This preliminary comprehensive learning followed by targeted refinement achieves both coverage and accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A generation unit (131) learns data selected as unlearned data among learning data and generates a model calculating an anomaly score. A selection unit (133) selects, as unlearned data, at least some of data in which an anomaly score calculated by the model generated by the generation unit (131) is equal to or greater than a threshold among the learning data.