Anomaly Detection via Autoencoder Normal Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Anomaly pattern detection technologies based on machine learning face challenges in securing training data due to the difference in appearance frequency between normal and anomaly patterns, making it difficult to distinguish between the two and requiring significant resources for tagging rare anomaly patterns.
Innovation Solution
An anomaly pattern detection system that models input data as all normal patterns, using an auto-encoder neural network to generate a learning model and detect anomalies by comparing the similarity between input and output data, thereby eliminating the need for separate anomaly pattern tagging and reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly pattern detection is performed using traditional machine learning methods, then anomaly patterns can be detected, but it requires significant resources for tagging rare anomaly patterns and securing training data
Solution Approach 1:
The patent inverts the traditional approach by assuming all input data is normal and modeling only normal patterns. Anomalies are detected as deviations from this normal model, eliminating the need to tag or separately learn anomaly patterns. This is implemented through an autoencoder that learns to reconstruct normal data, where reconstruction errors indicate anomalies.
Solution Approach 2:
The system uses unsupervised learning where the model learns normal patterns autonomously without requiring labeled anomaly data. The autoencoder automatically identifies anomalies through reconstruction error thresholds, making the system self-sufficient and eliminating manual tagging requirements.
2Ease of manufacture
If all input data is used for modeling normal patterns, then the system can operate without anomaly tagging, but it may reduce detection precision for rare anomaly patterns
Solution Approach 1:
The patent adjusts the detection threshold parameter based on the reconstruction error distribution. By dynamically setting thresholds rather than using fixed values, the system maintains high precision for rare anomalies while benefiting from the simplicity of unsupervised learning on all data.
3Productivity
If traditional supervised learning is used with limited anomaly data, then training can be completed, but the model struggles to distinguish normal from anomaly patterns due to frequency differences
Solution Approach 1:
Instead of trying to learn both normal and anomaly patterns with limited anomaly data, the patent learns only normal patterns and detects anomalies as deviations. This inversion allows the model to achieve high reliability in pattern distinction by focusing computational resources on the abundant normal data rather than struggling with scarce anomaly examples.
Data Source
AI summary
Provided is an anomaly pattern detection system including an anomaly detection device connected to one or more servers. The anomaly detection device may include an anomaly detector configured to model input data by considering all of the input data as normal patterns, and detect an anomaly pattern from the input data based on the modeling result.