Neural Network Error Distributions for Adaptive Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection techniques in systems, particularly in microservice architecture-based cloud systems, rely on static thresholds that require manual adjustment and are inadequate for dynamically changing anomaly patterns, necessitating specialized administrators and continuous adjustments.
Innovation Solution
Anomaly detection system utilizing neural networks to predict metric values, generate error data, estimate error distributions, and compare with normal distributions for accurate anomaly detection without the need for static thresholds, and a method to analyze trace data for response time distributions to detect anomalies in microservices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static threshold-based anomaly detection is used, then the system is simple to implement, but the detection accuracy deteriorates due to inability to adapt to dynamically changing anomaly patterns
Solution Approach 1:
The patent applies dynamics by replacing static thresholds with dynamic anomaly detection using neural networks. The system continuously learns from new data to adapt its anomaly detection criteria, allowing it to respond to changing patterns in real-time rather than relying on fixed predetermined values.
Solution Approach 2:
The system performs self-service by automatically learning from historical data and adjusting its own detection criteria without human intervention. The neural network model trains on past anomaly patterns and automatically updates its understanding of normal versus abnormal behavior, eliminating the need for manual threshold reconfiguration.
2Manufacturing precision
If manual threshold setting is required, then the system can be configured for specific needs, but the ease of operation deteriorates due to requiring specialized administrators
Solution Approach 1:
The system eliminates the need for specialized administrators by performing self-configuration through automated neural network training. It automatically learns optimal detection parameters from historical data, replacing the manual configuration process that previously required expert knowledge and specialized skills.
Solution Approach 2:
The patent applies parameter changes by automatically adjusting detection parameters through neural network training rather than manual configuration. The system transforms fixed threshold parameters into dynamic, learned parameters that adapt to the specific characteristics of the monitored system through data-driven optimization.
3Use of energy by moving object
If static thresholds are used, then the system requires minimal computational resources, but the adaptability to changing patterns deteriorates
Solution Approach 1:
The system transitions from static to dynamic computational processing by employing neural networks that actively adapt to changing patterns. While this increases computational requirements, the dynamic nature of the model allows it to efficiently handle evolving anomaly patterns that static thresholds cannot detect.
Solution Approach 2:
The patent applies continuity by implementing continuous learning and adaptation through the neural network. The system continuously processes new data to refine its anomaly detection capabilities, maintaining high adaptability over time rather than requiring periodic manual reconfiguration of thresholds.
4Measurement precision
If continuous threshold adjustments are required, then the system can maintain accuracy under changing conditions, but the loss of time increases due to ongoing maintenance
Solution Approach 1:
The system performs self-maintenance by automatically adjusting its detection parameters through continuous neural network training on new data. This eliminates the need for manual threshold adjustments and reduces maintenance time, as the system self-optimizes its performance without human intervention.
Solution Approach 2:
The patent applies feedback mechanisms where the neural network continuously learns from new data and adjusts its parameters accordingly. This closed-loop feedback system automatically maintains detection accuracy under changing conditions without requiring manual intervention or time-consuming reconfiguration.
Data Source
AI summary
An anomaly detection system is provided. The anomaly detection system may comprise at least one processor and a memory storing a computer program executed by the at least one processor, wherein the computer program includes instructions for operations of: acquiring a neural network model configured to predict a metric value for a specific time point for a target system, outputting a predicted value for a specific metric via the neural network model, generating error data for the specific metric based on a difference between the predicted value and a measured value, estimating a distribution of errors for the specific metric using the generated error data, and performing anomaly determination for the specific metric by comparing the estimated distribution with a normal distribution, which is a distribution derived from error data for the specific metric obtained when the target system operates normally.