Anomalous Behavior Identification From Dynamic Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In complex distributed systems, it is challenging to efficiently detect anomalous behavior without requiring extensive labeled data, as existing methods either rely on costly data labeling or unstable clustering algorithms.
Innovation Solution
The system employs time-series data analysis to determine eigenvalues and fluctuations for each entity, calculates standard points, and identifies anomalous behavior based on differences between the determined data and the standard points, allowing for efficient identification with minimal human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised clustering algorithms are used to avoid data labeling, then data labeling cost is reduced, but cluster stability deteriorates over time due to changes in time-series data
Solution Approach 1:
The patent transforms the clustering problem by changing parameters: instead of clustering raw time-series data directly (which becomes unstable over time), it extracts statistical features (mean, standard deviation, trend, seasonality) and clusters these derived parameters. This parameter transformation makes the clustering results more stable and comparable across different time periods, while still avoiding the need for expensive manual data labeling.
2Ease of operation
If metric thresholds are used for anomaly detection, then detection simplicity is maintained, but detection accuracy deteriorates due to noise and false positives
Solution Approach 1:
The patent introduces statistical features (mean, standard deviation, trend, seasonality) as intermediary representations between the raw metric data and the anomaly detection decision. Instead of directly comparing raw metrics to fixed thresholds, the system first transforms the data into these statistical features, then performs clustering and anomaly detection on the features. This intermediary transformation layer filters out noise and provides more meaningful patterns for accurate anomaly detection while maintaining operational simplicity.
3Measurement precision
If a classifier is trained to detect anomalous behavior, then detection precision is improved, but data labeling cost increases due to requirement of vast labeled data
Solution Approach 1:
The patent implements self-service anomaly detection by automatically extracting statistical features and performing unsupervised clustering to identify anomalies without requiring manual data labeling. The system serves itself by using the inherent statistical properties of the time-series data (mean, standard deviation, trend, seasonality) to automatically distinguish normal from anomalous behavior patterns, eliminating the need for expensive expert-labeled training data while maintaining high detection precision.
Data Source
AI summary
Systems and methods include reception of time-series data of a metric for each of a plurality of computer servers, determination, for each computer server, of a representative value of the metric based on the time-series data of the metric for the computer server, determination, for each computer server, of a fluctuation value of the metric based on the time-series data of the metric for the computer server, determination of a standard value of the metric based on the determined representative values, determination of a standard fluctuation value based on the determined fluctuation values, determination, for each computer server, of a difference value based on a difference between the standard value and the representative value for the computer server and a difference between the standard fluctuation value and the fluctuation value for the computer server; and identification of one or more anomalous computer servers based on the difference values.