Circumstance-Specific Detectors for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large data centers, network operators face challenges in monitoring and analyzing performance metrics across numerous network layers and elements, leading to delayed identification of anomalous behaviors, which can impact service level agreements and user experience.
Innovation Solution
The technology employs circumstance-specific detectors to monitor performance data, using a combination of statistical analysis and machine learning techniques to identify anomalies by training classifiers with feature vectors generated from user feedback, enabling timely detection and automation of system behavior changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network operators manually monitor and analyze performance metrics across numerous network layers and elements, then they can identify system issues, but the identification of anomalous behaviors is delayed and operators are overwhelmed
Solution Approach 1:
The system performs self-service by automatically monitoring performance metrics, generating anomaly event candidates, and identifying anomalous behaviors without continuous human intervention. The circumstance-specific detectors and classifiers autonomously process time series data to detect anomalies, freeing operators from manual monitoring while maintaining high detection accuracy and reducing identification time.
Solution Approach 2:
The monitoring system is segmented into multiple circumstance-specific detectors, each specialized for detecting anomalies under specific conditions or in specific domains. This segmentation allows parallel processing of different metric types and conditions, improving both detection speed and accuracy by distributing the analysis workload across specialized components rather than using a single general-purpose detector.
2Reliability
If forensic examination is conducted by examining protocols or log files of past or recent running processes, then anomalous behaviors can be identified, but the examination is performed only after anomalies have already occurred
Solution Approach 1:
The system performs preliminary action by continuously monitoring performance metrics and detecting anomalies in real-time, before they escalate into system failures. The circumstance-specific detectors analyze time series data proactively to identify early signs of problems, enabling operators to take preventive actions before service level agreements are violated, rather than reacting after failures occur.
Solution Approach 2:
The system implements feedback by continuously comparing current performance metrics against learned normal patterns and providing real-time alerts when deviations are detected. This closed-loop feedback mechanism enables rapid response to anomalies by immediately notifying operators of potential issues, allowing them to take corrective actions before service level agreements are breached.
3Productivity
If automation is increased in network monitoring environments, then timely actions can be taken to maintain service level agreements, but the complexity of monitoring and analyzing performance metrics increases
Solution Approach 1:
The system applies local quality by creating circumstance-specific detectors tailored to different monitoring scenarios, metric types, and anomaly patterns. Each detector is optimized for its specific domain rather than using a single complex general-purpose detector. This approach increases productivity by providing specialized detection capabilities while managing complexity through modular, context-specific designs.
Solution Approach 2:
The system utilizes parameter changes by adapting detection thresholds, time windows, and analysis parameters based on the specific circumstances being monitored. The classifiers learn optimal parameters from training data and adjust them dynamically to match different operating conditions. This allows the system to maintain high productivity across varying scenarios while keeping the underlying architecture relatively simple through parameter adaptation rather than structural complexity.
Data Source
AI summary
The technology disclosed relates to learning how to efficiently display anomalies in performance data to an operator. In particular, it relates to assembling performance data for a multiplicity of metrics across a multiplicity of resources on a network and training a classifier that implements at least one circumstance-specific detector used to monitor a time series of performance data or to detect patterns in the time series. The training includes producing a time series of anomaly event candidates including corresponding event information used as input to the detectors, generating feature vectors for the anomaly event candidates, selecting a subset of the candidates as anomalous instance data, and using the feature vectors for the anomalous instance data and implicit and/or explicit feedback from users exposed to a visualization of the monitored time series annotated with visual tags for at least some of the anomalous instances data to train the classifier.