Multivariate Anomaly Detection for Real-Time Network Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing and controlling computer networks and data centers is challenging due to the difficulty in effectively utilizing large sets of multivariate time series metrics, which are often out-of-date and require machine-driven analysis to handle millions of data points accurately and efficiently.
Innovation Solution
A method for detecting anomalous patterns in multivariate time series metrics by generating a model that identifies operating thresholds and using a steady state distance matrix to detect anomalies, followed by a pattern distance matrix for accurate and scalable analysis, enabling real-time control and management of computing entities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine-driven analysis is used to handle millions of data points, then measurement precision and productivity are improved, but device complexity increases
Solution Approach 1:
The patent segments the complex analysis task into distinct functional modules: a data reception module that collects metrics, a model generation module that creates anomaly detection models, and an anomaly detection module that applies the models. This segmentation allows each module to handle specific aspects of the analysis independently, improving precision while managing complexity through modular architecture.
Solution Approach 2:
The patent implements preliminary action by generating anomaly detection models in advance based on historical metric data and operational thresholds. These pre-generated models are stored and ready for deployment, allowing the system to perform rapid anomaly detection without performing complex analysis in real-time, thus improving response speed while managing computational complexity.
2Productivity
If real-time anomaly detection is implemented, then productivity and response time are improved, but use of energy and computational resources increases
Solution Approach 1:
The system performs computationally intensive model generation in advance using historical data, storing the resulting anomaly detection models for later use. During real-time operation, the system only needs to apply these pre-generated models to incoming metrics, significantly reducing real-time computational resource consumption while maintaining fast detection speed.
Solution Approach 2:
The patent replaces complex real-time computational analysis with simpler pattern matching operations. Instead of performing heavy mathematical computations on incoming data points, the system substitutes this with direct comparison against pre-computed anomaly patterns and thresholds, reducing energy consumption while maintaining detection capability.
Data Source
AI summary
A system and method for detecting anomalies in a data stream is described. The system receives the data stream that comprises values of metrics derived from observations of operation of a computing entity over a time window. A model comprising variances of the data over the time window is formed. The model identifies operating thresholds for each metric based on the variances of the data for each metric in the data stream. The system computes a steady state distance matrix of the data stream. The system determines that the steady state distance matrix exceeds a steady state threshold. In response to determining that the steady state distance matrix exceeds the steady state threshold, the system computes a pattern distance matrix based on the steady state distance matrix. The anomaly in the data stream is detected based on the pattern distance matrix. The system generates an alert indicating the anomaly.