Anomaly Detection Manager for Metric Vector Deviation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection methods in IT ecosystems require deep domain knowledge and expertise, making it difficult for database administrators (DBAs) to identify operational issues caused by anomalies in large datasets without extensive math backgrounds or machine learning expertise.
Innovation Solution
A method and system that use a simple query syntax to construct metric vectors from time series data, calculate probability densities using a Multivariate Gaussian Distribution algorithm, and identify outlier metrics, making anomaly detection accessible to non-data analyst experts through a non-transitory computer-readable medium and anomaly detection manager with data input, vector generation, probability, and outlier engines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex machine learning algorithms and deep domain knowledge are used for anomaly detection, then measurement precision and reliability improve, but device complexity and ease of operation worsen
Solution Approach 1:
The patent introduces an intermediary system (anomaly detection manager with pre-configured algorithms and templates) that mediates between raw data and expert analysts. This intermediary handles the complex machine learning computations automatically, providing accurate anomaly detection results without requiring end-users to understand or configure the underlying complex algorithms, thus resolving the contradiction between measurement precision and device complexity
Solution Approach 2:
The system enables self-service anomaly detection by automatically performing complex statistical analyses, algorithm selection, and result interpretation without requiring expert intervention. The anomaly detection manager autonomously executes multivariate Gaussian distributions, auto-correlation analyses, and cross-correlation analyses, making expert-level anomaly detection accessible to non-experts and reducing the perceived system complexity
2Measurement precision
If expert analysis and manual investigation are performed, then anomaly detection accuracy improves, but loss of time increases
Solution Approach 1:
The patent applies preliminary action by pre-configuring multiple anomaly detection algorithms, statistical models, and analysis templates before runtime. The system pre-establishes multivariate Gaussian distributions, auto-correlation parameters, and cross-correlation frameworks, so that during operation, these pre-prepared tools can be rapidly deployed to detect and investigate anomalies without requiring time-consuming manual setup or expert configuration during the investigation itself
3Measurement precision
If comprehensive data analysis tools are provided, then measurement precision improves, but ease of operation worsens
Solution Approach 1:
The anomaly detection manager serves as an intermediary that shields users from complex data analysis tools while maintaining high measurement precision. It automatically selects and configures appropriate algorithms (multivariate Gaussian, auto-correlation, cross-correlation) based on data characteristics, presenting simplified results to users without requiring them to understand or manually configure the comprehensive analysis tools, thus resolving the contradiction between measurement precision and ease of operation
Solution Approach 2:
The system creates simplified copies or representations of complex analysis results that are easy to interpret. Instead of presenting raw statistical outputs from multivariate Gaussian distributions or correlation analyses, the system generates user-friendly anomaly scores, confidence levels, and actionable insights that maintain the precision of comprehensive analysis while being accessible to non-experts
Data Source
AI summary
In an example, metrics that cause a deviation in data may be identified by collecting the data for selected metrics stored in a plurality of tables. A metric vector is constructed based on the data for the selected metrics. A probability density may be calculated for the metric vector that indicates a deviation value for the metric vector relative to other metric vectors. Moreover, an outlier metric from the metric vector that causes the deviation value for the metric vector may be identified.