Cloud Cluster Outlier Detection via Peer Metric Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monitoring and managing cloud-based applications hosted in virtualized, multi-tenant infrastructure environments is challenging due to the dynamic and elastic nature of cloud resources, where resources can fluctuate rapidly, and there is limited visibility and control for tenants over the selected resources provided by independent IaaS providers.
Innovation Solution
A monitoring system that uses a data gateway and analysis module to collect live system-level metrics and detect outliers by comparing them against peer resources, employing correlation analysis, ANOVA analysis, and regression analysis to identify resources that deviate significantly, allowing for timely notification and potential issue resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud resources are made elastic and dynamically provisioned, then resource utilization and scalability are improved, but monitoring and detection of performance anomalies become more difficult
Solution Approach 1:
The monitoring system dynamically adapts to changing cloud infrastructure by continuously collecting metrics from virtual resources and automatically adjusting its analysis parameters. The system evolves its monitoring approach based on the elastic nature of cloud resources, making the monitoring itself dynamic rather than static.
Solution Approach 2:
The patent introduces an intermediary monitoring system that sits between the IaaS provider's virtual resources and the tenant applications. This intermediary collects metrics from multiple sources including the virtual infrastructure layer and the application layer, reconciling data from different levels to detect anomalies that would be invisible at any single layer.
2Measurement precision
If multiple data sources are integrated for comprehensive monitoring, then measurement completeness is improved, but system complexity increases
Solution Approach 1:
The monitoring system is segmented into distinct functional modules: a data collection component that gathers metrics from virtual resources, an analysis component that processes the metrics using statistical methods, and a notification component that alerts tenants. This segmentation allows each module to be independently developed and maintained while working together to achieve comprehensive monitoring.
Solution Approach 2:
The monitoring system is designed with universal components that can handle multiple types of virtual resources (compute instances, storage, networking) and multiple types of metrics (CPU utilization, memory usage, network throughput). The same statistical analysis framework applies across different resource types, reducing overall system complexity through reuse.
3Measurement precision
If statistical analysis methods are applied to detect outliers, then anomaly detection accuracy is improved, but computational requirements increase
Solution Approach 1:
The system applies statistical analysis selectively rather than uniformly to all metrics and all virtual resources. It focuses computational effort on identifying potential outliers using correlation analysis, then applies more intensive methods like ANOVA only when initial screening indicates potential anomalies. This partial application of analysis methods reduces overall computational energy consumption while maintaining detection accuracy for critical issues.
Data Source
AI summary
The present disclosure is directed to a system for monitoring and analyzing operation of a widely distributed service operated by an Infrastructure-as-a-Service (IaaS) tenant but deployed on a set of virtual resources controlled by an independent IaaS provider. The set of virtual resources can be organized into clusters in which resources are expected to behave similarly to each other. Virtual resources that do not behave similar to peer resources in the same cluster, i.e., outliers, may be indicative of problems that need to be addressed. The monitoring system can collect performance metric data from virtual resources, and compare the performance of each virtual resource in a cluster with the performance of every other virtual resource in the cluster to detect outliers. This comparison can involve correlation analysis, ANOVA analysis, or regression analysis.