Workload Correlation Analysis for System Dependency Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing distributed computing systems is challenging due to excess capacity, leading to increased costs and inefficiencies, as existing methods lack effective ways to determine dependencies and relationships between systems for consolidation.
Innovation Solution
An empirical method for detecting relationships and dependencies between systems through correlation analysis of workload activity levels, using CPU utilization, memory usage, disk I/O, network rates, and latency, which provides a correlation map to visualize and quantify system interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If non-empirical methods (inspection of configuration and run-time settings) are used to determine system dependencies, then domain knowledge and configuration analysis can identify relationships, but the method lacks precision and may miss actual runtime correlations
Solution Approach 1:
The patent replaces manual inspection methods (mechanical/configuration-based analysis) with automated empirical measurement systems that collect and analyze runtime workload data. This substitution enables precise detection of actual system dependencies through quantitative correlation analysis rather than qualitative configuration review.
Solution Approach 2:
The system continuously collects workload metrics from multiple systems, computes correlation coefficients to identify dependencies, and uses this feedback to guide consolidation decisions. This closed-loop approach allows ongoing refinement of dependency detection accuracy based on actual runtime behavior.
2Productivity
If distributed computing systems are expanded to meet organizational needs, then system capacity and functionality increase, but excess capacity leads to increased costs and inefficiencies
Solution Approach 1:
The patent extracts and identifies redundant systems from the distributed computing environment through correlation analysis. By detecting systems with low or no workload correlation, the method enables removal of excess capacity while maintaining necessary functionality, thereby reducing power consumption and operational costs.
Solution Approach 2:
The system identifies and discards redundant computing resources that do not contribute unique functionality. The consolidation process recovers value by eliminating duplicate systems while preserving essential capacity, transforming waste into cost savings.
3Loss of energy
If system consolidation is pursued to reduce costs, then operational expenses decrease, but determining suitable consolidation solutions becomes a daunting task due to many possible combinations
Solution Approach 1:
The patent provides feedback through correlation maps and quantitative metrics that rank system combinations by their consolidation suitability. This feedback mechanism transforms the daunting combinatorial problem into a guided decision-making process, where the most promising consolidation targets are clearly identified based on empirical data.
Solution Approach 2:
The system changes the parameters used for consolidation planning from qualitative configuration data to quantitative workload correlation metrics. This parameter transformation enables objective comparison of consolidation options and identifies optimal solutions based on measured system behavior rather than theoretical considerations.
4Reliability
If more servers are deployed to ensure adequate capacity, then system reliability and availability improve, but redundant servers increase capital expenses, maintenance costs, and heat production
Solution Approach 1:
The patent extracts the essential functionality from multiple servers by identifying which systems provide unique, non-redundant capabilities. Through correlation analysis, it separates necessary servers from redundant ones, maintaining reliability while reducing the total quantity of physical hardware.
Solution Approach 2:
The system merges functionality from multiple correlated servers into fewer consolidated systems. By combining workloads from systems with similar patterns, the patent maintains service availability while reducing server count, thereby lowering capital expenses, maintenance costs, and thermal output.
Data Source
AI summary
Relationships between systems can be inferred through a correlation analysis of the system workload activity levels. A method, computer readable medium and system are provided for analyzing correlations between the system workloads. The method comprises obtaining a set of quantile-based workload data pertaining to a plurality of systems. The correlation coefficient limit may then be used to compute the workload correlation scores for the plurality of systems and a result indicative of relationships between the systems then provided.