Workload Profiling for Cloud Host Allocation Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud bursting mechanisms rely heavily on manual monitoring and administration to identify resource shortages or excesses, leading to inefficiencies in resource allocation and increased costs, as they lack automated tools to accurately determine the necessary resources for workload processing and optimize host system allocation.
Innovation Solution
The system automatically profiles workload processing characteristics, such as completion duration and resource consumption, to determine the required resources for pending workloads and identify allocatable host systems that can be removed or added, using updated aggregate completion duration and consumed resources to optimize resource provisioning and de-provisioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual monitoring and administration are used to identify resource shortages or excesses, then resource allocation can be adjusted, but the process becomes inefficient and costly due to lack of automation
Solution Approach 1:
The system enables automated self-service by having the workload manager automatically profile workloads, calculate aggregate completion durations, determine resource requirements, and identify hosts for addition or removal without human intervention. The system monitors itself and adjusts resource allocation autonomously based on observed workload patterns.
Solution Approach 2:
The system implements continuous feedback loops where the workload manager profiles completed workloads, updates aggregate completion durations, and uses this feedback to dynamically adjust resource allocation decisions. The system learns from past performance data to optimize future resource provisioning.
2Productivity
If cloud hosts are added to meet resource demand, then workload processing capacity increases, but costs increase due to provisioning additional infrastructure
Solution Approach 1:
The system dynamically adjusts cloud host allocation based on real-time workload profiling and aggregate completion duration calculations. Rather than static over-provisioning, the system continuously adapts resource quantity to match actual workload demands, adding hosts only when aggregate completion durations indicate genuine capacity shortages.
Solution Approach 2:
The system changes the parameter of host allocation quantity based on calculated resource requirements derived from workload profiling. The workload manager uses aggregate completion duration metrics to determine optimal host quantities, transitioning from fixed allocation to demand-driven dynamic allocation.
3Quantity of substance
If cloud hosts are removed to reduce costs, then infrastructure expenditure decreases, but workload processing capacity may be insufficient
Solution Approach 1:
The system performs preliminary actions by proactively profiling workloads and calculating aggregate completion durations before making removal decisions. The workload manager identifies potential capacity shortages in advance and adds hosts before they are actually needed, preventing performance degradation while avoiding unnecessary provisioning.
Solution Approach 2:
The system uses partial action by removing only the specific number of hosts that calculations show are excess, rather than blanket removal or conservative retention. The workload manager precisely determines the optimal host quantity based on workload profiling, removing just enough to reduce costs while maintaining sufficient capacity.
4Measurement precision
If aggregate completion duration is tracked for all workloads, then resource provisioning accuracy improves, but data processing and storage requirements increase
Solution Approach 1:
The system extracts only the essential metric of aggregate completion duration from workload data, focusing on this single key parameter for resource provisioning decisions rather than tracking all possible workload attributes. The workload manager profiles workloads to extract completion time information specifically, reducing data processing overhead while maintaining provisioning accuracy.
Data Source
AI summary
Provided are a computer program product, system, and method for profiling workloads in host systems allocated to a cluster to determine adjustments to allocation of host systems to the cluster. A determination is made of workloads processing and resource usage in a computing system. An aggregate completion duration for determined workloads that have completed processing is updated. An aggregate consumed resources, comprising an aggregate of resources consumed by workloads, by resources consumed by the determined workloads is updated. The aggregate completion duration and the aggregate consumed resources are used to determine resources to provision for workloads.