Workload Virtual Machine Scaling for Spot Eviction Risk
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to dynamically adjust the number of virtual machines to maintain target capacity while minimizing costs, especially when using a mix of on-demand and spot virtual machines, due to variable eviction rates and configuration times.
Innovation Solution
A system and method for predicting the target number of virtual machines based on eviction probabilities and costs over multiple time horizons, using integer linear programming to optimize the split between on-demand and spot machines, and automatically adjusting the number of machines ahead of time to maintain capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If spot virtual machines are used to reduce costs, then cost savings are achieved, but the virtual machines may be evicted at any time when spare capacity remains
Solution Approach 1:
The system performs preliminary actions by predicting future VM evictions before they occur and proactively configuring replacement VMs in advance. The predictive model analyzes historical data and current conditions to forecast which spot VMs will be evicted, allowing the system to prepare replacements during the current stage so they are ready when needed, thus maintaining availability while utilizing cost-effective spot instances.
Solution Approach 2:
The system dynamically adjusts the mix of spot and on-demand VMs based on real-time conditions and predictions. Rather than using a static configuration, the system continuously monitors spot VM performance, eviction risks, and workload requirements to optimize the proportion of spot versus on-demand instances, enabling cost savings while maintaining service level objectives through adaptive resource allocation.
2Reliability
If the number of virtual machines is increased to maintain target capacity, then workload capacity is ensured, but costs increase
Solution Approach 1:
The system employs self-service mechanisms through automated predictive modeling and dynamic resource allocation. The predictive model automatically identifies which spot VMs are at risk of eviction and calculates the optimal number of replacements needed, eliminating the need for manual intervention. This self-managing approach ensures capacity maintenance while minimizing costs by precisely configuring only the necessary number of replacement VMs based on actual predicted evictions rather than over-provisioning.
Solution Approach 2:
The system changes key parameters such as the spot-to-on-demand VM ratio, prediction time horizons, and replacement thresholds to optimize the balance between capacity and cost. By dynamically adjusting these parameters based on workload characteristics, historical eviction patterns, and current pricing conditions, the system maintains target capacity with minimal cost impact rather than using fixed, conservative configurations.
3Reliability
If virtual machines are configured in advance to account for configuration time, then machines are ready to service workload, but the system complexity increases
Solution Approach 1:
The system segments the VM management process into distinct stages: prediction (identifying at-risk spot VMs), configuration (preparing replacement VMs), and deployment (activating replacements when needed). This segmentation allows each component to be optimized independently and simplifies the overall system by breaking down the complex predictive-configurate-deploy workflow into manageable, automated steps that can be executed sequentially based on predicted eviction timelines.
Solution Approach 2:
The system introduces an intermediary predictive model that acts as a mediator between spot VM monitoring and VM configuration actions. This intermediary layer analyzes eviction risks and generates predictions that trigger appropriate configuration actions, simplifying the decision-making process. Rather than directly managing complex real-time VM orchestration, the system uses the predictive model as an intermediary to translate observed patterns into actionable configuration decisions, reducing overall system complexity.
Data Source
AI summary
A method for dynamically adjusting a number of virtual machines for a workload, includes: receiving a probability indicator for each of a plurality of N sequential stages, where N is a natural number greater than 1, of a likelihood that a virtual machine assigned to a workload will be evicted during the N sequential stages; predicting a target number of virtual machines to configure in a current stage for a subsequent stage from among the plurality of N sequential stages based on the probability indicator, a target capacity for the workload, and a current price for maintaining a virtual machine; and configuring a number of virtual machines for the workload during the current stage based on the target number to be loaded for the workload for the subsequent stage.