ML Workload Forecasting for Cloud Worker Instance Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud-based computing platforms face challenges in managing workload spikes, leading to allocation request throttling, increased queue depth, and slowness, which are often addressed reactively and ineffectively by manually increasing worker instances.
Innovation Solution
A device and method utilizing a machine learning (ML) model to predict a time series forecast of workload for an availability zone, compute the required number of worker instances, and generate a recommendation to increase worker instances proactively, differentiating between genuine customer demand and system disruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of worker instances is increased manually to handle workload spikes, then the system can handle higher workload, but the response time is delayed (days for provisioning) and the approach is reactive rather than proactive
Solution Approach 1:
The patent applies preliminary action by using machine learning models to predict future workload spikes and proactively provisioning worker instances before the actual demand occurs. The system forecasts workload patterns and automatically scales resources in advance, eliminating the reactive delay and ensuring capacity is ready when needed.
Solution Approach 2:
The system implements feedback by continuously monitoring actual workload patterns and comparing them with predictions, then using this feedback to refine future predictions and adjust provisioning decisions. This closed-loop approach improves prediction accuracy over time and optimizes resource allocation dynamically.
2Reliability
If the number of worker instances is increased to distribute workload, then throttling is reduced, but the system complexity increases and manual investigation is required
Solution Approach 1:
The system applies self-service by implementing automated machine learning-based prediction and decision-making for worker instance provisioning. The system autonomously monitors workload patterns, predicts future demands, determines optimal scaling decisions, and executes provisioning without manual intervention, thereby reducing operational complexity while maintaining high reliability.
Solution Approach 2:
The patent replaces manual mechanical processes (engineering team investigation and decision-making) with an automated computational system using machine learning models. This substitution eliminates the need for human intervention in monitoring and scaling decisions, reducing complexity and enabling faster, more consistent responses to workload changes.
3Speed
If worker instances are increased proactively using ML prediction, then allocation slowness is minimized, but the system requires sophisticated prediction algorithms and computing resources
Solution Approach 1:
The system performs preliminary computation by training machine learning models on historical workload data in advance, so that when real-time scaling decisions are needed, the models are already prepared to make rapid predictions. This pre-computation approach enables fast allocation decisions without requiring complex real-time analysis.
Solution Approach 2:
The patent introduces machine learning models as intermediary components between workload monitoring and resource provisioning. These models act as intelligent mediators that translate raw workload data into actionable scaling decisions, simplifying the overall system architecture while enabling sophisticated prediction capabilities.
Data Source
AI summary
Described are examples for recommending increase in worker instance count for an availability zone in a cloud-based computing platform. A machine learning (ML) model can be used to predict a time series forecast of a workload for the availability zone in a future time period. A predicted number of worker instances to handle the predicted workload can be computed, and if the number of worker instances in the availability zone is less than the predicted number of worker instances, a recommendation to increase the number of worker instances in the availability zone can be generated.