Experience Pruning for Cloud Workload Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing providers face challenges in efficiently allocating resources to meet service level agreements (SLAs) due to unpredictable workload demands and varying resource requirements, leading to inefficiencies and potential SLA violations.
Innovation Solution
The implementation of a multi-agent reinforcement learning-based system that dynamically allocates resources and migrates workloads to optimize resource usage, using a Restricted Boltzmann Machine to prune experiences that generate low rewards, thereby focusing the learning model on high-reward scenarios and improving convergence speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a static number of resources is dedicated to each user, then service level agreements are ensured, but resource efficiency decreases and idle resources increase
Solution Approach 1:
The patent implements dynamic resource allocation where the system continuously monitors workload characteristics and automatically adjusts resource assignment. Instead of static dedication, resources are dynamically reassigned based on real-time workload analysis, phase detection (IO-intensive vs compute-intensive), and predictive modeling, allowing the system to adapt to changing demands while maintaining SLA compliance.
Solution Approach 2:
The system incorporates feedback mechanisms through reinforcement learning agents that observe workload performance, SLA compliance metrics, and resource utilization. This feedback loop enables the system to learn from past allocations, identify patterns in workload behavior, and continuously optimize resource distribution to balance SLA adherence with resource efficiency.
2Reliability
If excessive resources are allocated to a single workload, then SLA compliance is achieved, but the number of workloads that can be served in parallel decreases
Solution Approach 1:
The system changes the parameters of resource allocation by using predictive modeling to estimate future workload requirements rather than allocating maximum resources upfront. By analyzing workload characteristics, historical patterns, and current system state, the system adjusts resource parameters dynamically, allocating only what is needed to meet SLAs while preserving resources for other workloads.
Solution Approach 2:
Instead of allocating excessive resources to ensure SLA compliance, the system applies partial action by allocating precisely the amount of resources needed based on predictive analysis. The reinforcement learning agents determine optimal allocation levels that are sufficient for SLA compliance but not excessive, thereby maximizing the number of concurrent workloads the system can support.
3Measurement precision
If reinforcement learning analyzes all experiences, then learning accuracy is improved, but training time and computational overhead increase
Solution Approach 1:
The patent extracts and removes low-value experiences from the training dataset using the experience pruning module. By analyzing experience characteristics and identifying those with minimal learning value (redundant, obviously suboptimal, or low-impact experiences), the system eliminates these from the training process, significantly reducing training time and computational overhead while preserving the accuracy-contributing experiences.
Solution Approach 2:
The system applies local quality by treating different experiences differently based on their value. Rather than uniformly processing all experiences, the pruning module selectively retains high-value experiences for training while discarding low-value ones. This differentiated approach maintains learning accuracy by preserving critical experiences while reducing overall processing burden.
Data Source
AI summary
One example method includes defining experiences for a workload that are to be analyzed at a first machine-learning (ML) model. The experiences define an association between the workload and microservices having computing resources that execute the workload. A probability of using each of the microservices of the experiences to execute the workload is generated at a second ML mode. A determination is made of which of the experiences have a probability that indicates that the experience will generate a low reward when analyzed by the first ML model. The experiences that generate the low reward are removed from the experiences to be analyzed at the first ML model. The experiences that have not been removed are analyzed at the first ML model to determine which experience includes microservices that should be used to execute the workload.