Oversubscription Reinforcement Learning for Adaptive Cloud Resource Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing oversubscription policies in cloud computing struggle to balance efficient resource utilization with minimizing overload risks, as they often result in either overloading or underutilization due to challenges in predicting user demand and resource utilization behaviors.
Innovation Solution
An oversubscription reinforcement learner is trained using prototype learning and imitation learning to generate adaptive oversubscription rates, incorporating human feedback to refine policies and address noise and complexity in resource usage trajectories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If an aggressive oversubscription policy is applied, then resource utilization efficiency is improved, but the risk of overloading increases
Solution Approach 1:
The system dynamically adjusts oversubscription rates based on real-time resource usage patterns and learned trajectories. Instead of applying a static aggressive oversubscription policy, the reinforcement learner continuously adapts the oversubscription rate to current system conditions, allowing high utilization when safe and reducing it when overload risk is detected, thus resolving the contradiction between productivity and reliability
Solution Approach 2:
The reinforcement learning framework implements continuous feedback loops where the system monitors resource usage outcomes and uses this feedback to update its policy. The learner observes the consequences of previous oversubscription decisions and adjusts future rates accordingly, enabling the system to maintain high resource utilization while avoiding overload through learned experience and adaptive control
2Reliability
If a conservative oversubscription policy is applied, then the risk of overloading is reduced, but resource utilization efficiency deteriorates
Solution Approach 1:
The system transitions from static conservative policies to dynamic adjustment. The reinforcement learner starts with conservative rates but gradually learns to increase oversubscription as it accumulates experience and understands system patterns, allowing the system to achieve high reliability initially while progressively improving resource utilization efficiency through adaptive learning
Solution Approach 2:
The system performs preliminary learning during initialization and training phases, accumulating knowledge about resource usage patterns before deploying aggressive oversubscription. This preliminary action allows the system to establish safe baseline policies while preparing to optimize for higher efficiency once sufficient learning has occurred
3Adaptability or versatility
If traditional reinforcement learning is used, then adaptability is improved, but training time and computational complexity increase
Solution Approach 1:
The system performs preliminary clustering of resource usage trajectories before reinforcement learning training. By pre-organizing the state space into clustered groups, the system reduces the complexity of the learning problem, allowing the reinforcement learner to adapt to different workloads more quickly without sacrificing policy adaptability, thus reducing training time while maintaining versatility
Solution Approach 2:
The state space is segmented into clustered groups of similar trajectories. Instead of learning from all possible states uniformly, the system divides the complex state space into manageable clusters, enabling faster learning within each cluster while maintaining overall adaptability through the clustered structure. This segmentation reduces computational burden and training time
Data Source
AI summary
A computing system including one or more processing devices that train an oversubscription reinforcement learner at least in part by receiving computing resource usage trajectories. At the oversubscription reinforcement learner, the training further includes generating prototypes based at least in part on the computing resource usage trajectories. The training further includes, based at least in part on the prototypes, generating an oversubscription rate. The training further includes outputting a prototype feedback query and/or an oversubscription rate feedback query. The training further includes receiving a prototype feedback input and/or an oversubscription rate feedback input. Based at least in part on the computing resource usage trajectories, the prototypes, and the prototype feedback input and/or the oversubscription rate feedback input, the training further includes computing an objective function value and training the oversubscription reinforcement learner based at least in part on the objective function value.