A3C Cloud Resource Allocation for Job Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud data centers face challenges in adaptive resource allocation due to dynamic system states and diverse user demands, leading to inefficient resource utilization, high energy consumption, and increased maintenance costs, with existing solutions struggling to balance short-term and long-term job scheduling and energy efficiency.
Innovation Solution
A deep reinforcement learning-based method using an asynchronous advantage actor-critic (A3C) algorithm for cloud data center resource allocation, which defines a state space, action space, and reward function as a Markov decision process, and asynchronously updates policy parameters among multiple DRL agents to optimize job scheduling, reducing variance and improving training efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If rule-based resource allocation policies are used to meet instant user demands, then short-term job scheduling is improved, but long-term adaptability and energy efficiency deteriorate
Solution Approach 1:
The system employs reinforcement learning agents that autonomously learn and adapt resource allocation policies through continuous interaction with the cloud environment. The agents self-optimize by receiving rewards based on allocation outcomes, eliminating the need for manual policy formulation and enabling automatic adaptation to dynamic conditions while maintaining rapid response to user demands.
2Adaptability or versatility
If value-based deep reinforcement learning is used to process large action spaces, then policy learning is achieved, but training efficiency deteriorates due to low convergence speed
Solution Approach 1:
The patent combines multiple reinforcement learning agents with different expertise into a unified multi-agent system. Each agent specializes in specific aspects of resource allocation (e.g., CPU, memory, storage), and their policies are integrated through ensemble methods. This merging approach enables the system to handle large action spaces effectively while improving training efficiency through parallel learning and knowledge sharing among agents.
Solution Approach 2:
The resource allocation problem is segmented into multiple sub-problems, with dedicated reinforcement learning agents handling specific resource types or allocation scenarios. This segmentation reduces the complexity of the action space for each agent, enabling faster convergence while maintaining the ability to handle comprehensive resource allocation through coordinated agent actions.
3Adaptability or versatility
If heterogeneous computing and storage resources are allocated to meet diverse user demands, then service coverage is improved, but resource allocation complexity increases
Solution Approach 1:
The patent implements a universal resource allocation framework that handles multiple resource types (CPU, memory, storage, networking) through a common multi-agent reinforcement learning system. Each agent can allocate different resource types using the same learning paradigm, reducing allocation complexity while maintaining the ability to meet diverse user demands through coordinated multi-resource management.
Data Source
AI summary
The present invention relates to a deep reinforcement learning-based cloud data center adaptive efficient resource allocation method. First, an operation (a scheduling job) is selected according to a score evaluated by critic (an evaluation operation) by using an actor parameterized policy (resource allocation). Then, a resource allocation policy is updated through gradient boosting, and a variance of a policy gradient is reduced by using an advantage function, to improve training efficiency; and wide simulation experiments are performed by using real data from a Google cloud data center. Compared with two advanced DRL-based cloud resource allocation methods and five classic cloud resource allocation methods, the method provided in the present invention has higher quality of service (QoS) in terms of a delay and a job dropout rate and has higher energy efficiency.