Data Center Power Control Using Joint Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data centers face significant power consumption challenges, with cooling systems accounting for a large portion of the costs, and existing deterministic approaches are not scalable or adaptable to changing environments, failing to optimize power usage efficiently while maintaining performance benchmarks.
Innovation Solution
A reinforcement learning framework is implemented to optimize power consumption by aggregating parameters from heterogeneous IT resources and cooling systems, training agents to generate controls that reduce power usage while maintaining performance benchmarks, using a Multi-Agent Deep Deterministic Policy Gradient algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional deterministic approaches are used to control data center operations, then system simplicity and ease of implementation are maintained, but power consumption optimization and adaptability to changing environments are insufficient
Solution Approach 1:
The patent implements dynamic control by training reinforcement learning agents that continuously learn and adapt to changing data center environments. The agents adjust cooling system parameters and IT resource allocations in real-time based on observed states, transforming the static deterministic control into a dynamic adaptive system that optimizes power consumption while responding to environmental changes
Solution Approach 2:
The reinforcement learning agents autonomously optimize data center operations without requiring external intervention. The agents self-train using historical data and continuously improve their control strategies, enabling the system to self-adjust cooling and resource allocation to minimize power consumption while maintaining performance benchmarks
2Reliability
If cooling systems operate at high capacity to maintain temperature benchmarks, then temperature control reliability is improved, but power consumption increases significantly
Solution Approach 1:
The reinforcement learning agents dynamically adjust cooling system parameters such as temperature setpoints, fan speeds, and chiller capacities based on real-time conditions. By optimizing these parameters rather than maintaining fixed high-capacity operation, the system achieves reliable temperature control while significantly reducing cooling system power consumption
3Productivity
If IT resources are distributed to maximize performance, then workload performance benchmarks are met, but overall power consumption of the data center increases
Solution Approach 1:
The reinforcement learning framework performs multiple optimization functions simultaneously: it coordinates IT resource allocation, manages cooling system operation, and optimizes overall power consumption. This multi-functional approach enables the system to meet performance benchmarks while reducing total power consumption by considering the interplay between compute workload and cooling requirements
Data Source
AI summary
An apparatus comprises a processing device configured to obtain first parameters characterizing an operating state of information technology (IT) resources of a data center and second parameters characterizing an operating state of cooling systems of the data center, to determine an overall operating state of the data center by aggregating the first and second parameters, to identify a power consumption profile based on the overall operating state, and to perform a joint training of first and second reinforcement learning agents based on the overall operating state and the power consumption profile. The processing device is also configured to generate first controls for the heterogeneous IT resources utilizing the first reinforcement learning agent and second controls for the cooling systems utilizing the second reinforcement learning agent, the first and second controls being configured to reduce power consumption while maintaining specified performance benchmarks for workloads executing in the data center.