Hierarchical RL for NFV Server Power Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computing and cloud-hosted services face thermal limitations in data centers, leading to restricted processor performance due to power density issues, where traditional frequency adjustment methods are inefficient for managing core and uncore frequencies in Network Function Virtualization (NFV) servers.
Innovation Solution
A hierarchical reinforcement learning algorithm dynamically optimizes processor core and uncore frequencies using a contextual Bayesian optimization model for initial optimization and a Deep Neural Network-based model for refinement, balancing power consumption and performance metrics like packet processing throughput and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional frequency adjustment methods are used to manage processor power, then power consumption is reduced, but performance metrics such as packet processing throughput and latency deteriorate
Solution Approach 1:
The patent segments the frequency control into two independent parts: core frequency and uncore frequency. The hierarchical reinforcement learning algorithm independently optimizes each frequency component based on different workload characteristics, allowing power-efficient core frequency scaling while maintaining uncore frequency for packet processing performance in NFV servers
Solution Approach 2:
The patent implements dynamic frequency adjustment through reinforcement learning agents that continuously monitor system state and adapt frequency settings in real-time. The algorithm dynamically transitions between different frequency states based on workload demands, achieving both power savings and performance maintenance through adaptive control
2Productivity
If processor frequency is increased to maintain performance, then packet processing throughput is maintained, but power consumption increases
Solution Approach 1:
The patent changes the control parameters from single-frequency adjustment to dual-frequency adjustment (core and uncore). By modifying the frequency parameters independently based on workload type, the system achieves better performance-power tradeoffs than traditional single-parameter control methods
3Loss of energy
If thermal loading limits are imposed to manage power density, then power consumption is controlled, but processor performance is restricted
Solution Approach 1:
The patent implements feedback control through reinforcement learning agents that continuously monitor thermal and performance metrics. The agents use this feedback to make intelligent frequency adjustment decisions, allowing the system to operate near thermal limits safely while maximizing performance when conditions permit
Data Source
AI summary
Methods and apparatus for hierarchical reinforcement learning (RL) algorithm for network function virtualization (NFV) server power management. A first RL model at a first layer is trained by adjusting a frequency of the core of processor while performing a workload to obtain a first trained RL model. The trained RL model is operated in an inference mode while training a second RL model at a second level in the RL hierarchy by adjusting a frequency of the core and a frequency of processor circuitry external to the core to obtain a second trained RL model. Training may be performed online or offline. The first and second RL models are operated in inference modes during online operations to adjust the frequency of the core and the frequency of the circuitry external to the core while executing software on the plurality of cores of to perform a workload, such as an NFV workload.