Accelerator Power Management via Usage Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current power management systems for accelerators in data centers are largely reactive and cannot predict when accelerators will be needed, leading to inefficient power usage and performance degradation due to deep sleep states and unnecessary power consumption.
Innovation Solution
Implementing tighter coupling between power management points at various levels, including application software and schedulers, to predict imminent accelerator usage and adjust power states dynamically based on workload thresholds and scheduler observations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If accelerators are placed in deep sleep states to reduce power consumption, then power efficiency is improved, but wake-up time increases and performance degrades
Solution Approach 1:
The system performs preliminary actions by predicting future accelerator usage based on scheduler observations and workload patterns. When usage is predicted, the accelerator is woken up in advance and kept in a ready state, eliminating the need for deep sleep transitions and reducing wake-up time penalties.
Solution Approach 2:
The power management system dynamically adjusts accelerator power states based on real-time predictions rather than using static deep sleep states. The accelerator transitions to intermediate power states that balance power consumption with readiness requirements, avoiding the extreme of deep sleep when quick response is needed.
2Productivity
If accelerators remain in active states to reduce wake-up time, then performance is improved, but power consumption increases
Solution Approach 1:
Instead of keeping accelerators permanently active, the system performs preliminary wake-up actions only when usage is predicted. This allows the accelerator to be ready when needed without maintaining continuous active state, reducing unnecessary power consumption during idle periods.
Solution Approach 2:
The system implements feedback loops where scheduler observations and workload patterns are continuously monitored to predict accelerator usage. This feedback enables intelligent decision-making about when to wake up the accelerator, balancing performance requirements with power efficiency by activating only when actually needed.
3Device complexity
If reactive power management is used to simplify system complexity, then device complexity is reduced, but power management efficiency deteriorates
Solution Approach 1:
The system introduces feedback mechanisms that monitor scheduler behavior and workload patterns to predict accelerator usage. This feedback enables proactive power management decisions that improve efficiency without requiring complex manual configuration, as the system learns and adapts to usage patterns automatically.
Solution Approach 2:
The power management system serves itself by automatically predicting accelerator usage based on observed workload patterns and making autonomous decisions about power state transitions. This self-service approach reduces the need for external control complexity while improving power management efficiency through intelligent automation.
Data Source
AI summary
A machine-readable storage medium having program code that when processed by one or more processing cores causes a method to be performed. The method includes determining from program code that is scheduled for execution and/or is being scheduled for execution that an accelerator is expected to be invoked by the program code. The program code to implement one or more application software processes. The method also includes, in response to the determining, causing the accelerator to wake up from a sleep state before the accelerator is first invoked from the program code's execution.