Multi-GPU Neural Network Power Management via Phase Misalignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-GPU systems implementing neural networks face performance limitations due to shared common power supplies with fixed power limits, leading to inefficient power usage and extended execution times when power consumption exceeds the limit.
Innovation Solution
The implementation of a phase misalignment scheme, where GPUs delay the start of execution for certain task phases, allowing power consumption to be spread out and reducing peak power demand, thereby optimizing power usage within the given power limit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple GPUs work together to implement a neural network with a common power supply, then the system can process neural network tasks, but the performance is limited by the total power that all GPUs have to share
Solution Approach 1:
The patent implements periodic action by having GPUs cycle through task execution phases in a coordinated manner. Each GPU executes tasks in periodic cycles, and by staggering the phases (e.g., GPU 0 executes phase 0 while GPU 1 executes phase 1), the system distributes power consumption over time rather than concentrating it at simultaneous peaks, thereby maintaining high productivity within power constraints
Solution Approach 2:
The system dynamically adjusts the execution phases of multiple GPUs based on real-time power consumption monitoring. When power consumption approaches the limit, the system dynamically reschedules task phases to misalign GPU operations, spreading peak power demands across different time windows. This dynamic phase adjustment allows the system to maintain high processing performance while adapting to power constraints
2Loss of time
If GPUs execute tasks simultaneously to maximize processing speed, then execution time is reduced, but peak power consumption exceeds the power supply limit
Solution Approach 1:
The patent segments the neural network task execution into distinct phases (e.g., data loading, computation, memory access) and divides these phases across multiple GPUs in a staggered manner. Instead of all GPUs executing all phases simultaneously, each GPU executes different phases at different times, segmenting the peak power demand across multiple time windows while maintaining overall execution efficiency
Solution Approach 2:
The system performs preliminary scheduling of task phases to determine the optimal execution sequence for each GPU. By pre-calculating phase assignments and timing, the system ensures that peak power consumption is distributed evenly across the power supply cycle, preventing exceedance of power limits while maintaining minimal execution time through efficient phase coordination
3Productivity
If the power supply operates continuously at high power to maintain processing speed, then productivity is maintained, but the power supply reaches its limit and throttles performance
Solution Approach 1:
The system implements feedback mechanisms where each GPU monitors its own power consumption and communicates with other GPUs about phase execution status. When power consumption approaches the supply limit, the feedback loop triggers phase rescheduling to misalign GPU operations, distributing the power load. This feedback-driven adjustment maintains processing speed by preventing power supply throttling while ensuring stable operation within power constraints
Data Source
AI summary
Systems, apparatuses, and methods for managing power consumption for a neural network implemented on multiple graphics processing units (GPUs) are disclosed. A computing system includes a plurality of GPUs implementing a neural network. In one implementation, the plurality of GPUs draw power from a common power supply. To prevent the power consumption of the system from exceeding a power limit for long durations, the GPUs coordinate the scheduling of tasks of the neural network. At least one or more first GPUs schedule their computation tasks so as not to overlap with the computation tasks of one or more second GPUs. In this way, the system spends less time consuming power in excess of a power limit, allowing the neural network to be implemented in a more power efficient manner.