Accelerator Load Balancing via Dynamic Queue Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In systems with multiple accelerators, unbalanced load distribution leads to underutilization of hardware accelerators, as existing technologies fail to efficiently manage and balance the workload across these accelerators, particularly in multi-user and multi-task operating systems.
Innovation Solution
A balancing device comprising a loading monitor and balancer that tracks busy periods and execution cycles of accelerators, adjusting command queues to equalize duty cycles and ensure balanced workload distribution by moving queues between accelerators when necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple accelerators are placed in the same chip to enable sharing among multiple applications, then the system can support parallel execution of multiple applications, but the load distribution becomes unbalanced leading to underutilization of hardware accelerators
Solution Approach 1:
The patent implements a feedback mechanism where the loading monitor continuously monitors the busy periods of multiple accelerators, and the loading balancer uses this feedback information to dynamically adjust command queue assignments. This closed-loop control ensures that load distribution is continuously optimized based on actual accelerator utilization, resolving the contradiction between parallel execution capability and load balance.
Solution Approach 2:
The patent introduces dynamic load balancing by continuously monitoring accelerator busy periods and adjusting command queue assignments in real-time. Instead of static assignment, the system adapts to changing workload conditions, dynamically redistributing tasks to maintain balanced utilization across all accelerators while preserving parallel execution capabilities.
2Productivity
If accelerators are not well dispatched, then the loads on accelerators become unbalanced, but implementing load balancing requires additional monitoring and control mechanisms
Solution Approach 1:
The loading monitor and loading balancer components serve multiple functions: they monitor busy periods, calculate duty cycles, make balancing decisions, and manage command queue assignments. This multi-functionality consolidates the control infrastructure, reducing overall system complexity while achieving effective load balancing and maximizing accelerator utilization.
Solution Approach 2:
The system implements self-service load balancing where the loading monitor and balancer automatically detect imbalances and redistribute workloads without external intervention. The accelerators themselves provide the necessary busy period information, and the control logic autonomously makes balancing decisions, reducing the need for complex external management infrastructure.
Data Source
AI summary
A balancing device, which is configured to balance a first duty cycle of a first accelerator and a second duty cycle of a second accelerator, includes a loading monitor and a loading balancer. The loading monitor is configured to monitor a first busy period of the first accelerator and a second busy period of the second accelerator. The loading balancer calculates the first duty cycle and the second duty cycle according to the first busy period and the second busy period, and moves at least one command queue of the first accelerator and the second accelerator according to the first duty cycle and the second duty cycle, such that the first duty cycle is close to the second duty cycle.