Cross-GPU Scheduling With a Dependency Table for Dependent Commands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GPU schedulers are unable to check dependencies across multiple GPUs, such as in rack-mounted systems, leading to inefficiencies in workload distribution and execution.
Innovation Solution
Implementing a primary GPU that maintains a dependency table to track and resolve cross-GPU dependencies, using doorbells to signal dependency resolution and schedule dependent commands for execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional GPU schedulers are used, then each GPU can execute commands independently, but dependencies across multiple GPUs cannot be checked leading to inefficiencies in workload distribution and execution
Solution Approach 1:
The patent merges the scheduling functions across multiple GPUs by implementing a primary GPU that maintains a centralized dependency table. This table tracks dependencies of commands assigned to different GPUs, allowing the system to check and resolve cross-GPU dependencies that conventional independent schedulers cannot detect, thereby improving workload distribution efficiency.
Solution Approach 2:
The dependency table acts as an intermediary data structure that mediates between multiple GPU schedulers. The primary GPU uses this table to monitor and resolve dependencies across GPUs, enabling coordinated execution without requiring complex inter-GPU communication protocols or increasing individual GPU scheduler complexity.
2Productivity
If a primary GPU maintains a dependency table to track cross-GPU dependencies, then dependent processes can be scheduled efficiently across multiple GPUs, but the system requires additional data structures and coordination mechanisms
Solution Approach 1:
The patent segments the scheduling functionality by designating one GPU as primary while others remain secondary. The primary GPU maintains the dependency table and handles dependency resolution, while secondary GPUs focus on command execution. This segmentation allows efficient cross-GPU scheduling without requiring all GPUs to have full scheduling capabilities, thus managing system complexity.
Solution Approach 2:
The primary GPU performs multiple functions: it acts as a regular GPU for command execution while simultaneously maintaining the dependency table and resolving cross-GPU dependencies. This multi-functionality reduces the need for separate dedicated scheduling hardware or software components, implementing efficient cross-GPU coordination within existing GPU architecture.
Data Source
AI summary
A primary processing unit includes queues configured to store commands prior to execution in corresponding pipelines. The primary processing unit also includes a first table configured to store entries indicating dependencies between commands that are to be executed on different ones of a plurality of processing units that include the primary processing unit and one or more secondary processing units. The primary processing unit also includes a scheduler configured to release commands in response to resolution of the dependencies. In some cases, a first one of the secondary processing units schedules the first command for execution in response to resolution of a dependency on a second command executing in a second one of the secondary processing units. The second one of the secondary processing units notifies the primary processing unit in response to completing execution of the second command.