Accelerator Command Dispatch Using Pull-Based Task Graph Scheduling

Solution Overview

Problem

Current dispatch models for accelerator devices face challenges with load balancing and head-of-line blocking due to barriers indicating data dependencies, leading to stalled queue draining and inefficient task scheduling by the host.

Innovation Solution

Implementing a pull-based model where a command processor in the accelerator device uses a task graph to determine resolved data dependencies before loading tasks into hardware queues, avoiding head-of-line blocking and improving scheduling efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the host statically schedules tasks into hardware queues using barriers to indicate data dependencies, then data dependency management is achieved, but head-of-line blocking occurs and queue draining is stalled

Engineering Contradiction:

Improvedata dependency managementVSAvoidqueue draining efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

Instead of the host pushing tasks into queues with barriers (push model), the command processor pulls tasks from queues only when dependencies are resolved (pull model). This inverts the control flow, allowing the accelerator to autonomously manage task dispatch based on actual data availability, eliminating head-of-line blocking while maintaining dependency correctness

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The command processor autonomously monitors data dependencies and dispatches tasks without host intervention. The system serves itself by having the CP track dependency resolution status and automatically load ready tasks into queues, eliminating the need for the host to manually manage barriers and task scheduling

Inventive Principle:

Principle #25Self-service

2Reliability

If the host generates and loads barriers into hardware queues, then data dependency tracking is maintained, but scheduling efficiency deteriorates due to lack of accelerator knowledge about dependencies

Engineering Contradiction:

Improvedata dependency trackingVSAvoidscheduling efficiency

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The command processor autonomously monitors data dependencies and dispatches tasks without host intervention. The system serves itself by having the CP track dependency resolution status and automatically load ready tasks into queues, eliminating the need for the host to manually manage barriers and task scheduling

Inventive Principle:

Principle #25Self-service

3Reliability

If barriers are used to indicate data dependencies in hardware queues, then data dependency enforcement is achieved, but load balancing becomes difficult and queues become blocked

Engineering Contradiction:

Improvedata dependency enforcementVSAvoidload balancing capability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

Instead of the host pushing tasks into queues with barriers (push model), the command processor pulls tasks from queues only when dependencies are resolved (pull model). This inverts the control flow, allowing the accelerator to autonomously manage task dispatch based on actual data availability, eliminating head-of-line blocking while maintaining dependency correctness

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The system transitions from static host-generated barriers to dynamic command processor decisions. The CP continuously evaluates dependency resolution status and adaptively dispatches tasks, allowing load balancing to respond to real-time data availability rather than following predetermined host schedules

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12578997B2Compiler-directed graph-based command dispatch for accelerators

Publication Date: 2026.03.17 ADVANCED MICRO DEVICES INC

AI summary

Embodiments herein describe a pull-based model to dispatch tasks in an accelerator device. That is, rather than a push-based model where a connected host pushes tasks into hardware (HW) queues in the accelerator device, the embodiments herein describe a pull-based model where a command processor (CP) loads tasks into the HW queues after any data dependencies have been resolved.