IC Command Queue Scheduling for Compute Unit Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In heterogeneous computing platforms, the host processor incurs significant delays and inefficiencies due to the need to continually poll compute units (CUs) for availability before offloading commands, leading to increased latency and reduced performance.

Innovation Solution

Implementing a scheduling system within the integrated circuit (IC) that tracks CU availability and stores commands in a queue, allowing the host processor to send commands without waiting for CU readiness, and automatically transfers commands to available CUs, reducing host processor involvement and latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the host processor continually polls compute units for availability before offloading commands, then the host processor can ensure proper command scheduling, but the system experiences increased latency and reduced performance

Engineering Contradiction:

Improvecommand scheduling correctnessVSAvoidcommand offloading latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

An interrupt controller is introduced as an intermediary between compute units and the host processor. The interrupt controller receives availability notifications from compute units and manages interrupt signals to the host processor, eliminating the need for continuous polling while ensuring proper command scheduling through controlled interrupt-driven communication

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The compute units are enabled to autonomously generate interrupt notifications when they become available for new commands. This self-service mechanism allows compute units to actively signal their readiness state without requiring the host processor to continuously query them, reducing latency while maintaining scheduling correctness

Inventive Principle:

Principle #25Self-service

2Speed

If the host processor waits for compute unit readiness before sending commands, then command timing can be optimized, but the host processor throughput is reduced

Engineering Contradiction:

Improvecommand timing optimizationVSAvoidhost processor throughput

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

The host processor is decoupled from compute unit readiness checks, allowing it to continuously issue commands without waiting. Commands are placed in a command queue in advance, and the interrupt controller manages the actual timing of command delivery to compute units, enabling host processor throughput optimization while maintaining proper command timing through queued preliminary actions

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The host processor maintains continuous productive operation by issuing commands without interruption or waiting periods. The interrupt-driven architecture ensures that commands are continuously supplied to the command queue, and compute units are continuously fed commands as they become available, eliminating idle time and maintaining continuous useful action across the system

Inventive Principle:

Principle #20Continuity of useful action

3Loss of time

If interrupt-based notification is implemented, then latency is reduced and performance increases, but device complexity increases

Engineering Contradiction:

Improvecommand processing latencyVSAvoidinterrupt control architecture

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The interrupt controller serves as a manageable intermediary that centralizes interrupt management functionality. By consolidating interrupt signal generation, filtering, and routing in a single dedicated component, the complexity of interrupt-based notification is contained and organized, making the increased device complexity tractable while achieving the latency reduction benefits of interrupt-driven operation

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10705993B2Programming and controlling compute units in an integrated circuit

Publication Date: 2020.07.07 XILINX INC

AI summary

An integrated circuit (IC) can include a command queue having a plurality of slots corresponding to commands from a host processor for execution by a plurality of compute units of the IC and a command request register having a plurality of locations corresponding to the plurality of slots in the command queue. The command request register is configured to generate an interrupt indicating a new command stored within the command queue. The IC can include a controller configured to, in response to the interrupt from the command request register, determine a selected compute unit that is idle from the plurality of compute units to execute the new command. The IC can also include a compute unit direct memory access circuit configured to provide the new command to the available compute unit.