Wavefront Resource Virtualization for GPU Scheduling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Graphics Processing Units (GPUs) face challenges in scheduling flexibility for efficient general-purpose computing, particularly due to limitations in managing hardware resources and buffer space, leading to inefficiencies in task execution and potential deadlocks.

Innovation Solution

Implementing a processor with hardware logic that allows wavefronts to yield resources, enabling the scheduling of new wavefronts before the previous one completes, and using a dispatcher to manage resource allocation and restart contexts, thereby improving scheduling flexibility and resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a wavefront uses hardware resources exclusively until completion, then resource allocation is simple, but scheduling flexibility is poor

Engineering Contradiction:

Improvescheduling flexibilityVSAvoidresource management complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic wavefront switching by allowing the dispatcher to preempt currently executing wavefronts and switch to different wavefronts based on scheduling needs. This enables the system to adapt resource allocation dynamically rather than committing resources to a single wavefront until completion, directly improving scheduling flexibility while managing complexity through structured state saving/restoration mechanisms

Inventive Principle:

Principle #15Dynamics

2Productivity

If multiple producer tasks are scheduled to increase throughput, then processing capacity increases, but buffer space may run out causing inefficiency

Engineering Contradiction:

Improvetask processing throughputVSAvoidtask execution delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The dispatcher monitors buffer space utilization and uses this feedback to dynamically adjust wavefront scheduling decisions. When buffers are nearing capacity, the dispatcher can preempt producer wavefronts and switch to consumer wavefronts that free up buffer space, preventing delays while maintaining high throughput through intelligent, condition-based scheduling adjustments

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If wavefront state is saved to restore context, then scheduling flexibility improves, but memory usage increases

Engineering Contradiction:

Improvewavefront switching capabilityVSAvoidmemory consumption

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential wavefront state information (register values, program counter, instruction state) that is necessary for restoration, rather than saving entire memory spaces or unnecessary data. This selective extraction approach enables wavefront switching capability while minimizing memory consumption by storing only the critical context needed to resume execution

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS10360652B2Wavefront resource virtualization

Publication Date: 2019.07.23 ADVANCED MICRO DEVICES INC

AI summary

A processor comprising hardware logic configured to execute of a first wavefront in a hardware resource and stop execution of the first wavefront before the first wavefront completes. The processor schedules a second wavefront for execution in the hardware resource.