GPU Kernel Self-Enqueuing for Parallel Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current execution models for GPUs require a host processor to enqueue work for GPU execution, limiting the ability of GPUs to perform parallel computations independently and leading to inefficient resource management and complex application execution graphs.
Innovation Solution
Allowing GPUs to enqueue tasks and kernels for themselves, enabling data-parallel and task-parallel operations through built-in functions and the OpenCL runtime API, allowing kernels to execute out-of-order and independently manage execution dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a host processor is required to enqueue all work for GPU execution, then execution control and resource management are centralized, but GPU independence and parallel computation efficiency are reduced
Solution Approach 1:
The patent enables GPUs to enqueue their own work items and kernels without requiring host processor intervention. Compute units can autonomously create and manage their own execution queues, allowing independent parallel computation while reducing host-GPU communication overhead and improving overall system productivity
Solution Approach 2:
The execution model is segmented into host-enqueued work and device-enqueued work. This segmentation allows different parts of the system to operate independently - the host manages initial work submission while the GPU autonomously manages subsequent parallel computation tasks, resolving the contradiction between centralized control and distributed efficiency
2Adaptability or versatility
If kernels are enqueued by the host processor, then execution order is controlled centrally, but the ability to express complex execution patterns directly on the device is limited
Solution Approach 1:
The patent adds a new dimension to the execution model by allowing device-side kernel enqueueing. This creates a two-level enqueueing structure (host and device) that enables complex execution patterns to be expressed directly on the GPU while maintaining host-level control when needed, enhancing adaptability without sacrificing ease of use
3Productivity
If all work must be created by a host processor, then resource management is simplified centrally, but GPU autonomy and performance are limited
Solution Approach 1:
The patent implements self-service by enabling compute units to autonomously enqueue work items and kernels for execution. This increases GPU autonomy and performance by eliminating the bottleneck of host processor-mediated work creation, while the system maintains overall coordination through the unified execution model
Data Source
AI summary
Graphics processing units (GPUs) and other compute units are allowed to enqueue tasks for themselves by themselves, without needing a host processor to queue the work for the GPU. Built-in functions enable kernels to enqueue kernels for execution on a device. In some embodiments, ndrange kernels execute over an N-dimensional range to provide data-parallel operations. Task kernels provide task-parallel operations. In some embodiments, kernels may be defined using clang block syntax. The order of execution of commands on a compute unit may be constrained or allow execution of commands out-of-order. Compute units may control when kernels enqueued by the compute unit begins execution.