Device-Side Kernel Enqueuing via Ring Buffer and CSP
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional host-side kernel enqueuing methods incur significant overhead, undermining performance as they require direct involvement and subsequent return of control, leading to inefficiencies in device-side execution.
Innovation Solution
An apparatus for enqueuing kernels on a device-side, incorporating a Memory Access Unit (MXU) and a Command Stream Processor (CSP), allows the CSP to execute commands from a ring buffer, enabling the Execution Unit (EU) to allocate space for hardware threads and write commands directly into the buffer, with a Physical-thread ID buffer managing thread profiles and ready flags to ensure efficient execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If host-side enqueues kernel to device-side, then control can be transferred and execution can be initiated, but significant overhead is consumed and performance is undermined
Solution Approach 1:
The device-side executes kernels autonomously by enqueuing child kernels directly on the device without requiring continuous host-side intervention. The Execution Unit allocates ring buffer space and writes commands asynchronously, enabling the system to serve itself and eliminate host-device control transfer overhead.
Solution Approach 2:
The kernel execution is divided into parent kernels executed on the host and child kernels executed on the device. This segmentation allows independent execution of computational tasks on the device while the host manages only the orchestration, reducing the overhead of continuous host-side control transfers.
2Productivity
If host-side completes kernel execution before enqueuing next kernel, then control transfer is clean, but concurrency is reduced and processing efficiency decreases
Solution Approach 1:
The device-side maintains continuous kernel enqueuing capability without waiting for parent kernel completion. The Execution Unit can allocate ring buffer space and write child kernel commands asynchronously, ensuring useful processing actions continue uninterrupted while maintaining manageable execution control through structured buffer management.
Data Source
AI summary
An apparatus for enqueuing kernels on a device-side is introduced to incorporate with at least a MXU (Memory Access Unit) and a CSP (Command Stream Processor): The CSP, after receiving a first command from the MXU, executes commands of a ring buffer, thereby enabling an EU (Execution Unit) to direct the MXU to allocate space of the ring buffer for a first hardware thread and subsequently write second commands of the first hardware thread into the allocated space of the ring buffer according to an instruction of a kernel.