GPU-CPU Function Callback via Shared Virtual Memory Deque
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous processing systems lack a mechanism for the Graphics Processing Unit (GPU) to offload work back onto the Central Processing Unit (CPU), limiting efficiency in executing sequential or irregular tasks that are not suited for parallel execution.
Innovation Solution
Implementing function callback requests between a GPU and a CPU using shared virtual memory (SVM) with double-ended queues, allowing the GPU to request CPU execution of specific workload portions through pointers and completion mechanisms, and utilizing continuation kernels to manage task resumption without blocking the GPU.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the GPU is used to offload computational tasks from the CPU, then processing performance and energy efficiency are improved for parallel intensive tasks, but there is no mechanism for the GPU to offload work back onto the CPU for sequential or irregular tasks
Solution Approach 1:
The patent introduces a callback mechanism as an intermediary communication channel between the GPU and CPU. When the GPU encounters a task suitable for CPU execution (sequential or irregular work), it can issue a callback request through a predefined interface, allowing the CPU to execute the appropriate code and return results. This resolves the contradiction by enabling bidirectional task distribution while maintaining the performance benefits of GPU offloading.
2Extent of automation
If function calls from CPU to GPU are implemented, then the CPU can offload work to the GPU for parallel computations, but there is no corresponding mechanism for the GPU to request CPU execution for sequential or irregular tasks
Solution Approach 1:
The patent applies inversion by reversing the traditional unidirectional CPU-to-GPU function call model. Instead of only the CPU initiating work on the GPU, the GPU is now empowered to initiate callback requests back to the CPU when it encounters suitable tasks. This creates a symmetric, bidirectional communication model where either processor can request execution on the other, completing the communication mechanism.
3Use of energy by moving object
If the GPU executes parallel intensive tasks, then energy consumption is reduced compared to CPU execution, but sequential or irregular control path tasks cannot be efficiently executed on the GPU
Solution Approach 1:
The patent introduces dynamic task distribution where the execution venue (CPU or GPU) is determined at runtime based on task characteristics. The GPU can dynamically identify sequential or irregular tasks during execution and callback to the CPU for those specific operations, while continuing to handle parallel tasks independently. This dynamic adaptation allows the system to optimize energy consumption for parallel tasks while maintaining compatibility with sequential tasks through selective CPU offloading.
Data Source
AI summary
Generally, this disclosure provides systems, devices, methods and computer readable media for implementing function callback requests between a first processor (e.g., a GPU) and a second processor (e.g., a CPU). The system may include a shared virtual memory (SVM) coupled to the first and second processors, the SVM configured to store at least one double-ended queue (Deque). An execution unit (EU) of the first processor may be associated with a first of the Deques and configured to push the callback requests to that first Deque. A request handler thread executing on the second processor may be configured to: pop one of the callback requests from the first Deque; execute a function specified by the popped callback request; and generate a completion signal to the EU in response to completion of the function.