GPU Memory Sharing via Kernel Execution Orders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) lack efficient management of memory resources, leading to excessive memory bandwidth consumption when executing multiple kernels that process the same or similar data, as they reload data into local memory for each kernel, resulting in inefficiencies and increased computational costs.
Innovation Solution
The implementation of kernel and workgroup execution orders, or 'instruction streams,' which allow data to be shared between workgroups of different kernels, enabling data to be loaded into local memory once and processed by multiple kernels sequentially, thereby reducing memory bandwidth consumption and arithmetic logic unit operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is reloaded into local memory for each kernel execution, then each kernel can process data independently, but memory bandwidth consumption increases significantly
Solution Approach 1:
The patent merges the data loading operations across multiple kernels by implementing a data loading mechanism that loads data once into local memory and makes it available to multiple kernels sequentially. This combining of loading operations eliminates redundant memory transfers while maintaining kernel execution independence through proper data management in local memory.
Solution Approach 2:
The patent applies preliminary action by loading data into local memory before kernel execution begins. The data is prepared and staged in local memory in advance, allowing subsequent kernels to access the data without additional memory bandwidth consumption. This preliminary data preparation resolves the contradiction by ensuring data availability while reducing memory operations.
2Use of energy by moving object
If data is loaded into local memory once for multiple kernels, then memory bandwidth consumption is reduced, but memory management complexity increases
Solution Approach 1:
The patent introduces an intermediary data management mechanism that sits between the global memory and local memory systems. This intermediary layer handles the complexity of managing data shared across multiple kernels, providing a simplified interface for kernel execution while managing the underlying memory sharing complexity. The intermediary abstracts the memory management complexity from the kernel execution flow.
3Productivity
If multiple kernels process the same data sequentially using shared local memory, then processing efficiency improves, but kernel execution time may increase due to sequential access
Solution Approach 1:
The patent implements continuity of useful action by keeping data continuously available in local memory across kernel boundaries. Instead of interrupting the data flow with repeated memory transfers, the system maintains continuous data availability in local memory, allowing kernels to execute sequentially without idle wait time for memory operations. This continuous data access pattern improves overall processing efficiency while managing execution time.
Data Source
AI summary
Aspects of this disclosure are directed to a method of processing data with a graphics processing unit (GPU). According to some aspects of the disclosure, the method comprises receiving input defining execution orders for a shader processor, wherein the execution orders comprise a plurality of kernel designations and a plurality of workgroup designations. The method may also include assigning workgroups of kernels identified in the plurality of workgroup designations and the plurality of kernel designations to the shader processor. The method may also include executing, by the shader processor, the workgroups of kernels identified in the plurality of workgroup designations and the plurality of kernel designations to process input data.