Compute Kernel Fusion for Off-Chip Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics and image processing pipelines often require frequent access to off-chip memory, leading to power consumption and performance issues due to large data sets not fitting in on-chip caches or local memory.
Innovation Solution
The technique involves fusing or interleaving compute kernels, maintaining intermediate data results in on-chip memory by identifying dependencies between producer and consumer kernels, and redefining work groups to optimize data processing and reduce external memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compute kernels process the whole data set sequentially with frequent off-chip memory access, then processing can be completed, but power consumption increases and performance decreases
Solution Approach 1:
The patent merges multiple compute kernels into a single fused kernel that processes multiple stages of image processing (e.g., decode, convert, scale) in one execution. This combining eliminates intermediate memory writes to external memory, keeping data in on-chip memory buffers throughout the processing pipeline, thereby reducing power consumption while maintaining processing throughput
2Productivity
If compute kernels access external memory for large data sets, then complete data processing is achieved, but data traffic to external memory increases causing performance degradation
Solution Approach 1:
The patent introduces a new dimension of processing by organizing computation around on-chip memory buffer dimensions rather than external memory access patterns. The fused kernel processes data through multiple transformation stages (spatial domain, frequency domain, scaling) while data remains in on-chip buffers, fundamentally changing the memory access paradigm from external sequential access to on-chip parallel processing
3Reliability
If intermediate data is written to external memory between kernel executions, then data persistence is ensured, but access time to external memory increases
Solution Approach 1:
The patent performs preliminary action by allocating and initializing on-chip memory buffers before the fused kernel execution begins. The kernel then operates on these pre-allocated buffers throughout its execution, eliminating the need for intermediate writes to external memory. Data persistence is maintained through the lifetime of the kernel execution within the on-chip buffer space
Data Source
AI summary
In a pipelined application having different stages of processing, such as a graphics application or an image processing application, there may be a dependence of one compute kernel upon another. Data associated with individual kernels needs to be written and read. A technique to minimize a need to read and write kernel data to external memory utilize at least one of fusing kernels, resizing workgroups, and performing interleaving of kernels.