GPU Array Processing With Thread-Sharing to Cut Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing a separable two-dimensional Gaussian filter operation on a graphics processing unit is inefficient due to the need for multiple reads and writes to global memory, leading to high latency and repeated memory access for each value.
Innovation Solution
A method where threads cooperate to determine contributions for neighboring sections of values, reducing the need for multiple memory accesses by sharing local memory and performing operations in parallel, thus minimizing the number of reads and writes to global memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each value is assigned to a respective thread for filtering operations, then parallel processing capability is improved, but the number of reads and writes to global memory increases significantly
Solution Approach 1:
The patent introduces local memory as an intermediary between global memory and processing threads. Threads first read values from global memory into local memory, then perform multiple filter operations using data from local memory without repeatedly accessing global memory. This mediator approach reduces global memory access latency while maintaining parallel processing throughput.
Solution Approach 2:
The patent implements preliminary action by having threads read and store required data into local memory before performing filter operations. By pre-loading data into faster local memory, the system eliminates repeated slow global memory accesses during the actual processing phases, thereby reducing overall latency while preserving parallel productivity.
2Manufacturing precision
If multiple phases of filter operations are performed on the same array, then comprehensive filtering is improved, but repeated memory accesses to the same values increase latency
Solution Approach 1:
Local memory serves as an intermediary that caches array values for use across multiple filter phases. Instead of repeatedly accessing global memory for each phase, threads utilize the cached data in local memory, maintaining filtering accuracy while dramatically reducing access latency across multiple operation phases.
Solution Approach 2:
The patent enables continuity of useful action by keeping filtered data in local memory across multiple phases. Threads continuously perform filtering operations using data already resident in local memory without interruption for global memory accesses, thereby maintaining high processing efficiency while achieving comprehensive multi-phase filtering accuracy.
3Manufacturing precision
If a separable two-dimensional Gaussian filter is implemented with multiple reads and writes to global memory, then filtering completeness is improved, but processing speed deteriorates
Solution Approach 1:
Local memory acts as an intermediary that enables complete separable two-dimensional Gaussian filtering without repeated global memory accesses. Threads perform both horizontal and vertical filtering phases using data cached in local memory, achieving filtering completeness while maintaining high processing speed by eliminating slow global memory round-trips.
Solution Approach 2:
The patent applies preliminary action by pre-loading the entire array into local memory before performing separable filtering operations. This upfront data preparation enables both horizontal and vertical filter phases to complete quickly using local memory, achieving comprehensive filtering while dramatically improving processing speed compared to repeated global memory accesses.
Data Source
Figure 1A~1B
Figure 2~3
Figure 4~5
AI summary
A computer-implemented method of performing an operation on an array of values at a processing unit, the method comprising: so as to perform a phase of the operation: for each of one or more one-dimensional sequences of values of the array of values: assigning a respective section of values of the one-dimensional sequence of values to each of a plurality of threads; and a first thread of the plurality of threads: determining at least one contribution, from the section of values assigned to the first thread, to the phase of the operation that is to be completed by a second thread of the plurality of threads for a neighbouring section of values of the one-dimensional sequence of values; and writing the at least one contribution to a memory; and the second thread of the plurality of threads: reading the at least one contribution from the memory; and completing the phase of the operation for the neighbouring section of values assigned to the second thread in dependence on the at least one contribution read from the memory in order to generate a section of processed values.