Separable 2D Array Processing With Transposed Shared-Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing a separable two-dimensional Gaussian filter operation on a graphics processing unit is inefficient due to the need for multiple reads and writes to global memory, leading to high latency and slow processing times, as each value is read from memory multiple times during the two phases of the operation.
Innovation Solution
The method involves dividing a two-dimensional array of values into sub-arrays and performing the separable operation in phases, where each thread processes a one-dimensional sequence of values and writes processed values to memory, allowing for reduced memory access by using a write buffer array to determine memory locations and minimizing bank conflicts, thus optimizing memory access and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each value is assigned to a respective thread for filtering in a separable two-dimensional Gaussian filter operation, then the operation can be performed in parallel across the array, but the number of memory reads and writes increases significantly, leading to high latency and slow processing
Solution Approach 1:
The two-dimensional array is divided into multiple blocks, and each block is further divided into sub-arrays that can be processed by groups of threads. This segmentation allows parallel processing while reducing the memory access overhead by processing smaller chunks of data in parallel batches.
Solution Approach 2:
The patent introduces a block dimension to the traditional thread-per-value model. Instead of a single dimension of threads processing values sequentially or in simple parallel, the block structure creates an additional organizational dimension that allows for more efficient memory access patterns and reduced latency.
2Ease of operation
If each thread reads all values in the filter kernel from global memory for each value it processes, then the filtering operation can be completed, but the memory bandwidth is overwhelmed and processing speed decreases
Solution Approach 1:
The patent performs preliminary actions by loading filter kernel values into shared memory before the actual filtering operation. This allows threads to access kernel values from the fast shared memory rather than repeatedly reading from global memory, significantly improving processing speed.
Solution Approach 2:
Shared memory acts as an intermediary between global memory and thread registers. Instead of threads directly accessing global memory for each kernel value, the shared memory serves as a buffer that holds frequently accessed kernel data, reducing global memory access latency.
3Productivity
If processed values are written back to global memory after each phase, then the operation can proceed to the next phase, but the repeated writes increase latency and reduce overall processing efficiency
Solution Approach 1:
The patent merges the write-back operation with the next phase's read operation by using shared memory as a persistent storage location. Instead of writing to global memory and then reading back in the next phase, the intermediate results remain in shared memory, eliminating redundant memory transactions.
Solution Approach 2:
The use of shared memory maintains continuity of useful action by keeping intermediate filtering results available in fast memory throughout the entire two-phase operation. This eliminates the interruption caused by global memory write-latency cycles and keeps the processing pipeline continuously active.
Data Source
Figure 1A~1B
Figure 2~3
Figure 4~5
AI summary
A computer-implemented method of performing a separable operation on a two-dimensional array of values at a processing unit comprising a memory, the memory comprising a plurality of memory banks, wherein in each writing or reading step each memory bank can be written into or read from by only one respective thread, the method comprising: dividing the two-dimensional array of values into a plurality of two-dimensional sub-arrays of values; for each of the plurality of sub-arrays: performing, using a plurality of threads, an initial phase of the separable operation for said sub-array of values in order to generate a respective processed value for each value of said sub-array of values; each of the plurality of threads writing a respective first plurality of processed values to the memory over a plurality of writing steps, said first plurality of processed values corresponding to a one-dimensional sequence of values of said sub-array of values; each of the plurality of threads reading a respective second plurality of processed values from the memory over a plurality of reading steps, said second plurality of processed values corresponding to a perpendicular one-dimensional sequence of values of a sub-array of values in a transposed position within the array of values relative to said sub-array of values; and performing, using the plurality of threads, a subsequent phase of the separable operation for the plurality of processed values read by the plurality of threads in order to generate a respective output value for each value of the sub-array of values in the transposed position; wherein a respective processed value is written into each of the memory banks of the memory in at least one of the plurality of writing steps, and a respective processed value is read from each of the memory banks of the memory in at least one of the plurality of reading steps.