Separable 2D Array Processing With Conflict-Free Memory Bank Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing a separable two-dimensional Gaussian filter operation on a graphics processing unit is inefficient due to the need for multiple reads and writes to global memory, leading to high latency and slow processing times, as each value is read and written multiple times during the operation.
Innovation Solution
The method involves dividing a two-dimensional array of values into sub-arrays and performing the separable operation in phases, where each thread processes a one-dimensional sequence of values and writes processed values to memory in a way that minimizes memory bank conflicts, allowing for efficient reading and writing, thereby reducing the number of memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a simple approach is used where each value is assigned to a respective thread for filtering, then the operation can be performed in parallel, but the number of memory reads and writes increases significantly
Solution Approach 1:
The patent divides the two-dimensional array into multiple blocks, where each block is further divided into sub-blocks that can be processed independently by different thread groups. This segmentation allows parallel processing while reducing memory access overhead by keeping data locally within thread groups during intermediate calculations.
Solution Approach 2:
The patent introduces a new dimension of organization by arranging threads in a two-dimensional grid structure that matches the block structure of the data. This allows threads to efficiently access and share data within their block while maintaining parallel processing capability across the entire array.
2Ease of operation
If each thread reads all kernel values from global memory for every value it filters, then filtering can be performed independently, but global memory bandwidth is overwhelmed
Solution Approach 1:
The patent merges the filtering operations of multiple threads that work on adjacent values within the same block. By having these threads share a common set of kernel values loaded into shared memory, the system reduces redundant memory reads from global memory while maintaining independent filtering computation for each value.
Solution Approach 2:
The patent introduces shared memory as an intermediary between global memory and thread registers. Kernel values are loaded once into shared memory from global memory, then efficiently accessed by multiple threads during the filtering operation, reducing the burden on global memory bandwidth.
3Ease of manufacture
If the same one-dimensional Gaussian filter operation is performed sequentially (horizontally then vertically), then the separable operation is straightforward to implement, but processing time increases
Solution Approach 1:
The patent dynamically assigns different filtering directions (horizontal or vertical) to different thread groups based on their position in the two-dimensional thread grid. This allows both phases of the separable operation to proceed in parallel across different thread groups, significantly reducing total processing time while maintaining implementation clarity.
Solution Approach 2:
The patent performs preliminary organization of threads into a two-dimensional grid structure that corresponds to the block structure of the data array. This preliminary arrangement enables efficient parallel execution of both horizontal and vertical filtering phases without requiring complex dynamic scheduling during the actual filtering operation.
Data Source
AI summary
A separable operation on a two-dimensional array of values is performed at a processing unit. A two-dimensional array of values is divided into a plurality of sub-arrays of values. An initial phase of the separable operation is performed for a sub-array to generate a processed value for each value of the sub-array. Threads write a first plurality of processed values to the memory over a plurality of writing steps. Each of the threads reads a second plurality of processed values from the memory over a plurality of reading steps. A subsequent phase of the separable operation is performed for the processed values read by the threads to generate an output value for each value of the sub-array in a transposed position; wherein a respective processed value is written into each of the memory banks of the memory in at least one of the writing steps, and a respective processed value is read from each of the memory banks of the memory in at least one of the reading steps.