GPU Array Processing with Shared Thread Contributions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing separable two-dimensional Gaussian filter operations on graphics processing units (GPUs) is inefficient due to the need for multiple memory reads and writes, leading to increased latency and resource utilization, especially when using global memory.
Innovation Solution
Implement a method where threads collaborate to share contributions across neighboring sections of an array, reducing the need for repeated memory access by leveraging local memory and optimizing thread communication within workgroups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each thread reads all kernel values from global memory independently, then each thread can complete its filtering operation, but the number of memory reads increases significantly (each value read multiple times)
Solution Approach 1:
Multiple threads that need to read the same kernel values from global memory merge their reads into a single read operation. The first thread in a group reads all required kernel values once into shared memory, and subsequent threads in the same group read from this shared copy, eliminating redundant memory accesses while ensuring all threads get the correct data for their filtering operations.
Solution Approach 2:
Kernel values are pre-loaded into shared memory by the first thread before other threads need them. This preliminary action of reading data once and storing it in a faster shared memory location allows subsequent threads to access the data quickly without repeating the slow global memory read operation, thus improving processing speed while maintaining correctness.
2Ease of operation
If a separable two-dimensional Gaussian filter operation is performed using two phases with independent thread assignments, then the operation can be completed, but the number of memory writes increases (two sets of writes)
Solution Approach 1:
The patent merges the write operations of multiple threads into a single write operation per memory location. Instead of each thread independently writing its result to global memory (causing multiple writes to the same location), threads write to shared memory first, and then a single thread performs the final write to global memory, reducing the total number of memory write operations while maintaining the two-phase separable filter implementation.
Solution Approach 2:
Shared memory acts as an intermediary between thread computations and global memory writes. Threads write their intermediate results to shared memory (a fast, local storage), and then a coordinating thread transfers these results to global memory in a single operation. This intermediary layer reduces the number of slow global memory write operations while preserving the simplicity of the two-phase filter approach.
3Quantity of substance
If threads access global memory for each filtering operation, then data can be read, but latency increases due to the distance and bandwidth constraints of global memory
Solution Approach 1:
The patent segments memory into two distinct layers: shared memory (fast, local to the thread group) and global memory (slower, distant). Frequently accessed kernel values are segmented and stored in shared memory, while less frequently accessed data remains in global memory. This segmentation allows threads to access critical data quickly from shared memory, reducing latency, while still maintaining access to the full dataset through global memory when needed.
Solution Approach 2:
Kernel values are preliminarily loaded from global memory into shared memory at the beginning of the operation. This preliminary action moves data from the slow global memory to the fast shared memory before threads need to perform their filtering computations, ensuring data availability while minimizing the latency of subsequent accesses by reading from the faster shared memory instead of global memory.
Data Source
AI summary
A computer-implemented method of performing an operation on an array of values at a processing unit so as to perform a phase of the operation. For each of one or more one-dimensional sequences of values of the array of values a respective section of values of the one-dimensional sequence of values is assigned to each of a plurality of threads, and a first thread of the plurality of threads determines at least one contribution, from the section of values assigned to the first thread, to the phase of the operation that is to be completed by a second thread of the plurality of threads for a neighbouring section of values of the one-dimensional sequence of values. The at least one contribution is written to a memory, and a second thread of the plurality of threads reads the at least one contribution from the memory. and completes the phase of the operation for the neighbouring section of values assigned to the second thread in dependence on the at least one contribution read from the memory in order to generate a section of processed values.