Run-time Memory Access Uniformity Checking for Vector Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern microprocessors' memory bandwidth is limited by contention among threads accessing the same memory locations, leading to inefficient memory access and reduced performance due to uniform memory access patterns in wavefronts.
Innovation Solution
A computing system that performs run-time checking for access uniformity among threads in a wave, replacing vector memory access instructions with scalar or cross-thread data sharing instructions to optimize memory access and improve bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads in a wave access the same memory location using vector memory access instructions, then SIMD parallelism is achieved, but memory bandwidth utilization deteriorates due to contention
Solution Approach 1:
Multiple threads that access the same memory location are merged into a single thread execution. The system detects uniform memory access patterns among threads in a wave and consolidates their memory accesses, so that instead of each thread independently accessing memory, one thread performs the access and results are shared with other threads. This eliminates redundant memory transactions and improves memory bandwidth utilization while preserving SIMD parallelism for computational operations.
2Speed
If vector memory access instructions are executed by multiple threads simultaneously, then execution speed is improved through parallelism, but memory latency increases due to contention
Solution Approach 1:
The patent merges memory access operations from multiple threads into a single operation when access patterns are uniform. By detecting that multiple threads in a wave are accessing the same memory location, the system consolidates these accesses into one memory transaction, significantly reducing memory latency. The computational parallelism is preserved by executing the merged result across all threads through data sharing mechanisms.
3Ease of operation
If each thread in a wave accesses memory independently, then thread autonomy is maintained, but power consumption increases due to redundant memory accesses
Solution Approach 1:
The system merges redundant memory access operations from multiple autonomous threads into a single access when uniformity is detected. This consolidation eliminates power-wasting redundant memory transactions while preserving thread autonomy for computational tasks. After the merged memory access completes, results are distributed back to the original threads, maintaining their operational independence while reducing overall power consumption.
Data Source
AI summary
Systems, apparatuses, and methods for performing run-time checking of access uniformity of vector memory access instructions are disclosed. A system includes a vector unit, a scalar unit, and a memory. The system performs a run-time check to determine if two or more threads of a wave have access uniformity to the memory prior to executing a vector memory access instruction for the wave on the vector unit. The system replaces the vector memory access instruction with a group of instructions responsive to determining that two or more threads of the wave have access uniformity to the memory. The group of instructions includes a scalar access instruction to memory followed by a cross-thread data sharing instruction. The scalar access instruction is executed on the scalar unit. Alternatively, the group of instructions can include a vector memory access instruction by only a single thread in each group having access uniformity.