Parallel Processor Memory Request Priority Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory divergence in parallel processors, where some threads within a wavefront experience cache hits while others miss, leading to significant performance issues due to varying memory request latencies, is a challenge that existing technologies have not adequately addressed.
Innovation Solution
A system that assigns priorities to memory requests of wavefronts on a per-lane basis using priority vectors, with dynamic updates based on detected events, to reorder and allocate shared resources effectively, reducing memory divergence by prioritizing critical path requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory requests are processed in parallel without priority assignment, then throughput is maintained, but memory divergence causes significant latency variation and performance degradation
Solution Approach 1:
The patent changes the parameter of memory request processing by introducing priority levels and reordering mechanisms. Memory requests are assigned priorities based on wavefront ID, lane ID, and request type, then processed in priority order rather than simple parallel execution, reducing latency variation caused by memory divergence
Solution Approach 2:
The patent implements dynamic priority assignment where priorities are not fixed but determined at runtime based on wavefront characteristics, lane information, and memory request types. This dynamic approach allows the system to adapt to different execution scenarios and minimize latency variation
2Loss of time
If memory requests are reordered and prioritized, then memory divergence is reduced, but system complexity increases due to priority management mechanisms
Solution Approach 1:
The patent segments the memory request processing by dividing wavefronts into individual lane requests and assigning priorities to each segment. This segmentation allows independent priority management for each memory request while maintaining overall system coordination through the prioritization logic
Solution Approach 2:
The patent performs preliminary priority assignment before memory requests are fully processed. By determining priorities early based on wavefront and lane characteristics, the system prepares the request queue in advance, reducing the need for complex runtime reordering and management overhead
Data Source
AI summary
Systems, apparatuses, and methods for implementing memory request priority assignment techniques for parallel processors are disclosed. A system includes at least a parallel processor coupled to a memory subsystem, where the parallel processor includes at least a plurality of compute units for executing wavefronts in lock-step. The parallel processor assigns priorities to memory requests of wavefronts on a per-work-item basis by indexing into a first priority vector, with the index generated based on lane-specific information. If a given event is detected, a second priority vector is generated by applying a given priority promotion vector to the first priority vector. Then, for subsequent wavefronts, memory requests are assigned priorities by indexing into the second priority vector with lane-specific information. The use of priority vectors to assign priorities to memory requests helps to reduce the memory divergence problem experienced by different work-items of a wavefront.