Processor Compute Unit Prioritization for Cache Thrashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The throughput of GPU applications does not scale linearly with the number of compute units due to cache thrashing behavior caused by memory-intensive applications competing for shared cache resources.
Innovation Solution
A method is introduced to dynamically designate a subset of compute units as priority units, allowing them to have exclusive access to the shared cache and prioritized access to main memory, while non-prioritized units bypass the shared cache and have lower memory access priority.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If all compute units access the shared cache simultaneously, then memory-intensive applications can utilize more compute units, but cache thrashing occurs which increases the LLC miss rate and reduces throughput
Solution Approach 1:
The patent segments the compute units into priority and non-priority groups, allowing differentiated access to the shared cache. This segmentation resolves the contradiction by enabling selective access that prevents cache thrashing while maintaining high throughput for critical compute units.
Solution Approach 2:
The patent applies local quality by giving different access priorities to different compute units based on their workload characteristics. Priority compute units receive preferential cache access while non-priority units have reduced access, optimizing overall system performance and preventing cache thrashing.
2Productivity
If the number of compute units is increased to improve throughput, then processing capacity increases, but cache thrashing behavior increases which causes the throughput to not scale linearly
Solution Approach 1:
The patent implements dynamic prioritization where compute units can be transitioned between priority and non-priority states based on runtime conditions. This dynamic approach allows the system to adapt to changing workload patterns and maintain optimal throughput scaling as compute units are added.
Solution Approach 2:
The patent uses feedback mechanisms to monitor cache performance metrics and adjust compute unit prioritization accordingly. This feedback loop enables the system to maintain linear throughput scaling by preventing cache thrashing through adaptive resource management.
Data Source
AI summary
Methods, devices, and systems for managing performance of a processor having multiple compute units. An effective number of the multiple compute units may be determined to designate as having priority. On a condition that the effective number is nonzero, the effective number of the multiple compute units may each be designated as a priority compute unit. Priority compute units may have access to a shared cache whereas non-priority compute units may not. Workgroups may be preferentially dispatched to priority compute units. Memory access requests from priority compute units may be served ahead of requests from non-priority compute units.