Per-Line Cache Locking for Work Stealing in Heterogeneous Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The performance of heterogeneous processing systems is hindered by the conflict between work stealing and memory scoping requirements, as conventional memory scoping methods incur performance costs such as global data flushing and RMW operation blocking, which limit scalability and parallel execution.
Innovation Solution
Implementing per-address lock, flush, invalidate, and RMW commands using locking tables within caches to allow work stealing across different memory scopes while maintaining memory consistency, allowing for concurrent execution of operations and reducing the need for global synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If work stealing is implemented between compute units with different memory scopes, then resource utilization and parallelism are improved, but memory consistency guarantees are compromised
Solution Approach 1:
The patent segments the memory scope into work-group scope and device scope, allowing work stealing to occur within the work-group scope while maintaining separate consistency guarantees for device scope operations. This segmentation enables fine-grained control over where work stealing is permitted without compromising global memory consistency.
Solution Approach 2:
The patent applies different memory scope rules to different locations in the memory hierarchy. Work-group scoped operations allow work stealing with relaxed consistency requirements, while device scoped operations maintain strict consistency guarantees. This local differentiation resolves the contradiction by allowing work stealing where safe and maintaining consistency where required.
2Reliability
If global memory scope is used to enforce memory consistency, then consistency guarantees are maintained, but synchronization performance degrades due to slower global memory access
Solution Approach 1:
The patent divides memory operations into work-group scope and device scope categories. Work-group scoped operations use faster local synchronization mechanisms within compute units, while device scoped operations use slower global memory synchronization. This segmentation allows the system to use the fastest appropriate synchronization mechanism for each operation type.
Solution Approach 2:
The patent applies different synchronization qualities to different memory scopes. Local work-group scoped operations achieve consistency through fast compute unit-level mechanisms, while global device scoped operations use slower but more comprehensive global memory barriers. This resolves the speed-consistency tradeoff by matching synchronization strength to the actual consistency requirements of each operation.
3Productivity
If memory scope is increased to enable work stealing, then resource utilization improves, but performance cost increases due to global data flushing and RMW operation blocking
Solution Approach 1:
The patent segments the memory consistency domain into work-group scope and device scope, allowing work stealing to proceed at the work-group level without triggering global data flushing. This segmentation confines the impact of work stealing to local operations, avoiding the performance penalty of global synchronization.
Solution Approach 2:
The patent applies memory scope promotion selectively only when work stealing between different compute units is required, rather than applying global memory scope universally. This partial application of the scope promotion reduces unnecessary data flushing and operation blocking, maintaining high performance for workloads that don't require cross-compute-unit work stealing.
Data Source
AI summary
A processing system includes one or more first caches and one or more first lock tables associated with the one or more first caches. The processing system also includes one or more processing units that each include a plurality of compute units for concurrently executing work-groups of work items, a plurality of second caches associated with the plurality of compute units and configured in a hierarchy with the one or more first caches, and a plurality of second lock tables associated with the plurality of second caches. The first and second lock tables indicate locking states of addresses of cache lines in the corresponding first and second caches on a per-line basis.