Flex-CAT Priority-Based Cache Line Eviction Algorithm
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud service providers face challenges in meeting service-level agreements by limiting performance variation and degradation of high-performance jobs caused by co-located low-performance jobs, as existing approaches lack dynamic priority-driven cache partitioning and lead to overprovisioning, underutilization, and performance degradation.
Innovation Solution
The Flexible Cache Allocation Technology (Flex-CAT) dynamically determines the optimal number of cache ways per set based on utilization, using model-specific registers to prioritize cache evictions and specify cache partitions, ensuring priority-driven LLC evictions and efficient resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multi-tenancy is used to achieve high system utilization, then system utilization improves, but performance degradation of high-priority jobs occurs due to co-located low-priority jobs
Solution Approach 1:
The cache is segmented into multiple ways within each set, where different ways can be allocated to different priority levels. This allows high-priority and low-priority jobs to coexist in the same cache set without interfering with each other's performance, thus maintaining both high system utilization and performance determinism
Solution Approach 2:
Different regions (ways) of the cache have different quality characteristics assigned based on priority. High-priority jobs are guaranteed access to specific cache ways, while low-priority jobs use other ways. This local differentiation ensures that high-priority jobs maintain performance determinism while the overall system achieves high utilization through multi-tenancy
2Reliability
If cache partitioning is implemented to protect high-priority jobs, then performance determinism improves, but system utilization decreases due to overprovisioning
Solution Approach 1:
The cache allocation is dynamic rather than static. The system continuously monitors cache usage and dynamically adjusts the number of ways allocated to high-priority jobs based on actual demand. When high-priority jobs need performance guarantees, they receive dedicated ways; when utilization is low, the allocation can be reduced, thus avoiding overprovisioning while maintaining performance determinism when needed
3Device complexity
If static cache allocation is used, then device complexity is reduced, but adaptability to different workload priorities deteriorates
Solution Approach 1:
The cache allocation system is self-managing through automated priority tracking and dynamic way allocation. The system automatically monitors job priorities, tracks cache usage patterns, and adjusts allocations without requiring complex external control mechanisms. This maintains relatively simple device complexity while achieving high adaptability to different workload priorities
Data Source
AI summary
Disclosed embodiments relate to a cache line eviction algorithm. In one example, a system includes a last level cache (LLC) having multiple ways, each allocated to one of multiple priorities, each having specified minimum and maximum ways to occupy, a cache control circuit (CCC) to store an incoming cache line (CL) having a requestor priority to an invalid CL, if any, otherwise, when the requestor priority is a lowest priority and has an occupancy of one or more, or when the occupancy is at a maximum, to evict a least recently used (LRU) CL of the requestor priority, otherwise, when the occupancy is between a minimum and a maximum, to evict a LRU CL of the requestor or a lower priority, otherwise, when the occupancy is less than the minimum, to evict a LRU CL, if any, of the lower priority, and otherwise, to evict a LRU CL of a higher priority.