Shared Cache Priority-Aware LRU for Multi-Workload Retention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current caches are not configured to efficiently operate as shared caches among multiple workloads with varying priority levels, leading to inefficient data management and access.
Innovation Solution
Implementing a priority-aware least-recently-used (LRU) replacement policy in a shared cache that positions cache lines in a replacement queue based on their priority levels, allowing higher priority data to remain longer and optimizing data retention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a standard least-recently-used (LRU) replacement policy is implemented in a shared cache, then cache replacement is simple and fast, but data from lower priority workloads may be incorrectly retained while higher priority workload data is evicted
Solution Approach 1:
The cache is segmented into multiple priority levels (e.g., first priority level and second priority level), with each level having its own LRU queue. This segmentation allows the cache to differentiate between high-priority and low-priority workload data, ensuring that high-priority data is retained preferentially during cache replacement operations.
Solution Approach 2:
Different replacement policies are applied to different segments of the cache based on priority levels. High-priority cache lines use one LRU queue while low-priority cache lines use another, allowing each segment to have optimized retention behavior suited to its priority classification.
2Adaptability or versatility
If cache lines are repositioned in the LRU queue based on priority levels, then higher priority data is retained longer, but the cache management complexity increases
Solution Approach 1:
The LRU queue structure is made dynamic by allowing cache lines to be repositioned based on their priority level. When a cache line is accessed, its position in the LRU queue is adjusted according to its priority, enabling adaptive retention behavior without requiring a complete redesign of the cache replacement mechanism.
Solution Approach 2:
The replacement policy parameters are changed by introducing priority levels as an additional dimension. Instead of a single LRU queue, the system uses multiple LRU queues or priority-weighted positioning, where the queue position is determined by both recency of use and priority level parameters.
3Reliability
If multiple LRU queues are maintained for different priority levels, then priority-aware replacement is achieved, but the cache structure and management overhead increase
Solution Approach 1:
The cache is divided into multiple segments or sets, each associated with a specific priority level. Each segment maintains its own LRU queue, allowing independent management of high-priority and low-priority data. This segmentation ensures that eviction decisions are made within the appropriate priority context.
Solution Approach 2:
A priority determination mechanism acts as an intermediary between the cache access logic and the LRU queue structure. This intermediary determines the priority level of accessed data and routes it to the appropriate LRU queue or adjusts its position accordingly, simplifying the overall management complexity.
Data Source
AI summary
The disclosure provides for improved systems and methods for operating a shared cache among multiple workloads that have different levels of priority. For example, a plurality of processor cores can have access to a shared system cache, and these processor cores can be configured to execute different workloads in parallel with one another. Moreover, some workloads may be designated as having a higher priority than other workloads, and the shared cache can be configured to maintain cached data in a manner that can allow data associated with higher priority workloads to remain within the shared cache for a longer period of time than data for lower priority workloads. In accordance with aspects of the disclosure, a shared cache may implement an altered least-recently-used (LRU) replacement policy in which cache lines of data are positioned within a replacement queue based on the priority level of the data that is being stored.