Cache Allocation Circuit for Shared Memory QoS Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional shared cache memory systems lack user control over allocation, leading to underutilization and performance issues due to resource interference and inflexible Quality of Service (QoS) schemes, which fail to address various use cases such as strict capacity allocation and throughput maximization.
Innovation Solution
A cache controller with a cache allocation circuit that uses minimum and maximum mapping bitmasks to selectively allocate cache lines based on Quality of Service (QoS) identifiers, allowing for fine-grained control and partitioning of shared cache memory among multiple QoS classes, ensuring optimal allocation and utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional shared cache memory allocation is used without user control, then the system protects against worst case performance through underutilization, but the cache memory remains underutilized and overall performance deteriorates due to resource interference and cache pollution
Solution Approach 1:
The patent implements dynamic cache partitioning where the cache allocation circuit continuously monitors cache usage by different QoS classes and dynamically adjusts partition boundaries based on current workload characteristics. This allows the system to adapt to changing conditions, ensuring reliability when needed while maximizing utilization when possible, resolving the contradiction between protected performance guarantees and efficient resource use
Solution Approach 2:
The system changes the parameter of cache allocation from fixed/static to variable/dynamic by introducing QoS-based control mechanisms. The cache allocation circuit modifies partition sizes and boundaries based on QoS identifiers and usage patterns, enabling the system to transition between conservative allocation (for reliability) and aggressive allocation (for utilization) based on real-time conditions
2Adaptability or versatility
If existing cache QoS and partitioning schemes are used, then some quality of service control is provided, but the schemes lack the flexibility necessary to address multiple use cases such as strict capacity allocation and throughput maximization
Solution Approach 1:
The patent segments the shared cache memory into multiple partitions, each associated with a specific QoS class. The cache allocation circuit uses separate bitmask data structures (minimum mapping bitmask and maximum mapping bitmask) for each QoS class to independently control allocation. This segmentation enables flexible adaptation to different use cases while maintaining manageable complexity through modular organization
Solution Approach 2:
The cache allocation circuit is designed as a universal controller that handles multiple QoS classes and supports various allocation policies (strict capacity allocation, throughput maximization, etc.) through a unified mechanism. By using QoS identifiers and configurable bitmask mappings, the same hardware circuit can serve multiple functions and adapt to different workload requirements without requiring separate specialized controllers for each use case
3Productivity
If reference streams with little temporal locality are allowed to access shared cache memory, then the cache memory is fully utilized, but cache pollution occurs and overall performance deteriorates
Solution Approach 1:
The patent introduces QoS classes and the cache allocation circuit as an intermediary layer between reference streams and the shared cache memory. This intermediary controls which streams can access which cache partitions based on their QoS identifiers, allowing the system to permit high utilization from beneficial streams while blocking or limiting harmful streams that would cause cache pollution, thus resolving the contradiction between utilization and pollution prevention
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Providing shared cache memory allocation control in shared cached memory systems is disclosed. In one aspect, a cache controller of a shared cache memory system comprising a plurality of cache lines is provided. The cache controller comprises a cache allocation circuit providing a minimum mapping bitmask for mapping a Quality of Service (QoS) class to a minimum partition of the cache lines, and a maximum mapping bitmask for mapping the QoS class to a maximum partition of the cache lines. The cache allocation circuit receives a memory access request comprising a QoS identifier (QoSID) of the QoS class, and is configured to determine whether the memory access request corresponds to a cache line of the plurality of cache lines. If not, the cache allocation circuit selects, as a target partition, the minimum partition mapped to the QoS class or the maximum partition mapped to the QoS class.