System-Level Cache Way Partitioning for Dynamic Buffer Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing system-level caches (SLCs) lack flexibility in adapting to changing execution contexts, leading to inefficient power consumption and increased latency due to inadequate partitioning policies, which fail to dynamically allocate cache resources based on active client devices and memory buffers.
Innovation Solution
Implementing a way-partitioning policy in SLCs that allocates primary ways dedicated to specific memory buffers and secondary ways shared among multiple buffers, allowing dynamic adjustment of cache configurations based on execution contexts, thereby optimizing power efficiency and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a system-level cache uses a static partitioning policy, then the cache configuration is simple and stable, but it cannot adapt to changing execution contexts, leading to increased power consumption and latency
Solution Approach 1:
The patent implements a dynamic partitioning policy where the system-level cache can reallocate ways between partitions based on changing execution contexts. The cache controller monitors access patterns and dynamically adjusts the number of ways allocated to each partition, allowing the cache to adapt to different client device requirements and memory access patterns without requiring a complete reconfiguration of the cache structure.
Solution Approach 2:
The patent changes the parameter of cache way allocation from a fixed static value to a dynamic value that can be adjusted based on execution context. The system modifies the number of ways allocated to each partition according to observed access patterns and performance requirements, enabling adaptation to changing workloads while maintaining a relatively simple cache structure.
2Productivity
If the system-level cache allocates more ways to a partition, then caching performance for that partition improves, but power consumption increases due to more cache lines being maintained
Solution Approach 1:
The patent applies dynamic allocation of cache ways to partitions based on actual access patterns. The cache controller monitors which partitions are actively being accessed and allocates more ways to high-performance partitions while reducing or deallocating ways from inactive partitions. This dynamic adjustment allows the system to optimize caching performance for active workloads while conserving power by not maintaining cache lines for inactive partitions.
Solution Approach 2:
The patent implements partial allocation of cache ways to partitions based on current needs. Rather than allocating a fixed number of ways to each partition regardless of usage, the system allocates only the necessary number of ways to each partition at any given time, adjusting the allocation partially based on observed access patterns and performance requirements.
3Productivity
If the system-level cache uses a dynamic partitioning policy, then it can optimize for different execution contexts, but the complexity of managing and switching partitions increases
Solution Approach 1:
The patent implements a self-service dynamic partitioning mechanism where the cache controller automatically monitors access patterns and performs reallocation of ways between partitions without external intervention. The system uses hardware-based monitoring and control logic that autonomously adjusts partition configurations based on observed workloads, reducing the management complexity that would otherwise require complex software controllers or manual configuration.
4Reliability
If the system-level cache maintains dedicated ways for each partition, then cache thrashing is minimized, but the flexibility to serve multiple client devices efficiently is reduced
Solution Approach 1:
The patent implements a dynamic partitioning policy where the number of dedicated ways allocated to each partition can change based on execution context. During periods when a partition is actively being accessed, the system allocates more dedicated ways to that partition to prevent cache thrashing. When the partition becomes inactive, the system reduces or deallocates ways, making them available for other active partitions. This dynamic adjustment maintains thrashing prevention for active workloads while providing flexibility to serve multiple different client devices and workloads over time.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a system-level cache to allocate cache resources by a way-partitioning process. One of the methods includes maintaining a mapping between partitions and priority levels and allocating primary ways to respective enabled partitions in an order corresponding to the respective priority levels assigned to the enabled partitions.