Memory Hierarchy Bandwidth Shaping for Fair Cache-Hit Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional bandwidth shaping mechanisms lack flexibility in managing traffic patterns across multiple levels of a memory hierarchy, leading to penalization of traffic classes that primarily hit the upper memory, affecting overall bandwidth allocation and efficiency.
Innovation Solution
Implementing a dual-bandwidth shaper system comprising a primary and secondary shaper to differentiate between cache hits and system memory accesses, adjusting bandwidth allocation based on historical subscription and hit bandwidth consumption, ensuring fair allocation and preventing penalization of upper memory-hitting traffic classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional bandwidth shaping mechanisms are used, then bandwidth allocation is simplified, but traffic classes that primarily hit upper memory are penalized and bandwidth efficiency deteriorates
Solution Approach 1:
The bandwidth shaping mechanism is segmented into multiple independent shapers (e.g., L1 shaper, L2 shaper) that operate at different levels of the memory hierarchy. Each shaper independently manages bandwidth for its specific level, allowing cache hits to be shaped separately from cache misses, thereby preventing penalization of upper memory-hitting traffic while maintaining overall system bandwidth efficiency.
Solution Approach 2:
The patent introduces a new dimension to bandwidth shaping by adding level-awareness to the traditional single-level shaping approach. The mechanism distinguishes between different memory hierarchy levels (L1 cache, L2 cache, system memory) and applies shaping policies accordingly, transforming a one-dimensional bandwidth management problem into a multi-dimensional solution that accounts for hierarchical structure.
2Adaptability or versatility
If traffic classes are differentiated by memory access patterns, then fair bandwidth allocation is achieved, but monitoring and control complexity increases
Solution Approach 1:
The bandwidth shaping mechanism leverages the memory controller's existing self-service capabilities to monitor and classify traffic patterns. The controller automatically detects whether a memory access is a cache hit or miss and applies appropriate shaping without requiring external intervention or complex monitoring infrastructure, thus achieving adaptive traffic classification while minimizing added complexity.
Solution Approach 2:
The system implements feedback mechanisms where the memory controller monitors actual memory access patterns and adjusts bandwidth allocation accordingly. The shapers receive feedback about traffic characteristics and dynamically modify their behavior to ensure fair allocation, allowing the system to adapt to changing workload patterns while maintaining manageable control through closed-loop operation.
Data Source
AI summary
Bandwidth shaping mechanisms in a memory hierarchy operate a first bandwidth shaper on a first memory, such as a cache memory, to shape bandwidth to a second memory, such as a Dynamic Random Access Memory, and replenish the first bandwidth shaper from a second bandwidth shaper based on a hit bandwidth on the first memory.