Memory Bandwidth Constraint Subsystem Throttling Request Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The oversubscription of memory links in processing systems due to shared memory resources among multiple processor cores leads to increased memory access latencies and reduced system performance.
Innovation Solution
Implementing a memory bandwidth constraint subsystem that throttles memory requests by monitoring access latency and prefetch accuracy metrics to adjust prefetcher aggressiveness and the number of pending memory transactions, thereby controlling memory bandwidth consumption and improving link efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory resources are shared among multiple processor cores, then resource utilization efficiency is improved, but memory link saturation occurs leading to increased access latencies
Solution Approach 1:
The patent implements a feedback mechanism where the memory bandwidth constraint subsystem continuously monitors memory access performance metrics (such as access latency and memory link utilization) and dynamically adjusts the throttling of memory requests from individual processor cores. When memory link saturation is detected, the subsystem reduces the memory request rate from offending cores, and when the link is underutilized, it allows higher request rates, thereby maintaining optimal resource utilization while preventing latency degradation.
Solution Approach 2:
The patent applies dynamic throttling of memory requests based on real-time memory link conditions. The memory bandwidth constraint subsystem adjusts the memory request rate from each processor core dynamically rather than using fixed allocation, allowing the system to adapt to changing workloads and memory access patterns. This dynamic adjustment ensures that memory resources are efficiently shared while preventing any single core from saturating the memory link.
2Productivity
If memory request throttling is implemented, then memory link oversubscription is reduced, but system complexity increases
Solution Approach 1:
The patent introduces a memory bandwidth constraint subsystem as an intermediary component between the processor cores and the memory system. This subsystem acts as a mediator that manages memory request throttling, monitoring memory link conditions and adjusting request rates from individual cores. By placing this intelligence at the memory interface rather than in each processor core, the patent achieves effective memory link management while minimizing the complexity burden on the core processing units.
Data Source
AI summary
A processing system includes an interconnect fabric coupleable to a local memory and at least one compute cluster coupled to the interconnect fabric. The compute cluster includes a processor core and a cache hierarchy. The cache hierarchy has a plurality of caches and a throttle controller configured to throttle a rate of memory requests issuable by the processor core based on at least one of an access latency metric and a prefetch accuracy metric. The access latency metric represents an average access latency for memory requests for the processor core and the prefetch accuracy metric represents an accuracy of a prefetcher of a cache of the cache hierarchy.