Rate Limiting Circuit for Fair Memory Bandwidth Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer architectures, ensuring fair distribution of processor memory bandwidth among multiple transaction generators is challenging due to limitations in interconnect design, cache behavior, and unpredictable latency caused by retries when memory channels become full.
Innovation Solution
A rate limiting circuit is introduced to manage tokens for each transaction generator, allowing transactions only when sufficient tokens are available, and recycling tokens to maintain fair bandwidth distribution among virtual machines, while monitoring cache hits and misses to adjust token allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple transaction generators access memory channels without rate limiting, then memory bandwidth utilization increases, but transaction latency becomes unpredictable and fairness deteriorates due to retries when channels become full
Solution Approach 1:
The rate limiting circuit performs preliminary actions by allocating tokens to transaction generators before they access memory channels. This advance allocation ensures that generators have permission to access channels without causing congestion, thereby maintaining predictable latency while achieving fair bandwidth distribution among multiple virtual machines
Solution Approach 2:
The rate limiting circuit acts as an intermediary between transaction generators and memory channels. It mediates access by controlling token distribution, allowing generators to access channels only when tokens are available, thus preventing channel overflow and retries while maintaining high bandwidth utilization through efficient resource arbitration
2Reliability
If rate limiting is implemented to ensure fair bandwidth distribution, then transaction latency becomes predictable, but system complexity increases due to token management overhead
Solution Approach 1:
The rate limiting circuit changes parameters by using simple token counts and allocation rates instead of complex scheduling algorithms. This parameter-based approach achieves fair bandwidth distribution through configurable token distribution while keeping the implementation simple and avoiding complex state management overhead
3Productivity
If memory channels are allowed to fill up to maximize throughput, then bandwidth utilization improves, but retry rates increase causing performance degradation
Solution Approach 1:
The rate limiting circuit provides beforehand cushioning by controlling the flow of transactions to memory channels. It prevents channels from becoming overloaded by regulating token allocation, thus cushioning against the harmful effect of retries while maintaining high bandwidth utilization through smooth traffic flow management
Data Source
AI summary
In various implementations, provided are systems and methods for an integrated circuit implementing a processor that can include a rate limiting circuit that attempts to fairly distribute processor memory bandwidth between transaction generators in the processor. The rate limiting circuit can maintain a count of tokens for each transaction generator, where a transaction generator can only transmit a transaction when the transaction generator has enough tokens to do so. Each transaction generator can send a request to the rate limiting circuit when the transaction generator wants to transmit a transaction. The rate limiting circuit can then check whether the transaction generator has sufficient tokens to transmit the transaction. When the transaction generator has enough tokens, the rate limiting circuit will allow the transaction to enter the interconnect. When the transaction generator does not have enough tokens, the rate limiting circuit will not allow the transaction to enter the interconnect.