Credit-Based Interconnect Control for Memory Request Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing memory systems face challenges in managing the flow of memory requests and responses efficiently, leading to potential overflow in request and response queues, which can result in degraded bandwidth, increased latency, and errors, especially when the memory device is oversaturated or when there is a mismatch between the speed of the memory array and the interconnect.
Innovation Solution
Implementing a credit-based system where the memory-side controller monitors the quantity of memory responses in the response queue and adjusts the credit returns to the host-side controller, using a lookup table to modulate the rate of credit returns based on the queue depth, thereby controlling the flow of memory requests and preventing queue overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the memory device operates at high speed to meet processor demands, then bandwidth is improved, but queue overflow and latency increase due to oversaturation
Solution Approach 1:
The patent implements a feedback mechanism where the memory device monitors response queue depth and sends credit updates to the host device. When the response queue approaches saturation, the memory device reduces credit issuance, thereby throttling incoming request flow and preventing queue overflow while maintaining high-speed operation
Solution Approach 2:
The credit-based flow control dynamically adjusts the request flow rate based on real-time memory device state. The system transitions from static bandwidth allocation to dynamic adaptation, modifying credit return rates according to response queue occupancy to prevent saturation while maximizing throughput
2Productivity
If credits are returned rapidly to maintain high request flow, then bandwidth is improved, but response queue saturation occurs increasing latency
Solution Approach 1:
The memory device uses feedback from response queue depth monitoring to regulate credit return timing. When latency increases due to queue saturation, the feedback mechanism automatically reduces credit return rate, thereby reducing request flow and allowing the response queue to drain, which reduces latency
Solution Approach 2:
The system implements periodic credit updates based on response queue state rather than continuous credit issuance. This periodic action allows the memory device to rhythmically control request flow, preventing sustained queue saturation and associated latency penalties
3Reliability
If flow control is implemented to prevent queue overflow, then reliability is improved, but bandwidth is reduced due to throttled request flow
Solution Approach 1:
The flow control mechanism is dynamic rather than static, adjusting credit return rates based on real-time memory device state. When the response queue is clear, credit issuance increases to maximize bandwidth; when saturation is detected, credit issuance decreases to prevent overflow, thereby maintaining both reliability and optimal bandwidth utilization
Data Source
AI summary
Described apparatuses and methods enable a receiver of requests, such as a memory device, to control the arrival of future requests using a credit-based communication protocol. A transmitter of requests can be authorized to transmit a request across an interconnect responsive to possession of a credit. If the transmitter exhausts its credits, the transmitter waits until a credit is returned before transmitting another request. The receiver can manage credit returns based on how many responses are present in a response queue. The receiver can change a rate at which the credit returns are transmitted by changing a size of an interval of responses that are being transmitted, with one credit being returned per interval. This can slow the rate of credit returns while the response queue is relatively more filled. The rate adjustment can decrease latency by reducing an amount of requests or responses that are pooling in backend components.