Buffer Memory HOL Entry Processing via Tiered Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In buffer memory devices with multiple-source multiple-destination queues, Head of Line (HOL) requests are often blocked due to insufficient storage, leading to reduced throughput and increased latency.
Innovation Solution
The implementation of a method that involves a higher tier memory and a lower tier memory, where HOL entries are stored in the higher tier memory and non-HOL entries are tiered down to the lower tier memory, with an overprovisioned portion of the higher tier memory to accommodate HOL entries, and dual residency of Tail of Line (TOL) entries to manage storage efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the higher tier memory size is increased to accommodate HOL entries, then throughput is improved, but device cost and complexity increase
Solution Approach 1:
The memory system is segmented into two distinct tiers: a smaller, faster higher tier memory (e.g., SRAM) and a larger, slower lower tier memory (e.g., DRAM). This segmentation allows the higher tier to be optimized for speed with minimal size, while the lower tier provides bulk storage capacity, thereby maintaining high throughput without requiring a large higher tier memory
Solution Approach 2:
The patent introduces a hierarchical dimension to the memory architecture by adding vertical tiering rather than simply expanding the horizontal size of the higher tier. This dimensional change enables the system to achieve large effective memory capacity through the lower tier while preserving the speed characteristics of the smaller higher tier
2Device complexity
If the higher tier memory size is reduced to lower cost, then device complexity is reduced, but HOL request blocking increases
Solution Approach 1:
The lower tier memory acts as an intermediary bulk storage layer that receives and holds entries when the higher tier is full. This intermediary layer prevents HOL blocking by providing an additional storage destination, allowing the higher tier to remain small while still supporting reliable HOL entry storage through the tiering mechanism
Solution Approach 2:
The system performs preliminary tiering actions by proactively moving non-HOL entries to the lower tier memory in advance before the higher tier becomes completely full. This preliminary action creates space in the higher tier for incoming HOL entries, preventing blocking before it occurs
3Productivity
If entries are tiered down to lower tier memory, then higher tier utilization is improved, but access latency increases
Solution Approach 1:
Different quality levels are assigned to different memory tiers: the higher tier provides high-speed local access for active and frequently accessed entries, while the lower tier provides lower-speed bulk storage for less frequently accessed entries. This local quality differentiation ensures that time-critical HOL operations use the fast higher tier while bulk storage uses the slower lower tier, minimizing overall latency impact
Data Source
AI summary
A method of a buffer memory device, a storage system, and a buffer memory device are provided. The method of the buffer memory device, the buffer memory device having a lower tier memory and a higher tier memory, may include receiving a new entry request, determining that the new entry request includes an HOL entry, selecting an entry on the higher tier memory to be tiered down to the lower tier memory in response to determining that the new entry request includes an HOL entry, removing the selected entry from the higher tier memory, storing the HOL entry in the higher tier memory of the buffer memory device, and outputting the HOL entry to an arbiter.