Pending Write Queue Cache for Memory Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Increasing demand for memory bandwidth in network and switching devices, driven by higher port density and speeds, leads to limitations in data access performance due to insufficient read and write speeds, resulting in packet drops and reduced throughput.
Innovation Solution
Implementing a memory management system that partitions memory into sub-blocks with parity blocks for increased read performance and a pending write queue/cache for enhanced write performance, allowing simultaneous operations across multiple dimensions and avoiding collisions through set-associative memory organization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory bandwidth is increased to support higher port density and speeds, then data access performance is improved, but the complexity of the memory system increases
Solution Approach 1:
The memory system is segmented into multiple banks (first bank, second bank, third bank) that can operate independently in parallel. This allows simultaneous read and write operations to different banks, increasing data access performance without requiring a complete redesign of the memory system. Each bank can be accessed concurrently, effectively multiplying the bandwidth without proportionally increasing control complexity.
Solution Approach 2:
The patent introduces a time dimension to memory operations by implementing pipelined access patterns. While one operation is being executed in one bank, another operation can be initiated in a different bank. This temporal dimension allows overlapping of operations, increasing throughput without adding spatial complexity to individual memory access paths.
2Speed
If read speed is increased to keep pace with port speeds, then throughput is improved, but the memory architecture complexity increases
Solution Approach 1:
The memory is divided into multiple banks that can be read from simultaneously. The controller can initiate read operations to multiple banks in parallel, effectively multiplying the read bandwidth. For example, if three banks can be accessed simultaneously, the read speed appears tripled without requiring each individual access path to be physically faster.
Solution Approach 2:
The patent merges multiple memory banks into a unified address space that can be accessed through a single controller interface. This allows the system to present a simplified single-port interface to the processor while internally utilizing multiple banks operating in parallel, thereby achieving high read speeds without proportionally increasing interface complexity.
3Reliability
If write speed is increased to prevent packet drops, then data integrity is improved, but the memory system complexity increases
Solution Approach 1:
Write operations are distributed across multiple memory banks, allowing parallel write execution. The controller can issue write commands to multiple banks simultaneously, effectively multiplying the write bandwidth and preventing write bottlenecks that would cause packet drops, without requiring each write path to be individually faster.
Solution Approach 2:
The system implements write buffering and coalescing mechanisms that prepare write data in advance before it needs to be committed to memory. By accumulating and organizing write operations beforehand, the system can efficiently batch writes to multiple banks, improving effective write speed without requiring complex real-time write arbitration.
4Productivity
If multiple simultaneous operations are allowed, then bandwidth utilization is improved, but collision probability increases
Solution Approach 1:
By dividing memory into multiple isolated banks, the system allows simultaneous operations in different banks without interference. A read operation in bank 1 cannot collide with a write operation in bank 2, as they operate in independent memory spaces. This segmentation eliminates cross-operation collisions while maintaining high bandwidth utilization through parallel access.
Solution Approach 2:
The patent adds a bank dimension to the memory address space, transforming potential two-dimensional conflicts (read/write to same location) into three-dimensional operations that can be separated by bank assignment. The controller can assign conflicting operations to different banks, effectively resolving collisions by utilizing the additional bank dimension rather than increasing operational speed.
Data Source
AI summary
Techniques are described for a memory device. In various embodiments, a scheduler/controller is configured to manage data as it read to or written from a memory. A memory is partitioned into a group of sub-blocks, a parity block is associated with the sub-blocks, and the sub-blocks are accessed to read data as needed. A pending write buffer is added to a group of memory sub-blocks. Such a buffer may be sized to be equal to the group of memory sub-blocks. The pending write buffer handles collisions for write accesses to the same block.