Memory Controller Command Reordering for SRAM Bank Conflicts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory designs face performance issues when multiple requestors need to access the same memory bank simultaneously, leading to stalls and significant performance overhead, especially with address patterns that result in back-to-back accesses to the same bank, particularly in machine learning applications using single-port SRAM banks with limited access cycles.
Innovation Solution
A memory controller that reorders read and write requests based on memory bank availability and priority queues, using read and write queues to manage access conflicts and distribute addresses to avoid problematic access patterns, along with an arbitration network to separate read and write traffic and implement command queues for efficient execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple requestors access the same memory bank simultaneously using single-port SRAM banks, then memory capacity and parallel access capability are improved, but performance degradation occurs due to stalls and conflicts
Solution Approach 1:
The memory system is segmented into multiple banks (e.g., Bank 0, Bank 1, Bank 2, Bank 3) that can be accessed in parallel. Each bank operates independently, allowing multiple requestors to access different banks simultaneously without conflicts. This segmentation resolves the contradiction by providing both high memory capacity through multiple banks and high access speed through parallel operations.
Solution Approach 2:
The patent introduces a bank selection dimension to the memory access process. Instead of a single requestor sequentially accessing one bank, multiple requestors can simultaneously access different banks by selecting appropriate bank indices. This dimensional expansion from single-access to multi-access resolves the performance degradation issue while maintaining capacity.
2Reliability
If address patterns result in back-to-back accesses to the same bank, then data locality is improved, but performance overhead increases significantly (up to 300%)
Solution Approach 1:
The patent performs preliminary bank selection and requestor assignment before memory access occurs. By pre-determining which bank each request should access and assigning requestors to specific banks, the system avoids back-to-back conflicts. This preliminary action eliminates the need for stalls and reduces performance overhead while maintaining correct data access sequences.
Solution Approach 2:
The patent introduces an intermediary arbitration mechanism that mediates between multiple requestors and memory banks. This intermediary layer schedules requests, selects appropriate banks, and coordinates access to prevent conflicts. The mediator resolves the contradiction by ensuring data locality through proper request sequencing while eliminating performance overhead through efficient conflict resolution.
3Use of energy by stationary object
If SRAM banks accept one access every other cycle, then physical area and power consumption are reduced, but access speed decreases
Solution Approach 1:
The patent implements dynamic request scheduling that adapts to the every-other-cycle access constraint of SRAM banks. The system dynamically adjusts the timing and sequencing of requests to match the bank's access capability, maximizing utilization within the constraints. This dynamic approach resolves the contradiction by maintaining low power consumption through proper request timing while achieving high effective access speed through optimized scheduling.
Solution Approach 2:
The patent changes the access pattern parameters to match the SRAM bank capabilities. By adjusting request timing, bank selection, and requestor assignment parameters, the system optimizes performance within the every-other-cycle constraint. This parameter optimization resolves the contradiction by maintaining low power consumption while achieving high throughput through intelligent parameter management.
4Device complexity
If one requestor is allowed to access a bank at a time with stalls for others, then access control is simplified, but productivity decreases due to waiting
Solution Approach 1:
The patent segments the single-bank sequential access model into a multi-bank parallel access model. By dividing the memory into multiple independently accessible banks, the system eliminates the need for sequential stalls while maintaining simple per-bank access control. This segmentation resolves the contradiction by providing both simple access control at the per-bank level and high throughput through parallel bank operations.
Solution Approach 2:
The patent adds a bank dimension to the access control problem. Instead of controlling access to a single bank sequentially, the system manages access across multiple banks in parallel. This dimensional expansion resolves the contradiction by maintaining simple control logic for each bank while dramatically improving throughput through concurrent access to different banks.
Data Source
AI summary
A system for handling requests that includes a set of memory banks coupled to a memory controller which comprises a set of read queues, including a read queue currently designated as the priority read queue. The memory controller loads read requests from an associated processor into the set of read queues. To process the read requests, the memory controller is configured to schedule the read requests of the priority read queue based on an availability of the associated memory bank, and if not in the priority read queue, also based on whether the read requests conflict with a recently scheduled read request from the priority read queue. Upon an execution of a read request from the priority read queue, the memory controller designates a different one of the set of read queues as the priority read queue, if the read request was at a front of the priority read queue.