Memory Queue Architecture for Multi-Processor Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance multi-processor systems face memory bandwidth limitations due to memory conflicts in multi-bank memory organizations, where simultaneous data accesses often lead to stalls and reduced performance, especially in applications requiring high parallelism and bandwidth.
Innovation Solution
The proposed solution involves a memory queue architecture with dedicated queues for each functional unit and memory bank, combined with software techniques for optimizing data layout and access scheduling, to reduce memory conflicts and improve data throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a true multi-port memory is used to provide high memory bandwidth, then data throughput is improved, but cost and complexity increase non-linearly
Solution Approach 1:
The memory system is segmented into multiple single-port memory banks instead of using a single true multi-port memory. Each memory bank can be independently accessed, and the banks are organized in a way that allows parallel data transfers. This segmentation approach achieves high data throughput while keeping individual memory components simpler and more cost-effective.
Solution Approach 2:
Multiple single-port memory banks are merged to function as a pseudo multi-port memory system. By combining the capabilities of several simpler memory units, the system achieves the performance characteristics of a true multi-port memory without the associated complexity and cost penalties.
2Device complexity
If multiple single-port memory banks are assembled to form a pseudo multi-port system, then cost is reduced, but memory conflicts occur when multiple accesses target the same bank
Solution Approach 1:
The patent introduces an additional dimension to memory addressing by implementing memory banks with multiple ports or by creating a hierarchical memory structure. This allows memory accesses to be routed through different paths or layers, enabling simultaneous non-conflicting accesses to different data while using the same physical memory bank, thereby eliminating memory conflicts without increasing overall system complexity.
Solution Approach 2:
An intermediary arbitration mechanism or multiplexer is introduced between the memory banks and the functional units. This intermediary component manages the memory access requests, ensuring that multiple simultaneous accesses to the same bank are resolved without stalling the system, thus maintaining high data throughput while using simpler single-port memory banks.
3Loss of time
If memory accesses are serviced immediately without queuing, then response time is reduced, but memory conflicts cause stalls and performance degradation
Solution Approach 1:
Memory access requests are placed in queues at functional units or at memory banks in advance of actual data transfer. This preliminary queuing allows the system to prepare and buffer requests, then service them in an optimized manner that avoids conflicts. The queue acts as a buffer that decouples the immediate access need from the actual memory operation timing, enabling conflict-free simultaneous accesses.
Solution Approach 2:
The memory queue system ensures continuous productive operation by buffering memory requests and maintaining steady data flow. Instead of stalling when conflicts occur, the queue allows overlapping of memory operations, ensuring that functional units continue executing instructions without interruption while memory bandwidth is efficiently utilized.
Data Source
AI summary
A hardware memory architecture or arrangement suited for multi-processor systems or arrays is disclosed. In one aspect, the memory arrangement includes at least one memory queue between a functional unit (e.g., computation unit) and at least one memory device, which the functional unit accesses (for write and/or read access).