Memory Controller Sparse Access Coalescing for Command Queue Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The memory controller's command queue in DRAM organizations faces bottlenecks due to its fixed size and the inefficiency in scheduling sparse data access requests, particularly in applications like image processing and stencil computation.
Innovation Solution
Implementing a sparse element processor that identifies and combines sparse memory access requests into denser requests, utilizing a coalescing unit to group sparse requests and a memory controller with a command queue that supports efficient scheduling by converting sparse requests into denser ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the command queue uses associational memory organized as content addressable memory (CAM) with an arbiter that scans the command queue, then efficient scheduling of memory access requests is achieved, but the command queue size is fixed and becomes a bottleneck for sparse data access requests
Solution Approach 1:
The patent segments the command queue into multiple separate queues (e.g., read queue and write queue, or multiple priority queues). This segmentation allows each queue to be optimized independently and enables better management of sparse access requests by distributing them across multiple smaller queues, thereby resolving the bottleneck caused by a single large fixed-size queue.
Solution Approach 2:
The patent implements dynamic queue management where the command queue structure can adapt its organization based on the type and characteristics of memory access requests. Different queue configurations can be activated dynamically to handle sparse versus dense access patterns, allowing the system to optimize performance for varying workload conditions rather than being constrained by a fixed queue structure.
2Adaptability or versatility
If the memory controller handles sparse data access requests directly, then support for fine-grained DRAM organizations is maintained, but scheduling efficiency deteriorates due to the scattered nature of sparse requests
Solution Approach 1:
The patent merges multiple sparse memory access requests into a single consolidated memory access operation when possible. By combining adjacent or related sparse requests into a unified access pattern, the system reduces the total number of memory transactions and improves scheduling efficiency while still supporting fine-grained access patterns through the merging process.
Solution Approach 2:
The patent introduces an intermediary processing stage between request generation and memory access execution. This intermediary layer analyzes incoming sparse requests, identifies opportunities for merging or optimization, and transforms them into more efficient access patterns before submission to the memory controller, thereby improving scheduling efficiency without losing sparse access capability.
3Quantity of substance
If the command queue is enlarged to handle more sparse requests, then request capacity increases, but the associational memory structure and arbiter scanning efficiency are compromised
Solution Approach 1:
The patent divides the large command queue into multiple smaller segmented queues. Each segment can be scanned independently and more quickly by the arbiter, reducing the total scanning time compared to a single large queue. The segmented structure maintains overall capacity while improving access efficiency through reduced scan depths in each segment.
Solution Approach 2:
The patent transforms the single-dimension large queue structure into a multi-dimensional queue organization (e.g., multiple queues arranged in a matrix or tree structure). This dimensional change allows the arbiter to navigate and scan requests more efficiently by utilizing multiple access paths or hierarchical levels, thereby increasing capacity without proportionally increasing scanning time.
Data Source
AI summary
A data processing system includes a data processor and a memory controller receiving memory access requests from the data processor and generating at least one memory access cycle to a memory system in response to the receiving. The memory controller includes a command queue and a sparse element processor. The command queue is for receiving and storing the memory access requests including a first memory access request including a small element request. The sparse element processor is for causing the memory controller to issue a second memory access request to the memory system in response to the first memory access request with a density greater than a density indicated by the first memory access request.