Request Scheduler Batching for Multi-Threaded Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory controllers in multi-threaded environments face inefficiencies due to increased interference between thread requests in shared memory systems, leading to bottlenecks, unpredictable performance, and user productivity losses.
Innovation Solution
A request scheduler that uses request batching and parallelism-aware scheduling to prioritize and service thread requests, optimizing row-hit criteria and intra-thread bank-level parallelism by grouping requests into batches and ranking threads based on age and priority.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory controllers service concurrent thread requests in multi-threaded environments, then memory access requests are handled, but interference between thread requests increases causing bottlenecks and unpredictable performance
Solution Approach 1:
The patent segments concurrent thread requests into separate batches, where each batch contains requests from multiple threads that are serviced together as a unit. This segmentation reduces interference between threads by ensuring that requests within a batch are handled in a coordinated manner, improving both throughput and performance predictability in multi-threaded environments.
2Productivity
If thread requests are serviced individually without batching, then request processing is simple, but thread stall times increase and system performance decreases
Solution Approach 1:
The patent merges multiple thread requests into batches that are serviced together. By combining requests from multiple threads into a single batch operation, the system reduces the overhead of individual request handling and minimizes thread stall times, thereby improving overall system performance while maintaining simple request processing logic.
3Productivity
If requests are serviced in strict FIFO order, then scheduling is simple and fair, but row conflict requests increase causing memory bandwidth inefficiency
Solution Approach 1:
The patent implements a dynamic scheduling approach where the servicing order of requests within a batch can be adjusted based on row-hit criteria. Instead of strict FIFO ordering, the scheduler dynamically reorders requests to prioritize those that will result in row hits, thereby improving memory bandwidth utilization while maintaining manageable scheduling complexity through batch-based organization.
Data Source
AI summary
A “request scheduler” provides techniques for batching and scheduling buffered thread requests for access to shared memory in a general-purpose computer system. Thread-fairness is provided while preventing short- and long-term thread starvation by using “request batching.” Batching periodically groups outstanding requests from a memory request buffer into larger units termed “batches” that have higher priority than all other buffered requests. Each “batch” may include some maximum number of requests for each bank of the shared memory and for some or all concurrent threads. Further, average thread stall times are reduced by using computed thread rankings in scheduling request servicing from the shared memory. In various embodiments, requests from higher ranked threads are prioritized over requests from lower ranked threads. In various embodiments, a parallelism-aware memory access scheduling policy improves intra-thread bank-level parallelism. Further, rank-based request scheduling may be performed with or without batching.