Asynchronous Memory Scheduler for Multi-Core Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory access scheduling techniques are ineffective for multi-core processors and multi-threaded applications due to interleaving of memory access requests to disparate locations, leading to reduced throughput and increased latency.
Innovation Solution
An improved arbitration process that uses memory timing and state information, including predicted memory timing and state information, to asynchronously schedule memory access requests, allowing for greater consideration of pending requests and optimization of memory access sequences, while operating the scheduler at a different clock frequency than the memory access controller.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory access scheduling techniques are used, then the system can handle simple single-core workloads, but multi-core processors and multi-threaded applications experience reduced throughput and increased latency due to interleaved memory requests to disparate locations
Solution Approach 1:
The patent segments the scheduling problem by dividing memory access requests into different categories based on their characteristics (e.g., sequential vs. random access patterns). The scheduler processes different segments of requests using different scheduling strategies, allowing efficient handling of multi-core workloads without overwhelming complexity in the overall scheduling mechanism.
Solution Approach 2:
The patent implements preliminary action by predicting future memory access patterns and pre-scheduling requests before they are fully generated. The system uses history information from previous access patterns to anticipate upcoming memory requests and prepare scheduling decisions in advance, reducing latency and improving throughput for multi-threaded applications.
2Productivity
If memory accesses are scheduled to access the same local region consecutively, then access efficiency is improved, but multi-core processors and multi-threaded applications generate interleaved requests to disparate locations that defeat this optimization
Solution Approach 1:
The patent applies dynamics by making the scheduling strategy adaptive rather than static. The scheduler dynamically adjusts its behavior based on the current mix of memory requests from multiple cores and threads. When sequential access patterns are detected, the scheduler optimizes for locality; when random interleaved patterns dominate, it switches to strategies that minimize latency and maintain fairness across different access patterns.
Solution Approach 2:
The patent changes scheduling parameters based on workload characteristics. It monitors metrics such as access pattern entropy, core utilization, and memory bank utilization to dynamically adjust scheduling parameters like queue priorities, arbitration weights, and precharge timing. This allows the system to maintain high efficiency across diverse access patterns from multi-core and multi-threaded workloads.
3Productivity
If the scheduler operates at the same clock frequency as the memory access controller, then synchronization is simplified, but the scheduler cannot consider enough pending requests to optimize the access sequence
Solution Approach 1:
The patent uses preliminary action by having the scheduler operate asynchronously at a higher clock frequency to evaluate and pre-select optimal memory access sequences from pending requests. This allows the scheduler to consider more pending requests and make better optimization decisions before the memory controller executes the actual accesses, improving overall memory efficiency without adding execution time.
Solution Approach 2:
The patent introduces an intermediary mechanism where the asynchronous scheduler generates optimized access sequences that are then translated and synchronized with the memory controller's clock domain. This intermediary translation layer allows the scheduler to operate at optimal frequency for optimization while maintaining proper synchronization with the memory controller, resolving the conflict between optimization capability and time loss.
Data Source
AI summary
A data processing system employs a scheduler to schedule pending memory access requests and a memory controller to service scheduled pending memory access requests. The memory access requests are asynchronously scheduled with respect to the clocking of the memory. The scheduler is operated using a clock signal with a frequency different from the frequency of the clock signal used to operate the memory controller. The clock signal used to clock the scheduler can have a lower frequency than the clock used by a memory controller. As a result, the scheduler is able to consider a greater number of pending memory access requests when selecting the next pending memory access request to be submitted to the memory for servicing and thus the resulting sequence of selected memory access requests is more likely to be optimized for memory access throughput.