Double Threshold Memory Scheduling for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory sub-systems face challenges in optimizing latency and throughput due to varying latency characteristics of different memory types, such as DRAM and NAND flash memory, and the complexity of cross-point arrays of non-volatile memory cells, which restrict data bus utilization and lead to increased latency.
Innovation Solution
Implementing double threshold controlled scheduling in memory sub-systems, where two thresholds - a promotion threshold and an executed transaction threshold - are used to determine when to switch execution between read and write command queues, optimizing data bus utilization and reducing latency by ensuring balanced servicing of commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional scheduling schemes are used, then device complexity is reduced, but data bus utilization is restricted and latency increases
Solution Approach 1:
The patent segments the command queue into multiple sub-queues (read queue and write queue) and applies different scheduling thresholds to each type of command. This segmentation allows independent optimization of read and write operations, improving overall data bus utilization while maintaining manageable complexity through structured organization.
Solution Approach 2:
The patent implements dynamic threshold adjustment where the promotion threshold and executed transaction threshold are not fixed but adapt based on queue depths and system state. This dynamic approach allows the scheduler to respond to varying workload conditions, optimizing data bus utilization without requiring overly complex static scheduling rules.
2Loss of time
If single threshold scheduling is used, then ease of operation is maintained, but latency increases under varying workload conditions
Solution Approach 1:
The patent applies different threshold values and scheduling criteria to different command types (reads vs. writes) and different queue states. This local quality approach allows optimization of latency for specific command types without complicating the overall scheduling mechanism, as each sub-queue can be tuned independently based on its characteristics.
Solution Approach 2:
The patent changes scheduling parameters (promotion threshold, executed transaction threshold) based on system state and workload conditions. By adjusting these parameters dynamically, the system reduces latency under varying conditions while maintaining relatively simple control logic through parameter adaptation rather than complex decision trees.
3Productivity
If aggressive queue servicing is implemented, then throughput is improved, but queue overflow risk increases during intensive workloads
Solution Approach 1:
The patent implements feedback mechanisms where the scheduler monitors queue depths, executed transaction counts, and threshold conditions to dynamically adjust scheduling decisions. This feedback loop allows the system to maintain high throughput by aggressively servicing queues when conditions permit, while automatically reducing aggression when queues approach capacity, thus preventing overflow.
Solution Approach 2:
The patent introduces threshold criteria as intermediary control elements between the command queues and the execution engine. These thresholds act as mediators that buffer the system against overflow by providing staged promotion criteria, allowing aggressive servicing when thresholds are met while preventing direct overflow through controlled promotion rates.
4Loss of time
If balanced queue servicing is enforced, then latency is reduced, but data bus utilization decreases during asymmetric workloads
Solution Approach 1:
The patent implements dynamic threshold adjustment that adapts to workload asymmetry. During asymmetric workloads (e.g., read-intensive), the promotion threshold and executed transaction threshold are adjusted to favor the dominant command type, allowing aggressive servicing that maintains high data bus utilization. During balanced workloads, the thresholds enforce more balanced servicing to reduce average latency.
Solution Approach 2:
The patent changes scheduling parameters based on workload characteristics. The promotion threshold and executed transaction threshold are modified according to the mix of read and write commands, allowing the system to optimize for either latency reduction or utilization maximization depending on the current workload phase, thus resolving the contradiction between balanced servicing and asymmetric optimization.
Data Source
AI summary
A processing device in a memory system determines that a number of commands from an active queue that have been executed on a memory device does not satisfy an executed transaction threshold criterion, that a number of pending commands in an inactive queue satisfies a first promotion threshold criterion, and that a number of pending commands in the active queue does not satisfy a second promotion threshold criterion. In response, the processing device switches an execution grant from the active queue to the inactive queue.