JIT Scheduler for Memory Subsystems Reducing Host Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory subsystems face issues with increased latency due to die contention and head-of-line blocking, where background operations temporarily render the system unavailable to the host, and prioritizing host-initiated memory access can starve background operations, leading to inefficiencies in data transfer and resource management.
Innovation Solution
A just-in-time (JIT) scheduler is implemented to manage buffer allocations, data transfers, and memory operation submissions, using multiple scheduling pools and a quality of service (QoS) traffic manager to prioritize requests and allocate resources effectively, ensuring that no traffic class is starved and reducing latency by managing QoS credits and backend resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If background operations are executed in memory subsystems, then data management functions (wear leveling, garbage collection, ECC) are performed, but host-initiated memory access experiences increased latency due to die contention and head-of-line blocking
Solution Approach 1:
The patent segments memory operations into different pools based on priority and type. High-priority host-initiated operations are separated from lower-priority background operations into distinct scheduling pools, allowing them to be processed independently without mutual interference. This segmentation eliminates head-of-line blocking where background operations would otherwise delay host commands.
Solution Approach 2:
The patent implements dynamic scheduling where the memory controller adapts operation scheduling in real-time based on system conditions. The scheduler dynamically adjusts which operations are executed and when, prioritizing host commands when they arrive while still ensuring background operations eventually execute. This dynamic approach optimizes latency by responding to actual workload conditions rather than following a fixed schedule.
2Loss of time
If host-initiated memory access is prioritized to reduce latency, then host command response time improves, but background operations are starved and cannot execute properly
Solution Approach 1:
The patent ensures continuous execution of both host-initiated and background operations through multiple scheduling pools. Instead of allowing background operations to be completely starved during host access bursts, the system maintains continuous progress on background tasks by executing them in dedicated pools when resources are available. This ensures wear leveling, garbage collection, and ECC operations continue without interruption while still prioritizing host commands.
Solution Approach 2:
The patent implements feedback mechanisms where the memory controller monitors the execution status of both host and background operations. When background operations fall behind or resources become available, the scheduler adjusts priorities to ensure background tasks execute. This feedback loop prevents starvation by continuously monitoring and adjusting scheduling decisions based on actual system state and operation completion status.
3Productivity
If multiple memory operations access the same die simultaneously, then throughput is increased, but die contention occurs leading to increased latency
Solution Approach 1:
The patent segments operations destined for the same die into different scheduling pools based on priority and timing requirements. By separating high-priority host commands from lower-priority background operations that target the same die, the system allows high-priority operations to execute without being delayed by lower-priority contention. This segmentation maintains throughput by ensuring all operations eventually execute while eliminating latency spikes from contention.
Data Source
AI summary
A just-in-time (JIT) scheduling method includes the operations of: receiving a request to perform a memory operation using a hardware resource associated with a memory device; determining a type of the memory operation; identifying a traffic class corresponding to the memory operation; determining, based on the traffic class and the type of the memory operation, whether the memory operation is to be processed during a current scheduling time frame; and responsive to determining the memory operation is to be processed during the current scheduling time frame, submitting the memory operation to the memory device.