Memory Group Credit Scheduling for Heterogeneous I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The varying latencies of heterogeneous memory devices in a main memory system cause performance inconsistencies and resource starvation, leading to difficulties in maintaining uniform quality-of-service (QoS) and potential timeouts in input/output operations.
Innovation Solution
An input/output device schedules requests based on a latency table that measures and groups memory devices, allocating credits per memory group to ensure uniform performance and prevent resource starvation, thereby maintaining QoS.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If memory devices with different performance characteristics are served by a single queue, then queue management is simplified, but performance of fast memory devices is degraded due to slow memory devices
Solution Approach 1:
The patent divides the single I/O queue into multiple class-specific queues (CQs), one for each memory device type. This segmentation allows fast memory devices and slow memory devices to be served separately, preventing fast devices from waiting for slow ones, thereby resolving the contradiction between simplified queue management and maintaining high performance.
2Productivity
If class-specific queues are introduced for different memory devices, then performance of fast memory devices is improved, but system complexity increases
Solution Approach 1:
The patent introduces a classification unit as an intermediary that receives I/O requests and automatically directs them to the appropriate class-specific queue based on the target memory device type. This intermediary handles the complexity of queue management, allowing the system to maintain multiple specialized queues while keeping the overall system manageable through centralized classification logic.
3Loss of time
If I/O requests are processed immediately upon arrival, then response time is reduced, but fast memory devices may wait idle for slow memory devices
Solution Approach 1:
The patent segments I/O requests into different classes based on target memory device type, with each class having its own dedicated queue. This ensures that fast memory device requests are processed immediately without being blocked by slow memory device requests, reducing idle waiting time while maintaining high utilization of fast devices.
4Productivity
If memory devices are grouped by performance, then scheduling efficiency is improved, but the number of groups and management overhead increases
Solution Approach 1:
The patent creates a universal credit management mechanism that operates across all class-specific queues. The credit-based scheduling algorithm provides a unified approach to controlling request transmission rates to different memory device groups, reducing management overhead by using a single scheduling policy rather than requiring separate management mechanisms for each group.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of scheduling an input/output request for a main memory (120) includes acquiring information of a plurality of memory devices (121, 122) included in the main memory (120), measuring a plurality of latencies of the plurality of memory devices (121, 122), respectively, per unit data size of a type of each memory device, grouping the plurality of memory devices (121, 122) into a plurality of memory groups, determining a size of a time window to be periodically repeated, setting a target processing size of each memory group, based on the size of the time window and a latency of each memory group, allocating a credit value of each memory group for each time window, based on the target processing size of each memory group, and processing a plurality of input/output requests queued in an input/output queue associated with a corresponding memory group of the plurality of memory groups using the allocated credits of each memory group.