Size-Based I/O Scheduling Across Multiple Storage Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accessing large amounts of data from a single storage device can form a bottleneck, leading to increased data access latency and slowed computations, particularly in applications like deep learning recommendation models that rely on large embedding tables.
Innovation Solution
A system with multiple storage devices and a scheduler that distributes data across these devices based on request size, using queues and load modules to manage I/O requests, thereby reducing latency by allowing parallel data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is accessed from a single storage device, then device complexity is low, but data access latency increases and productivity decreases
Solution Approach 1:
The patent divides the storage system into multiple independent storage devices (e.g., SSDs) and assigns different data partitions to each device. The scheduler divides I/O requests into separate queues based on data size, with each queue mapped to specific storage devices. This segmentation allows parallel data retrieval from multiple devices simultaneously, resolving the bottleneck of single-device access while managing complexity through structured organization.
Solution Approach 2:
The patent introduces a scheduling dimension that organizes I/O requests by size categories (small, medium, large) and maps them to different storage devices. This dimensional approach transforms the traditional single-device sequential access model into a multi-device parallel access model, where the scheduler dimension coordinates access patterns across multiple storage devices to maximize throughput.
2Loss of time
If multiple storage devices are used, then data access latency decreases, but device complexity increases
Solution Approach 1:
The patent segments I/O requests into different queues based on data size (small, medium, large requests) and assigns specific storage devices to handle specific queues. This segmentation enables simultaneous parallel processing of different request types across multiple devices, reducing latency while managing complexity through clear queue-device mappings.
Solution Approach 2:
The scheduler acts as an intermediary component that receives I/O requests, determines their size category, selects appropriate storage devices, and coordinates the retrieval operations. This intermediary abstraction layer manages the complexity of multi-device coordination, presenting a simplified interface to the rest of the system while enabling low-latency parallel access.
3Productivity
If data is distributed across multiple storage devices, then productivity increases, but device complexity increases
Solution Approach 1:
The patent distributes data across multiple storage devices by segmenting I/O requests into size-based queues and mapping each queue to specific devices. Large requests are routed to devices optimized for bulk data retrieval, while small requests go to devices with lower latency. This segmentation strategy increases throughput by leveraging the complementary strengths of multiple devices.
Solution Approach 2:
The scheduler dynamically determines which storage devices should handle which I/O requests based on request size and current system state. This dynamic allocation allows the system to adapt to varying workloads and optimize data retrieval paths in real-time, maximizing productivity while distributing the complexity of device management across the scheduling layer.
Data Source
AI summary
A system is disclosed. A storage device may store a data. A load module may read the data from the storage device based at least in part on an input/output (I/O) request. A scheduler may place the I/O request in a queue for delivery to the load module based at least in part on a size of the I/O request being less than a threshold.