Multi-Device Storage Queues for Low-Latency Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accessing large amounts of data from a single storage device forms a bottleneck, leading to increased data access latency and potential failure to meet service level agreements in applications like deep learning recommendation models.
Innovation Solution
A system with multiple storage devices and a scheduler that distributes data across these devices based on request size, using queues and load modules to manage I/O requests, ensuring low latency by parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is accessed from a single storage device, then device complexity is low, but data access latency increases and productivity decreases
Solution Approach 1:
The patent segments the storage system into multiple independent storage devices (e.g., SSD1, SSD2, SSD3) with separate I/O queues (Q0, Q1, Q2). Each queue is dedicated to specific storage devices, allowing parallel data retrieval operations. This segmentation enables simultaneous data access from multiple devices, resolving the bottleneck of single-device sequential access and improving overall data access speed.
Solution Approach 2:
The patent introduces a multi-dimensional queue management structure where I/O requests are routed to different queues based on their characteristics (size, priority, type). This dimensional organization of request handling enables concurrent processing of multiple data access operations, transforming the single-dimensional sequential access model into a multi-dimensional parallel access model that significantly reduces latency.
2Loss of time
If multiple storage devices are used, then data access latency is reduced, but device complexity increases
Solution Approach 1:
The patent divides the I/O request management into separate queues (Q0, Q1, Q2), each responsible for specific storage devices. This segmentation isolates the complexity of managing multiple devices into independent queue units, making the overall system more manageable while enabling parallel data retrieval operations that reduce access time.
Solution Approach 2:
The patent introduces a scheduler as an intermediary component that receives I/O requests and routes them to appropriate queues based on request characteristics. This intermediary layer abstracts the complexity of multi-device management from the application layer, providing a simplified interface while enabling efficient parallel data access across multiple storage devices.
3Productivity
If I/O requests are processed sequentially, then queue management is simple, but data access latency increases
Solution Approach 1:
The patent segments I/O requests into different queues (Q0, Q1, Q2) based on their characteristics such as size and priority. This segmentation allows concurrent processing of requests in different queues, significantly improving throughput. The complexity is managed by maintaining separate queue structures that can be independently managed.
Solution Approach 2:
The patent implements dynamic queue scheduling where the scheduler adapts its routing decisions based on request characteristics and system state. Queues can be dynamically selected and managed according to current workload conditions, enabling optimal parallel processing while managing complexity through adaptive rather than static scheduling approaches.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system is disclosed. A storage device may store a data. A load module may read the data from the storage device based at least in part on an input/output (I/O) request. A scheduler may place the I/O request in a queue for delivery to the load module based at least in part on a size of the I/O request being less than a threshold.