Private I/O Queues for Kernel Bypass in Storage Controllers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems experience latency issues due to the segregation of virtual memory into kernel and user spaces, which increases the time it takes to execute storage commands as they traverse the kernel stack, impacting the performance of distributed storage systems.
Innovation Solution
Implementing concurrent user space and kernel space access through private, reserved I/O queues allows applications to bypass the kernel space for latency-sensitive commands, reducing latency by enabling direct access to storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If storage commands are routed through the kernel space for memory protection and hardware protection, then system security and stability are improved, but execution latency increases
Solution Approach 1:
The patent segments the I/O queue access paths into two distinct channels: kernel-space queues for administrative commands requiring security checks, and user-space queues for latency-sensitive I/O commands. This segmentation allows commands to be routed through appropriate paths based on their security requirements, enabling user-space applications to bypass the kernel stack for time-critical operations while maintaining security for administrative functions.
Solution Approach 2:
The patent introduces an intermediary mechanism where the device driver allocates and manages separate I/O queues that act as intermediaries between user-space applications and storage devices. These private queues serve as a buffer and communication channel that allows direct user-space access without requiring kernel-space mediation for each individual command, thus reducing latency while maintaining system control.
2Reliability
If all storage commands traverse the kernel stack for processing, then centralized control and security are maintained, but processing speed and throughput decrease
Solution Approach 1:
The patent divides the command processing pathway into segmented queues: administrative queues handled by the kernel-space device driver and private I/O queues accessible from user-space. This segmentation enables parallel processing where latency-sensitive commands can be submitted directly from user-space without traversing the entire kernel stack, while administrative commands continue to use the traditional kernel-mediated path, thus maintaining centralized control for management functions while improving overall processing speed.
Solution Approach 2:
The patent implements dynamic queue allocation where the device driver can allocate multiple I/O queues with different characteristics based on workload requirements. Applications can dynamically select appropriate queues for different types of operations, allowing the system to adapt processing paths in real-time based on performance requirements while maintaining centralized management through the device driver.
3Loss of time
If private I/O queues are allocated for direct user space access, then latency is reduced for I/O commands, but device complexity increases
Solution Approach 1:
The patent implements a universal queue management approach where the device driver maintains a pool of allocatable I/O queues that can serve multiple applications and multiple command types. This multi-functional queue system allows the same infrastructure to handle both administrative commands and user-space I/O commands, reducing the need for separate specialized mechanisms and thereby limiting the increase in device complexity despite the performance benefits.
Solution Approach 2:
The patent changes the parameter of queue accessibility from exclusively kernel-space to multi-space (kernel and user), allowing queues to be accessed from different execution contexts. This parameter change enables direct user-space access to reduce latency while the device driver maintains control over queue allocation and management, balancing performance improvement with manageable complexity through existing driver infrastructure.
Data Source
AI summary
A storage controller coupled to a storage array includes a device driver running in a kernel space that receives an administrative command from an application running in a user space of the storage controller and writes the administrative command to a first submission queue of a plurality of submission queues associated with a storage device in the storage array, where the first submission queue is reserved for use by the device driver. An input/output (I/O) command received from the application running in the user space, however, is written directly to a second submission queue of the plurality of submission queues without being routed through the kernel space, where the second submission queue being reserved for direct access by the application running in the user space.