Kernel-Endpoints for SCSI Volume Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face scalability issues when the number of volumes exceeds 20,000, leading to memory and CPU usage problems due to duplicated front-end configurations in the kernel driver, which affects performance and increases memory consumption.
Innovation Solution
The introduction of Kernel-Endpoints (KEs) abstracts the front-end configuration from the kernel, replacing the per-volume interface with a set of KEs that are not affected by system scale, allowing for efficient routing of IOs based on LUN IDs and reducing kernel interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If per-volume interface is used in kernel driver, then each volume can be configured separately with dedicated command queue, but memory consumption and CPU usage increase significantly when number of volumes exceeds 20,000
Solution Approach 1:
The patent extracts the front-end configuration data structure from the kernel space and places it in user space. The kernel driver no longer maintains per-volume configuration structures, file descriptors, or command queues. Instead, it uses a simplified interface that references volumes by ID, eliminating the memory overhead of duplicating configuration data in kernel space for each volume.
Solution Approach 2:
The patent introduces a universal volume handle mechanism where a single kernel driver instance manages all volumes through a common interface. Instead of having separate kernel configurations for each volume, the system uses a unified volume identification scheme (volume ID) that works across all volumes, allowing the kernel driver to serve multiple volumes with a single configuration structure.
2Reliability
If per-volume interface is used in kernel driver, then separate command queue per volume can be maintained for fairness, but context switches and CPU overhead increase with system scale
Solution Approach 1:
The patent removes the command queue infrastructure from the kernel driver entirely. Instead of maintaining separate command queues in kernel space for each volume, the system uses user-space command queues that are managed by the user-space daemon. The kernel driver processes commands synchronously without requiring kernel-space queue structures, eliminating the associated context switch overhead.
Solution Approach 2:
The patent introduces a user-space daemon as an intermediary between the application layer and the kernel driver. This daemon handles command queue management, volume configuration, and I/O scheduling in user space, allowing the kernel driver to focus solely on executing I/O operations without the overhead of maintaining complex queue structures and performing context switches.
3Adaptability or versatility
If front-end configuration is duplicated in kernel driver, then each volume can be created in both kernel and storage system software, but system complexity and memory usage increase
Solution Approach 1:
The patent extracts all front-end configuration data structures from the kernel driver, including volume configuration, initiator group configuration, and mapping information. These configurations are now maintained exclusively in user space by the storage system software, simplifying the kernel driver to a stateless I/O execution engine that receives commands with volume IDs and executes them without maintaining configuration copies.
Data Source
AI summary
A method, computer program product, and computer system for placing, by a computing device, an IO in a Kernel-Endpoint (KE) of a plurality of KEs, wherein the IO may be sent by a host and received in a kernel. The IO in the KE may be pulled from a kernel volume specific handle. The IO may be routed to a corresponding user volume associated with the IO.