Storage Node Key-Value Store for Disaggregated I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a storage disaggregation environment, existing data input/output methods face scalability limitations and performance degradation due to overhead in the kernel I/O stack and I/O amplification caused by log-structured merge-tree-based key-value storage, leading to reduced data transmission/reception rates between calculation and storage nodes.
Innovation Solution
Implementing a data input/output method using a storage node-based key-value store, where commands from calculation nodes are converted into I/O requests using logical block addresses, allowing direct storage and retrieval of data in a storage device, thereby reducing kernel I/O stack overhead and preventing I/O amplification through hash-based key-value storage in the user space of the storage node.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a key-value store is driven by a calculation node in a storage disaggregation environment, then data storage and retrieval can be performed, but overhead in the kernel I/O stack increases and I/O performance decreases
Solution Approach 1:
The patent extracts the key-value store functionality from the calculation node and relocates it to the storage node. This separation removes the kernel I/O stack overhead from the calculation node, as the key-value operations are now handled directly at the storage node without requiring complex kernel-level I/O processing in the calculation node.
Solution Approach 2:
The patent introduces a simplified user-space interface at the storage node that acts as an intermediary between the NVMe-oF protocol and the key-value store. This intermediary layer handles key-value operations directly in user space, bypassing the need for complex kernel I/O stack processing and reducing overhead.
2Productivity
If log-structured merge-tree-based key-value storage is used, then data can be stored and retrieved, but I/O amplification occurs and performance degrades
Solution Approach 1:
The patent changes the fundamental data structure parameter from log-structured merge-tree to hash table. This parameter change eliminates I/O amplification by providing direct key-to-location mapping through hashing, allowing O(1) access time without the need for multiple I/O operations that characterize LSM-tree structures.
3Adaptability or versatility
If a storage disaggregation environment uses traditional I/O methods, then data can be transmitted between calculation and storage nodes, but scalability is limited and data transmission rates decrease
Solution Approach 1:
The patent segments the storage system into independent storage nodes that can be scaled individually. Each storage node operates autonomously with its own key-value store, allowing the system to scale horizontally by adding more storage nodes without affecting the calculation nodes or requiring complex coordinated I/O operations.
Solution Approach 2:
The patent moves the key-value store operation from the calculation node dimension to the storage node dimension. This dimensional shift allows data transmission to occur directly at the storage node level using simple user-space interfaces, bypassing the need for complex kernel-level coordination and enabling faster data transmission rates.
Data Source
AI summary
Provided is a data input/output (I/O) method using a storage node-based key-value store in a storage disaggregation environment. The data I/O method using the storage node-based key-value store includes receiving a command converted from a key-value write request of an application from a calculation node according to a communication protocol used in the storage node, converting the command into an I/O request using a key included in the command, the I/O request including a logical block address and a value corresponding to the key, and storing the value in a storage device of the storage node using the logical block address of the I/O request.