Interactive continuous in-device transaction processing using key-value solid state drives

Abstract

Various aspects include an interactive continuous in-device KV transaction processing system and method. The system includes a host device and a KV-SSD. The KV-SSD includes a command handler module to receive and process command packets from the host device, to identify KV input / output (I / O) requests associated with a KV transaction, and to prepare a per-transaction index structure. The method includes receiving a command packet from a host device, and determining, by the command handler module, whether a transaction tag associated with the KV transaction is embedded in the command packet. Based on determining that the transaction tag is not embedded in the command packet, the method includes processing one or more KV I / O requests using a main KV index structure. Based on determining that the transaction tag is embedded in the command packet, the method includes individually processing the one or more KV I / O requests using a per-transaction index structure.

Description

technical field [0001] Presented embodiments relate to key-value (KV) storage systems, and more particularly, to systems and methods for providing interactive sequential in-device transactions using KV solid-state drives (KV-SSDs). Background technique [0002] Transactions are an integral part of the KV storage system, which ensures the consistency of KV pairs with applications. Transactions can be implemented using batch operations where a set of KV pairs can be staged in memory and sent to the infrastructure. Infrastructure devices can be stateless. However, because bulk writes may not be done atomically, additional redundancy, such as journaling or write-ahead logging (WAL), may be required, increasing write amplification, thus achieving sub-optimal performance and shortening the lifetime of the device. [0003] Offloading batched transactions to a storage device can help mitigate the side effects of batched transactions, but since the size of the transactions may be l...

AI Technical Summary

Problems solved by technology

However, because bulk writes may not be done atomically, additional redundancy such as journaling or write-ahead logging (WAL) may be required, increasing write amplification, thus achieving suboptimal performance and shortening the lifetime of the device

[0003] Offloading batched transactions to a storage device can help mitigate the side effects of batched transactions, but because the size of the transaction may be limited by the size of the write buffer in the device, problems may still exist

Additionally, an application running on the host may still need to keep the entire transaction in memory while it is being processed by the device

Therefore, it is difficult to share transactions with other applications running in the same or different storage nodes

It may be difficult or impossible to share dynamic views of data with others based on the output of individual requests

Images