Upsert Token Key-Value Storage Update
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional key-value/object-storage systems experience increased latencies and CPU utilization due to multiple software layers, particularly the mapping from the key-value interface to logical block addresses, leading to inefficiencies in update operations with high write amplification and read amplification.
Innovation Solution
Implementing a direct Key-to-Physical mapping using hash tables and logical bands within the storage device, introducing an upsert command that allows for efficient update-insert operations by inserting an upsert token with a pointer to the original location, and supporting value coalescing to minimize write amplification and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional multi-layer software mapping from key-value interface to logical block addresses is used, then data storage functionality is achieved, but latency increases and CPU utilization increases
Solution Approach 1:
The patent extracts the key-value mapping functionality from the host CPU software layers and relocates it to the storage device controller. The controller now performs hash-based key-to-physical-address mapping directly, eliminating the need for complex host-side mapping software and reducing host CPU utilization while decreasing update latency.
Solution Approach 2:
The patent introduces an upsert token as an intermediary data structure that facilitates efficient update operations. The token contains the key, new value, and original location information, allowing the storage device to handle updates without requiring the host to read-modify-write through multiple software layers, thus reducing latency and CPU overhead.
2Productivity
If conventional read-update-rewrite operations are performed, then KV pair updates are achieved, but read amplification increases and write amplification increases
Solution Approach 1:
The patent performs preliminary actions by pre-calculating the new physical address for the updated KV pair using hash-based mapping before the actual update occurs. The upsert token is prepared with all necessary information (key, new value, original location, new location), allowing the storage device to directly write the update without performing read-modify-write cycles, thereby reducing read and write amplification.
3Loss of energy
If direct KV interface mapping is implemented in storage device, then write amplification is reduced, but update operations still suffer from get-modify-put penalty
Solution Approach 1:
The patent merges the update and insert operations into a single atomic upsert operation. Instead of separately reading the KV pair, modifying it, and writing it back (get-modify-put), the system combines these into one operation that directly writes the updated value to a new location and updates the hash table, eliminating the time penalty of multiple sequential operations while maintaining low write amplification.
Data Source
AI summary
An update-insert (“upsert”) interface manages updates to key-value storage at a memory or storage device. An upsert token is used to store a key and data for a transform to update a previous value stored for a key-value pair. The upsert token processing includes an upsert command to generate the upsert token for an existing key-value pair and store the upsert token in one or more first non-volatile memory (NVM) devices maintained at a memory or storage device. A hash-to-physical (H2P) table or index stored in one or more second NVM devices of the memory or storage device is utilized to locate and read the data for the key and the data for the transform and coalesce the transform(s) into a current value for the key-value pair, thereby avoiding unnecessary read and write amplification when updating key-value storage.