Key-Value Store Direct Mapping for Cloud Storage Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in processing data operations due to the need for intermediate software layers like file systems and logical volume managers, which increase processing complexity and latency when handling key-value pairs.
Innovation Solution
A method and system that directly map key-value pairs to internal data storage system representations using a mapping table, allowing data operations to be performed without intermediate translations, and utilizing a key-value server to process requests according to an application-level protocol, thereby reducing reliance on block-based and file-based protocols.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If intermediate software layers (file systems and logical volume managers) are used to process data operations, then data storage systems can handle diverse data types and access patterns, but processing complexity and latency increase
Solution Approach 1:
The patent extracts and removes the intermediate file system and logical volume manager layers from the data operation path. By directly exposing the key-value store API to applications, it eliminates the unnecessary translation and processing steps that these intermediate layers introduced, thereby reducing latency while maintaining adaptability through the flexible key-value interface.
Solution Approach 2:
The patent segments the storage system into distinct functional layers: the key-value store layer that handles data operations directly, and the underlying physical storage layer. This segmentation allows applications to interact with the optimized key-value interface without passing through bulky intermediate layers, reducing processing complexity while preserving versatility.
2Adaptability or versatility
If intermediate processing steps are used to translate data operations, then compatibility with traditional storage interfaces is maintained, but performance and efficiency deteriorate
Solution Approach 1:
The key-value store acts as a new intermediary layer between applications and physical storage devices. Instead of using traditional file systems and logical volume managers that translate operations through multiple layers, the key-value store provides a direct, efficient interface that maintains compatibility with diverse data access patterns while improving throughput by eliminating redundant translation steps.
Solution Approach 2:
The patent inverts the traditional storage architecture by placing the key-value store at the forefront, allowing applications to access data directly through optimized key-value operations. This inversion eliminates the need for applications to go through traditional file system abstractions, thereby improving productivity while maintaining interface compatibility through the standardized key-value protocol.
3Ease of manufacture
If key-value pairs are mapped through traditional storage layers, then data can be stored on physical devices, but mapping complexity and access overhead increase
Solution Approach 1:
The patent extracts the key-value mapping function from the complex traditional storage stack and implements it directly in the key-value store layer. By removing the file system and logical volume manager intermediaries, the mapping process becomes simpler and more direct, reducing access overhead while maintaining the essential capability of mapping logical data to physical storage devices.
Data Source
AI summary
Described are techniques for processing data operations. A request is sent from an application to perform a data operation on a key-value data store. The request is in accordance with a first application-level protocol for performing the data operation on the key-value data store and includes a first key of a first key-value pair used by a data storage system to perform the data operation. The request is received at the data storage system such as by a key-value server that processes requests in accordance with the first application-level protocol for the key-value data store. In response to receiving the request at the data storage system, first processing is performed on the data storage system for the data operation and a response is returned to the application.