Key-Value Store Partial Data Access via Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing key-value stores face inefficiencies due to the need to access and modify entire values when only a portion is requested, leading to increased traffic, memory usage, and wear on storage media, as they lack partial data access capabilities.
Innovation Solution
The method involves generating smaller block objects with new keys and values based on the original data object's key and metadata, allowing for partial data access by storing these block objects in the key-value store instead of the entire data object, thereby enabling efficient access to specific portions of data without accessing the full value.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the full value is accessed when only a portion is requested, then the key-value store can maintain simple object storage architecture, but data traffic, memory usage, and processing overhead increase significantly
Solution Approach 1:
The patent divides the value into multiple blocks of fixed size (e.g., 64KB each), where each block can be independently stored and accessed. This segmentation allows the system to retrieve only the necessary blocks rather than the entire value, reducing data traffic and processing overhead while maintaining a relatively simple storage architecture.
Solution Approach 2:
The patent introduces a new dimension to the key-value store by adding block-level addressing capabilities. Instead of only accessing values at the object level, the system now operates at a finer granularity (block level), enabling partial reads and writes without requiring the full value to be loaded into memory.
2Device complexity
If the full value is read and modified when only a portion needs updating, then the key-value store structure remains simple, but storage wear and write amplification increase
Solution Approach 1:
By segmenting values into blocks, the system can perform write operations at the block level rather than requiring full value replacement. This reduces write amplification and extends storage media lifespan by minimizing the number of write cycles, while the store structure remains relatively simple through the use of block pointers and indexing mechanisms.
3Ease of operation
If block objects are created and stored instead of the entire data object, then partial data access is enabled, but the key-value store complexity increases
Solution Approach 1:
The patent introduces block objects as intermediary elements between the original data objects and the storage system. Each block object contains a pointer to the corresponding data block and metadata for tracking, serving as a mediator that enables partial access while organizing the complexity into a manageable layer structure.
Solution Approach 2:
When a data object is first stored, the system preliminarily divides it into blocks and creates corresponding block objects with metadata. This preliminary organization enables subsequent partial access operations to proceed efficiently without requiring complex on-demand segmentation, as the block structure is pre-established.
Data Source
AI summary
Apparatuses, systems, methods, and computer program products are disclosed for key-value stores with partial data access. An interface module is configured to receive a data object for storage in a key-value store. The data object may include a key and a value. A block object module is configured to generate a plurality of block objects smaller than the data object. A block object may include a new key and a new value. The new key may be based on the key for the data object and on metadata for the new value. The new value may be based on at least a portion of the value for the data object. A storage module is configured to store the block objects in the key-value store.