Key-Value Storage Prefix Compression and Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Key-value distributed storage systems face inefficiencies in storage space utilization, particularly when handling complex queries and high concurrency, as they are optimized for fast search speeds but not for complex criteria retrieval, leading to suboptimal use of storage resources.
Innovation Solution
A data storage method and apparatus that employs fixed-length and variable-length key storage strategies, where fixed-length keys store a common prefix uniformly and the remainder separately, and variable-length keys undergo prefix compression, optimizing storage space usage by reducing data block sizes through prefix compression and efficient data block management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If key-value distributed storage systems store data in traditional formats, then search speed is fast, but storage space utilization is low
Solution Approach 1:
The key is segmented into a common prefix and a remainder part. The common prefix is stored uniformly once, while each key only stores its unique remainder part. This segmentation allows the system to maintain fast search capability through the uniform prefix structure while significantly reducing storage space by eliminating redundant prefix repetitions across multiple keys.
2Ease of operation
If variable-length keys are stored in full storage manner, then retrieval is simple, but storage space is wasted
Solution Approach 1:
Multiple variable-length keys that share a common prefix are merged into a single stored prefix. The system stores the common prefix once and maintains references to all keys that share it. This merging approach preserves retrieval simplicity through the reference mechanism while dramatically reducing storage space by eliminating redundant prefix storage for each individual key.
3Quantity of substance
If prefix compression is applied to variable-length keys, then storage space is reduced, but data block size management becomes complex
Solution Approach 1:
The system performs preliminary organization of keys by identifying and extracting common prefixes before the actual storage operation. By pre-processing the key structure to separate common prefixes from unique remainders, the system simplifies subsequent data block management while achieving significant storage space reduction through the prefix compression mechanism.
Data Source
AI summary
Implementation manners of the present invention provide a data storage method and apparatus. A fixed-length key and a value thereof are stored into a first data block, where the storing a fixed-length key includes: uniformly storing a common prefix of each fixed-length key, and separately storing a remainder part of each fixed-length key after the common prefix is removed; and a variable-length key and a length thereof are stored into a second data block, where the storing a variable-length key includes: storing a variable-length key of a base-key type in a full storage manner, and performing prefix compression on a variable-length key of a prefix-compressed key type.