Key-Value Storage Range Query Stability During Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional key-value based data storage devices using a Log-structured Merge (LSM) tree structure struggle to manage summary tables effectively during compaction operations, which can prevent the completion of range query commands.
Innovation Solution
A key-value based data storage device is designed with a controller that processes range query commands by using summary data with version data, including a global version and a copy version. The copy version is created at the start of the range query command and referenced during processing, ensuring stability even during compaction operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a compaction operation is performed during range query processing, then the LSM tree structure is optimized, but the range query command cannot be completed due to summary table changes
Solution Approach 1:
The system creates a copy of the summary table before the range query command is executed. This preliminary action ensures that the query operates on a stable snapshot of the data structure, preventing interference from subsequent compaction operations that may modify the original summary table.
Solution Approach 2:
A copy of the summary table is generated and stored in memory before the range query is processed. This copied summary table remains unchanged during compaction operations, allowing the query to complete successfully while the original summary table can be updated by compaction in the background.
2Stability of the object's composition
If the summary table is updated during range query processing, then the data structure remains current, but the query results become inconsistent
Solution Approach 1:
The summary table is copied before the range query begins processing. This ensures that the query operates on a consistent snapshot of the data structure, preventing any subsequent updates or modifications to the original summary table from affecting the query results.
Solution Approach 2:
A snapshot copy of the summary table is created and stored in the memory buffer. This copied version is used exclusively for range query operations, while the original summary table can be freely updated by compaction operations without impacting query consistency.
3Productivity
If compaction operations are performed frequently, then storage efficiency is improved, but range query commands are interrupted more often
Solution Approach 1:
The system proactively creates a copy of the summary table at the start of each range query command. This preliminary copying action ensures that frequent compaction operations can proceed without interrupting the query, as the query operates on the stable copied version while compaction modifies the original.
Solution Approach 2:
By maintaining a copied summary table in memory, the system decouples the query processing from the compaction operations. This allows compaction to be performed frequently for storage optimization while queries continue uninterrupted using the stable copy.
Data Source
AI summary
A key-value (KV) based data storage device configured to process one or more range query commands includes a first table, summary data, a Log-Structured merge (LSM) tree area storing a plurality of second tables forming an LSM tree structure, and a value log area storing a value corresponding to a key stored in the LSM tree structure. The summary data includes version data including a global version representing current states of the first table and the plurality of second tables. The KV based storage device adds a copy version of the global version in the version data when initiating processing of a range query command and refers to that copy version while processing that range query command.