Key Value Store Zoned Storage Flush Compaction Segmentation

Solution Overview

Problem

Current key value store implementations on zoned storage experience significant interference in input/output performance due to concurrent flush and compaction operations, leading to increased latency and user experience issues.

Innovation Solution

The solution involves classifying storage zones into flush zones and compaction zones, where flush operations are performed using active and immutable flush zones, and compaction operations are performed using compaction zones, thereby minimizing interference during level 0 compaction operations and optimizing storage based on a log structured merge-tree (LSM-Tree) data structure.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If flush and compaction operations are performed concurrently in the foreground and background, then storage bandwidth utilization is improved, but input/output performance interference increases causing latency spikes

Engineering Contradiction:

Improvestorage bandwidth utilizationVSAvoidinput/output performance

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The storage device is divided into multiple zones (first zone, second zone, third zone, fourth zone) where the first and second zones are dedicated to flush operations while the third and fourth zones are dedicated to compaction operations. This spatial segmentation allows concurrent execution of flush and compaction operations without interference, resolving the contradiction between storage bandwidth utilization and input/output performance.

Inventive Principle:

Principle #1Segmentation

2Productivity

If compaction operation reads and writes large amounts of data, then compaction efficiency is improved, but storage bandwidth consumption increases causing interference with flush operation

Engineering Contradiction:

Improvecompaction efficiencyVSAvoidstorage bandwidth consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The storage device is divided into multiple zones (first zone, second zone, third zone, fourth zone) where the first and second zones are dedicated to flush operations while the third and fourth zones are dedicated to compaction operations. This spatial segmentation allows concurrent execution of flush and compaction operations without interference, resolving the contradiction between storage bandwidth utilization and input/output performance.

Inventive Principle:

Principle #1Segmentation

3Speed

If zone-based storage is used to improve sequential write performance, then write throughput is improved, but random read performance may be degraded due to zone boundaries

Engineering Contradiction:

Improvesequential write performanceVSAvoidrandom read performance

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

Different zones are assigned different functional characteristics: the first and second zones are optimized for flush operations (sequential writes), while the third and fourth zones are optimized for compaction operations. Within each zone, data is stored with local quality characteristics that optimize for the specific operation type, allowing sequential write performance to be improved without significantly impacting random read performance.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12079195B2Apparatus and method for storing file of key value store on zoned storage

Publication Date: 2024.09.03 IND ACADEMIC COOP FOUND YONSEI UNIV

AI summary

An apparatus for storing a key value store file according to an embodiment includes a memory configured to record one or more key values in a predefined unit space based on a data input request from an outside, a controller configured to store data received from the memory in a storage, and the storage configured to include a plurality of zones. The controller is configured to perform a flush operation of storing one or more key values received from the memory as a file in a predefined format in the storage and a compaction operation of merging a plurality of files existing in one level in the storage and recording the merged files as one file in another level.