Multi-Level Storage Imbalance Control for LSM Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-level data storage devices using an LSM tree, an imbalance occurs between the injection and compaction operations due to varying write performance of different storage devices, leading to increased latency and decreased overall performance, particularly when lower-level devices with lower bandwidth are used.
Innovation Solution
A multi-level storage device with an imbalance control circuit that calculates an imbalance index based on write operation latencies and adjusts write operations to maintain balance by introducing waiting times or altering data movement strategies, such as maintaining data at higher levels to reduce latency and improve read performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If lower-level storage devices with lower bandwidth are used, then cost is reduced, but imbalance between injection and compaction operations increases
Solution Approach 1:
The system dynamically adjusts write operation parameters based on real-time imbalance indices. The controller monitors compaction operation status and dynamically modifies write speeds, waiting times, and data distribution strategies to maintain balance between injection and compaction operations, preventing throughput degradation caused by lower-level storage bandwidth limitations
Solution Approach 2:
The system changes operational parameters such as write speed, waiting time between writes, and data distribution patterns. By adjusting these parameters based on imbalance indices, the system optimizes the trade-off between maintaining high throughput and preventing operation imbalance, even when using lower-bandwidth storage devices
2Speed
If write performance of top level storage device is improved, then number of key-value sets injected increases, but imbalance between injection and compaction operations increases
Solution Approach 1:
The system continuously monitors the imbalance between injection and compaction operations through imbalance indices and uses this feedback to adjust write operation parameters. When imbalance is detected, the controller modifies write speeds, introduces waiting times, or changes data distribution strategies to restore balance, preventing the accumulation of excessive key-value sets
Solution Approach 2:
The system performs preliminary actions by introducing waiting times between write operations and pre-adjusting write parameters based on predicted imbalance. This proactive approach prevents operation imbalance from developing, allowing the system to maintain high write performance without accumulating excessive data that would require compaction
Data Source
AI summary
A multi-level data storage device includes a first storage device; a second storage device located at a lower level than the first storage device; an input/output (I/O) control circuit configured to control a first write operation for the first storage device and a second write operation for the second storage device; and an imbalance control circuit configured to calculate an imbalance index corresponding to a write set that is generated when a sum of a number of first write operations and a number of the second write operations becomes a predetermined number, and configured to control the I/O control circuit to control imbalance of write operations performed in the multi-level data storage device by controlling the first write operation or the second write operation based on the imbalance index.