Counter-Based Compaction for Key-Value Store Tree Data Blocks

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Key-value store (KVS) tree data structures face efficiency issues due to accumulation of garbage data, leading to decreased search performance and increased write operations on storage media, particularly in systems using lower endurance memory devices like QLC NAND cells.

Innovation Solution

A counter-based merge operation is implemented to merge sub-sequences of sub-indexes into a merged sub-index, reducing the length of the index sequence, eliminating garbage data, and minimizing write operations by assigning merge count values and replacing sub-sequences with merged indices, thereby improving query performance and reducing data written to storage media.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If key-value store tree data structure is used for data storage, then data organization and retrieval capability is improved, but garbage data accumulation occurs leading to decreased search performance

Engineering Contradiction:
Improvesearch performanceVSAvoidgarbage data accumulation
Core Design Contradiction:
ProductivityVSLoss of information

Solution Approach 1:

The patent implements a compaction operation that identifies and removes obsolete key-value pairs (garbage data) from the tree structure. When a key is updated or deleted, the old version is marked as obsolete but retained in the structure. The compaction process systematically identifies these obsolete entries and removes them, recovering storage space and maintaining search performance by preventing garbage data accumulation.

Inventive Principle:
Principle #34Discarding and recovering

2Productivity

If frequent merge operations are performed to eliminate garbage data, then search performance is improved, but write operations on storage media increase

Engineering Contradiction:
Improvesearch efficiencyVSAvoidwrite operations on storage media
Core Design Contradiction:
ProductivityVSLoss of substance

Solution Approach 1:

The patent implements incremental compaction that processes only a portion of the tree structure at each operation rather than performing complete merges. The compaction operation traverses the tree and compacts subsets of nodes, progressively eliminating garbage data through multiple partial operations. This approach maintains search performance by continuously removing obsolete data without requiring full-tree rewrites, thereby reducing total write operations on storage media.

Inventive Principle:
Principle #16Partial or excessive action

3Loss of substance

If counter-based merge operation is implemented to reduce write operations, then write amplification is reduced, but system complexity increases

Engineering Contradiction:
Improvewrite amplificationVSAvoidsystem complexity
Core Design Contradiction:
Loss of substanceVSDevice complexity

Solution Approach 1:

The patent implements a counter-based compaction mechanism where each node in the tree maintains a counter value that automatically tracks the number of obsolete key-value pairs. When the counter reaches a threshold, the node is automatically scheduled for compaction without requiring external intervention or complex decision-making logic. This self-service approach reduces write amplification by triggering compaction only when necessary, while keeping system complexity manageable through simple counter-based decision rules.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS11599552B2Counter-based compaction of key-value store tree data block
Publication Date: 2023.03.07 MICRON TECHNOLOGY INC
  • US11599552B2 patent drawing
  • US11599552B2 patent drawing
  • US11599552B2 patent drawing

AI summary

Aspects of the present disclosure provide for operations of a key-value tree data structure that merges key-value pair data of a node, in a key-value tree data structure using counter values.