Blockchain Data Processing Device Using LSM and B+ Tree Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Blockchain systems using LevelDB experience decreased performance in storage and retrieval efficiency as data increases, due to the B+ tree data structure's O(nlogn) complexity, which is exacerbated by append-only operations.
Innovation Solution
Implementing a blockchain-based high-speed data processing device that stores values in a Log Structured Merge (LSM) tree data structure and keys in a B+ Tree data structure, allowing for improved storage and retrieval performance by separating key and value management units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If B+ tree data structure is used to store blocks in LevelDB, then data can be stored efficiently, but storage and retrieval performance deteriorates as data increases due to O(nlogn) complexity
Solution Approach 1:
The patent segments the storage system into two distinct components: a B+ tree structure for storing block keys and metadata, and an LSM tree structure for storing block values. This segmentation allows each data structure to be optimized for its specific function, with the B+ tree providing efficient key lookup and the LSM tree providing high-performance value storage and append-only operations.
Solution Approach 2:
The patent introduces an intermediary mechanism that maps block keys from the B+ tree to corresponding value storage locations in the LSM tree. This intermediary layer enables the system to leverage the strengths of both data structures: the B+ tree's efficient key management and the LSM tree's high-performance value storage, thereby resolving the performance deterioration issue.
2Reliability
If append-only operations are performed in blockchain system, then data integrity is maintained, but performance is inevitably lowered as blocks increase
Solution Approach 1:
The patent changes the fundamental parameter of how append-only operations are implemented by switching from traditional B+ tree structures to LSM tree structures for value storage. LSM trees are specifically designed to handle append-only workloads efficiently by organizing data in sequential writes to log files, which maintains data integrity while providing O(1) append performance regardless of data volume.
Solution Approach 2:
The patent introduces dynamic compaction mechanisms in the LSM tree that automatically optimize the storage structure as data grows. The system dynamically adjusts between write-optimized log storage and read-optimized compacted storage, maintaining high performance for append-only operations while adapting to increasing data volumes without compromising data integrity.
Data Source
AI summary
Proposed are a blockchain-based high-speed data processing device and a method thereof, which can manage data in a data structure format that can maximize performance when a blockchain system processes data. According to an embodiment, when <Key>:<Value> is stored, the final value is stored in a Log Structured Merge (LSM) tree data structure format, and the key used for storing the final value is separately stored in a B+ Tree data structure format, so that storage performance and reference (read) performance can be improved compared to those of conventional methods.