Log-Structured Storage for Blockchain Data Tiering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed ledger systems (DLSs) face high storage costs and inefficiencies due to each node storing a complete copy of blockchain data, leading to demanding storage requirements as the system scales.
Innovation Solution
A log-structured storage system that uses a tiered storage approach with append-only data files and index log files, supporting data tiering, compression, and erasure coding, to optimize storage and reduce costs by efficiently managing blockchain data across multiple storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each node stores a complete copy of blockchain data, then data reliability and trustworthiness are improved, but storage costs and storage requirements increase
Solution Approach 1:
The patent segments blockchain data into two categories: immutable data (transaction data, block data) that is stored on the blockchain, and mutable data (state data) that is stored externally. This segmentation allows nodes to store only essential immutable data while keeping mutable data externally, reducing storage requirements while maintaining data reliability through the immutability guarantee of blockchain-stored data.
2Adaptability or versatility
If each node stores a complete copy of blockchain data, then independence of each node is improved, but storage costs increase
Solution Approach 1:
The patent introduces an intermediary external storage system that holds mutable state data, allowing nodes to access this data without storing complete copies locally. The intermediary storage acts as a shared resource that all nodes can access, reducing individual storage costs while maintaining node independence through direct access to required data.
3Ease of manufacture
If traditional storage schemes are used, then implementation simplicity is maintained, but storage efficiency deteriorates
Solution Approach 1:
The patent segments the storage system into blockchain storage for immutable data and external storage for mutable data, with clear separation of concerns. This segmentation improves storage efficiency by placing different types of data in appropriate storage media while maintaining implementation simplicity through well-defined interfaces and protocols for data access and synchronization.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage devices, for data processing and storage. One of the method includes: maintaining a plurality of tiers of storage devices and one or more tiers of caches by a storage system for storing blockchain data, the plurality of tiers of storage devices including at least a higher-tier storage device and a lower-tier storage device; determining that a blockchain data object in a data log file stored in a lower-tier storage device is an active data object, wherein the blockchain data object is block data, transaction data, or state data; and writing the data object into a cache of the one or more tiers of caches.