Blockchain Metadata Management for Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional distributed hash table-based methods for metadata management in multi-cloud environments lack security, robustness, and efficiency, particularly in handling shard storage and replica retrieval, with vulnerabilities to attacks, network partition issues, and inefficient retrieval processes.
Innovation Solution
Implementing a blockchain-based metadata management system where nodes maintain blockchain databases with key-value pairs for node and shard identification information, enabling secure authentication, robust network partition handling, and efficient shard retrieval through associative storage and similarity-based determination of storage positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional distributed hash table-based methods are used for metadata management, then the system structure is simple, but security is lacking and the system is vulnerable to attacks
Solution Approach 1:
The patent introduces blockchain technology as an intermediary layer between nodes and metadata storage. The blockchain ledger acts as a trusted mediator that records and verifies all metadata operations, providing security and authentication without requiring nodes to directly trust each other. This resolves the security vulnerability while maintaining relative system simplicity through the use of established blockchain protocols.
2Reliability
If traditional distributed hash table-based methods are used, then device complexity is low, but robustness during network partitions is poor
Solution Approach 1:
The patent implements preliminary actions by pre-establishing a blockchain ledger that records the state of all nodes and shards before network partitions occur. The system continuously updates the blockchain with node status and shard location information, so that when a network partition occurs, nodes can quickly determine their valid state and continue operations using the pre-recorded blockchain data, ensuring robustness without increasing operational complexity.
3Productivity
If traditional distributed hash table-based methods are used, then system structure is simple, but retrieval efficiency is poor with log(n) time complexity
Solution Approach 1:
The patent adds another dimension to the retrieval process by introducing a two-hop lookup mechanism through the blockchain ledger. Instead of directly searching through all nodes (log(n) complexity), the system first queries the blockchain for shard location information (first hop), then retrieves the actual data from the indicated node (second hop). This dimensional change in the retrieval approach dramatically improves efficiency while leveraging the existing blockchain structure.
4Reliability
If blockchain-based metadata management is implemented, then security and robustness are improved, but device complexity increases
Solution Approach 1:
The patent makes the blockchain ledger serve multiple functions simultaneously: it acts as a security authentication mechanism, a robustness guarantee during partitions, a retrieval index, and a node state recorder. By making the blockchain multi-functional, the system achieves improved security and robustness without proportionally increasing complexity, as the same underlying structure provides multiple benefits.
Data Source
AI summary
According to embodiments of the present disclosure, a method, device and computer program product for information processing are proposed. The method comprises: obtaining identification information of a shard of metadata at a first node of a blockchain-based metadata management system; determining, based on similarities of the identification information of the shard and identification information of candidate nodes of the metadata management system, a second node for positioning the shard from the candidate nodes; and enabling the second node to process the identification information of the shard, to manage storage of the shard in the metadata management system. Therefore, the present solution can improve efficiency, security and robustness of the metadata management system.