Hierarchical Consensus for Distributed Database Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Decentralized distributed databases face performance issues due to high communication complexity in Byzantine Fault Tolerant (BFT) consensus algorithms, leading to low throughput and hinder widespread adoption.
Innovation Solution
A hierarchical consensus method is implemented, where subsets of nodes first reach consensus locally based on their status information, and then main nodes determine a global state, reducing communication complexity to O(n.logn) by minimizing direct node-to-node interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If BFT consensus algorithm is used in decentralized distributed database, then security and fault tolerance are improved, but communication complexity increases to O(N^2) leading to low throughput
Solution Approach 1:
The patent divides the distributed database into multiple shards, where each shard is managed by a subset of nodes rather than requiring all nodes to communicate with each other. This segmentation reduces the communication complexity from O(N^2) in traditional BFT to approximately O((N/k)^2) where k is the number of shards, thereby improving throughput while maintaining fault tolerance within each shard
Solution Approach 2:
The patent introduces a committee structure where a selected subset of nodes acts as intermediaries (committees) for each shard. These committee members handle the consensus process on behalf of the entire network, reducing the number of direct node-to-node communications required while maintaining the security guarantees of BFT consensus
2Adaptability or versatility
If number of nodes in distributed database increases, then decentralization and security are improved, but network performance deteriorates due to O(N^2) communication complexity
Solution Approach 1:
By organizing nodes into multiple shards with dedicated subsets handling each shard, the system allows linear scaling as N increases. Each shard operates independently with its own consensus process, so adding more nodes primarily increases the number of shards rather than the communication overhead per shard, enabling better network performance with increased decentralization
Solution Approach 2:
The patent adds a dimensional organization to the node structure by introducing shard assignments and committee selections. Instead of a flat N-node structure requiring O(N^2) communication, the system creates a multi-dimensional hierarchy (nodes → shards → committees) that reduces communication complexity while maintaining the benefits of having more nodes in the system
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Computer-implemented consensus method for obtaining a valid global state (VGS) for a distributed database, wherein the distributed database comprises main nodes, each main node storing a complete dataset, and a plurality of subsets of nodes, each subset configured to store a part of the dataset, the method comprising the steps: a. receiving (S1) a request for providing a valid global state (VGS) of the distributed database, b. determining (S2) a first local state (1_LS1,..., 1_LSi,..., 1_LSn) for each subset based on status information of nodes of the respective subset regarding the respective part of the dataset, c. combining (S3) these first local states of all subsets to a first combined state (CS1), d. determining (S4) a global state (GS) by the main nodes based on status information of the main nodes regarding the stored dataset, e. verifying (S5) the first combined state (CS1) by matching the first combined state (CS1) with the global state (GS) and f. outputting (S6) the first combined state (CS1) as valid global state (VGS) for the distributed database depending on the verification.