An asynchronous dynamic consensus method based on sharding

Abstract

The present invention discloses an asynchronous dynamic consensus method based on sharding, comprising the following steps: 1) guiding the election node to generate a sharding committee and enter the first era; 2) forming a consensus within the committee, processing blocks for cross-shard transactions, and forming a routing protocol between the committees; 3) the committee is reconstructed to solve the node storage space problem. The sharding asynchronous dynamic consensus algorithm for low-orbit satellites provided by the present invention has the following advantages: 1. First, it provides innovative ideas for the state sharding system in many aspects, and its design ideas and system architecture have great reference value. For example, the processing method of cross-sharding transactions, sharding routing protocols, committee reconstruction schemes, multi-round protocols for each era, etc.; 2. There are significant improvements in all aspects of performance, the system can tolerate the proportion of malicious nodes to 1 / 3, the throughput is increased to as high as 7380tps, the transaction confirmation delay is reduced to seconds, and the system robustness is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of electronic information security, and in particular relates to an asynchronous dynamic consensus method based on sharding. Background Art

[0002] In a distributed network, ensuring distributed consistency (all nodes reach consensus on the same proposal or data) is the core and most important issue. Due to the large number of nodes in a distributed network, communication delays are inevitable in the network, and nodes may experience complex situations such as downtime, failure, and failure.

[0003] Blockchain provides a good idea for solving the consistency problem of distributed systems, but traditional blockchain systems have problems such as low throughput, low scalability, high latency, low storage performance, and witch attacks, making it difficult to meet the requirements of high security and high scalability. Sharding is currently the best solution for balancing efficiency, security, and scalability. There are also some solutions in the industry that use sharding technology. Existing blockchain protocols based on sharding still require linear communication volume (number of participants) for each transaction, so the potential benefits of sharding can only be partially realized.

[0004] Therefore, how to design an efficient sharding dynamic consensus method for large-scale application scenarios is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The present invention overcomes the deficiencies of the prior art and provides an asynchronous dynamic consensus method based on sharding.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] S1. Guide the election nodes to generate the shard committee and enter the first era;

[0008] S2. The committee forms a consensus within itself, processes cross-shard transactions, and forms a routing protocol between the committees.

[0009] S3,committee reconstruction to solve the node storage space problem.

[0010] Preferably, in S1, the method of electing nodes to form a root group includes:

[0011] S101. Each initial participating node creates a random bipartite graph G(L, R) locally using hard-coded parameters and random seeds, where L is a node and R is a group. The neighbors of each node in R are randomly, independently and uniformly selected from L, changing from level 0 to level 1 to form an initial group.

[0012] S102. The initial group generates a random string s by running a distributed random number generation algorithm. Each node performs a hash operation on s and the ID of the subset. The nodes whose hash values ​​fall within a certain interval broadcast that the subset is the selected node. The other nodes in the group sign the identity ID and string s to prove that these nodes are the selected nodes. These selected nodes broadcast their identities, including the proof, to the entire network.

[0013] S103, these selected nodes form a root group;

[0014] S104. The root group is responsible for selecting members of the reference committee. The reference committee randomly divides the set of all nodes into shard committees and then enters the first era.

[0015] Preferably, in step 2, the method for reaching consensus within the committee includes:

[0016] S201. In each round of consensus formation, each committee selects a leader using the epoch random source;

[0017] S202, the leader collects transactions from the client and packages them into block Bi;

[0018] S203, the leader uses the IDA-Gossip protocol to broadcast the block Bi and creates a block header Hi containing the merkle tree and the round number;

[0019] S204. Committee members initiate a consensus protocol for block header Hi.

[0020] More preferably, in S204, the leader adds a propose tag to the block header Hi and broadcasts Hi through the network; after each node receives the block header Hi, each node responds to the received block header Hi, adds a response tag to Hi, and broadcasts it through the network.

[0021] Better yet, if a node receives a different version of a block header, it indicates that the leader is malicious, and the node rejects the block by broadcasting a pending tag. Conversely, if a node receives only one block header Hi and more than mf+1 response tags, it indicates that all committees have reached a consensus on block Bi. Once a consensus is reached among the committees, each node that receives at least mf+1 response tags broadcasts the block header again, and adds a received tag and a proof that the node was selected.

[0022] More preferably, for messages between different committees, each sender only sends to the nodes of the target committee he knows, and the node that receives the message sends it to other members of the committee through the IDA-gossip protocol; for client users, they only need to submit the transaction to any committee, and the committee that receives the transaction will forward the transaction to the target committee through the Kademlia routing protocol.

[0023] Preferably, in S3, based on the cuckoo principle, the reference committee divides the committee into two categories, namely, an active committee with a majority of active members, and a passive committee with a majority of inactive members; when a new node joins, the reference committee randomly adds the node to any active committee, and randomly adds a fixed number of nodes in the committee to different passive committees to complete the reconstruction.

[0024] Preferably, in S3, based on the idea of ​​state sharding, the storage space is reduced to 1 / k of the original, where k is the number of shards, thereby significantly reducing the storage space consumption of each shard.

[0025] The sharded asynchronous dynamic consensus algorithm for low-orbit satellites provided by the present invention has the following advantages:

[0026] 1. First of all, it provides innovative ideas for the state sharding system in many aspects. Its design ideas and system architecture have great reference value. For example, the processing method of cross-shard transactions, shard routing protocol, committee reconstruction plan, multi-round protocol in each era, etc.

[0027] 2. There are significant improvements in performance in all aspects, increasing the system's tolerable malicious node ratio to 1 / 3, throughput to 7380tps, transaction confirmation delay to seconds, and system robustness significantly improved.

[0028] 3. It includes a committee access control scheme based on the cuckoo principle, which can randomly add new nodes to the committee and resist partition attacks by adaptive opponents. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0030] Figure 1 It is a flow chart of an asynchronous dynamic consensus method based on sharding of the present invention;

[0031] Figure 2 It is a schematic diagram of the three important stages in the fast chain system of the present invention;

[0032] Figure 3 It is a schematic diagram of the three stages of the concept of era used in the fast chain of the present invention;

[0033] Figure 4 This is a flow chart of the present invention for selecting nodes to form a root group;

[0034] Figure 5 This is a schematic diagram of the three steps for the members of the present invention committee to reach a consensus on the block header Hi;

[0035] Figure 6 It is a classification diagram of the limited cuckoo principle of the present invention;

[0036] Figure 7 Schematic diagram of the message forwarding process between members of the present invention. DETAILED DESCRIPTION

[0037] The following is a further detailed description of a sharding-based asynchronous dynamic consensus method in conjunction with specific embodiments. These embodiments are only used for comparison and explanation purposes, and the present invention is not limited to these embodiments.

[0038] like Figure 1 As shown, an asynchronous dynamic consensus method based on sharding includes:

[0039] S1. Guide the election nodes to generate the shard committee and enter the first era;

[0040] S2. The committee forms a consensus within itself, processes cross-shard transactions, and forms a routing protocol between the committees.

[0041] S3,committee reconstruction to solve the node storage space problem.

[0042] like Figure 2 As shown in the figure, the Bootstrapping phase will only be run once at the beginning of the fast chain system. This phase is to create an initial random source and randomly select a special committee, called the reference committee. The members of this reference committee will then randomly assign nodes to form shard committees.

[0043] Preferably, Figure 3 As shown, in S1, the method of electing nodes to form a root group includes:

[0044] S101. Each initial participating node creates a random bipartite graph G(L, R) locally using hard-coded parameters and random seeds, where L is a node and R is a group. The neighbors of each node in R are randomly, independently and uniformly selected from L, changing from level 0 to level 1 to form an initial group.

[0045] S102. The initial group generates a random string s by running a distributed random number generation algorithm. Each node performs a hash operation on s and the ID of the subset. The nodes whose hash values ​​fall within a certain interval broadcast that the subset is the selected node. The other nodes in the group sign the identity ID and string s to prove that these nodes are the selected nodes. These selected nodes broadcast their identities, including the proof, to the entire network.

[0046] S103, these selected nodes form a root group;

[0047] S104. The root group is responsible for selecting members of the reference committee. The reference committee randomly divides the set of all nodes into shard committees and then enters the first era.

[0048] Preferably, in step 2, the method for reaching consensus within the committee includes:

[0049] S201. In each round of consensus formation, each committee selects a leader using the epoch random source;

[0050] S202, the leader collects transactions from the client and packages them into block Bi;

[0051] S203, the leader uses the IDA-Gossip protocol to broadcast the block Bi and creates a block header Hi containing the merkle tree and the round number;

[0052] S204. Committee members initiate a consensus protocol for block header Hi.

[0053] More preferably, in S204, Figure 5 As shown in the last step, the committee members reach a consensus on the block header Hi according to the three steps in the figure below: the leader adds a propose tag to the block header Hi and broadcasts Hi through the network; after each node receives the block header Hi, each node responds to the received block header Hi, adds a response tag to Hi, and broadcasts it through the network.

[0054] Better yet, if a node receives a different version of a block header, it indicates that the leader is malicious, and the node rejects the block by broadcasting a pending tag. Conversely, if a node receives only one block header Hi and more than mf+1 response tags, it indicates that all committees have reached a consensus on block Bi. Once a consensus is reached among the committees, each node that receives at least mf+1 response tags broadcasts the block header again, and adds a received tag and a proof that the node was selected.

[0055] How to process cross-shard transactions safely and quickly has always been the biggest challenge facing state sharding. Sharding the storage state means that each node in the system only stores part of the ledger instead of the complete ledger. The design of FastChain is based on the UTXO model. When a cross-shard transaction occurs, the system will assign the transaction to the corresponding committee based on the output hash of the transaction, and the committee that receives the transaction is called the output committee. At the same time, the transaction may have multiple inputs at the same time, and each input comes from a different shard. The committee where the input of the transaction is located is called the input committee.

[0056] When a client submits a transaction, and the system assigns the transaction to the corresponding output committee, the transaction may contain multiple inputs, and the inputs are stored in other committees. In order to verify the validity of the transaction, the verification node of the output committee must be able to verify the validity of each input. So how to verify whether the input from other shards is valid? OmniLedger and FastChain provide two completely different ideas.

[0057] In OmniLedger's solution, the client first makes a request to the input committee, hoping that each input committee will provide proof for its own assets. Each input committee checks the corresponding UTXO. If the UTXO is correct, it locks the corresponding UTXO and feeds back the asset proof to the client; if the UTXO is incorrect, the committee returns a rejection message to the client. The client collects the asset proofs of all input committees and submits them to the output committee. After the output committee verifies it, the client issues an "unlock to commit" unlock submission instruction to complete the transaction.

[0058] OmniLedger's way of handling cross-shard transactions ensures the atomicity of transaction processing, but it also has the following obvious defects:

[0059] First, the client needs to broadcast the transaction to the entire network, and for each transaction at least one proof needs to be generated and submitted, which results in a lot of communication overhead for the entire network.

[0060] Another problem is that the system is too dependent on the client, which violates the light client principle. This also requires the client to understand the entire network structure.

[0061] In one embodiment, there is a transaction tx, which has two inputs In1 and In2 and one output Out, and these two inputs and outputs belong to different shard committees Shard1, 2, and 3. Then the leader of the output committee will create three new transactions. In the transaction processing flow, the leader of the output committee will create three new transactions, which are as follows:

[0062] 1) Tx1, converts the input from Shard1 into the output of Shard3;

[0063] 2) Tx2, converts the input from Shard2 into the output of Shard3;

[0064] 3) Tx3, the input is the output of Tx1 and Tx2, and the output principle is the same as above.

[0065] The leader of the output committee will send Tx1 and Tx2 to Shard1 and Shard2 respectively through the inter-committee routing protocol. Then the leaders of Shard1 and Shard2 will verify the validity of the corresponding transactions respectively. If valid, they will add the transactions to the ledgers of their own shards and respond to Shard3. Shard3 will add Tx3 to its own ledger after receiving the responses from all input committees.

[0066] The confirmation requests between shards can be processed in batches. In each round, the output committee can send the UTXO transaction combination belonging to the same input committee to the input committee in batches at one time. In this way, the input committee can also verify the validity of UTXO transactions in batches and give feedback to the output committee in a unified manner, which can greatly reduce the complexity of communication between committees.

[0067] Fastchain requires a routing solution that enables clients and committee leaders to quickly find the committee to which they should send transactions. There are two traditional solutions:

[0068] 1) Let each node store the network information of all committee members. In this way, each node can quickly locate the IP of any member and communicate. Obviously, this has poor scalability. Each node needs to connect to a large number of nodes, which is difficult to apply to a network of thousands of nodes.

[0069] 2) Another solution is to let the reference committee be responsible for transaction routing. Each user or committee leader obtains network information through the reference committee and routes transactions through the reference committee. However, this will cause the reference committee to be too heavy and need to handle a large amount of communication, which may cause a single shard to overheat.

[0070] Finally, the routing protocol of Kuailian borrowed the design idea of ​​Kademlia routing algorithm. In simple terms, each node stores the routing information of all nodes of the same committee members, and only stores the information of log log (n) nodes in the log (n) committees closest to its own committee. That is, each node selects log (n) committees closest to its own committee, and then selects loglog (n) nodes from each committee, and only stores the information of these nodes, rather than all nodes in the network.

[0071] More preferably, for messages between different committees, each sender only sends to the nodes of the target committee he knows, and the node that receives the message sends it to other members of the committee through the IDA-gossip protocol; for client users, they only need to submit the transaction to any committee, and the committee that receives the transaction will forward the transaction to the target committee through the Kademlia routing protocol.

[0072] When verifying new nodes, before the end of each epoch, the reference committee will generate an epoch randomness source for the next epoch. In each epoch, while the committee node is verifying transactions, other nodes that want to join the network are looking for PoW solutions locally, and this PoW solution must rely on the randomness source of the previous epoch. Requiring nodes to find PoW puzzles is to defend against Sybil attach, i.e. witch attacks. And the PoW solution must rely on epoch randomness to avoid the node's solution being calculated in advance.

[0073] Preferably, in S3, based on the cuckoo principle, the reference committee divides the committee into two categories, namely, an active committee with a majority of active members, and a passive committee with a majority of inactive members; when a new node joins, the reference committee randomly adds the node to any active committee, and randomly adds a fixed number of nodes in the committee to different passive committees to complete the reconstruction.

[0074] Preferably, in S3, based on the idea of ​​state sharding, the storage space is reduced to 1 / k of the original, where k is the number of shards, thereby significantly reducing the storage space consumption of each shard.

[0075] When the era reaches the committee reconstruction stage, all new nodes that wish to join the committee submit their solutions to the reference committee, which is responsible for verifying the correctness of the PoW solution. The leader of the reference committee will randomly assign new nodes to each committee according to the bounded cuckoo principle.

[0076] When generating a new era random source, during the reconstruction phase of each era, members of the reference committee will also run the DRG protocol to reach a consensus on an unbiased random value. This random source will then be included in a reference block and sent to every member of the network. Fastchain chooses to use Feldman's verifiable secret sharing protocol to generate this random source.

[0077] The steps to generate a random source are summarized as follows:

[0078] 1) Each reference committee node selects a prime number Pi

[0079] 2) Each node VSS-shares the number they choose to all other members of the committee.

[0080] 3) Each node j will receive m shares from other nodes, namely P1j, P2j, P3j, …Pmj.

[0081] The steps to generate a random source are summarized as follows:

[0082] Finally, the epochrandomness is obtained by summing the results and reconstructing them using Lagrange interpolation techniques.

[0083] The committee is reconstructed using the limited cuckoo principle that can resolve partition attacks. In a sharded blockchain system, shards are susceptible to partition attacks, that is, in a single shard, the number of offline or malicious nodes exceeds the tolerable ratio, or the protocol cannot proceed normally, and in serious cases, wrong information may be recorded in the shard ledger. Therefore, in order to avoid such partition attacks, each committee must be reshuffled regularly.

[0084] In the existing technology, the committee reconstruction method of Elastico is to disband all committee nodes and re-elect all committees. However, this method has two fatal shortcomings:

[0085] 1) Re-electing all committees is very expensive and takes a long time, resulting in the sharding system performance not being significantly improved.

[0086] 2) If the nodes are always shuffled, it is difficult to have an independent ledger for each committee position. Since the original intention of state sharding is to store the ledgers separately in the shards to reduce storage space. However, it is obvious that the method of shuffling and redistributing all the nodes cannot achieve the expected effect.

[0087] Therefore, in a state sharding system, we cannot disrupt all nodes in the system and reconfigure all committees. We can only fine-tune the committees. We need to find a protocol that can randomly add new nodes to the committee while also resisting partition attacks by adaptive opponents. Fastchain's solution is excellent. It makes changes based on the Cuckoo rule and creates a limited Cuckoo rule to solve the challenges mentioned above.

[0088] The principle of the cuckoo principle is very simple. First, each node is randomly placed in the interval [0,1], and then this interval is divided into k regions. A group of nodes in each region forms a committee. When a new node joins, it is randomly placed in a position, and a fixed number of nodes in the same region will be moved to other different random regions. Figure 6 As shown in the finite cuckoo principle, in each era, the reference committee divides the committee into two categories: active committees with a majority of active members and passive committees with a majority of inactive members. When a new node joins, the reference committee randomly adds the node to an active committee and randomly adds a fixed number of nodes in the committee to different passive committees, such as Figure 7 shown.

[0089] Fastchain uses the idea of ​​state sharding, so each node in the system only needs to store all the blocks and UTXO information of its own shard. In theory, it completes linear expansion of the storage space, reducing the storage space to 1 / k of the original, where k is the number of shards.

[0090] However, this ideal state cannot be achieved in reality, because from the cross-shard transaction characteristics mentioned above, we know that a cross-shard transaction may eventually be split into three or even more transactions stored in different shards, so it is believed that the expansion effect on storage space is not significant.

[0091] Of course, this is also due to the characteristics of the UTXO model. If the system uses an account-based model and users can only have accounts in one shard, the storage space consumption of each shard can be greatly reduced. Therefore, it is believed that the account-based model is more suitable for the state sharding system than the UTXO model.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. An asynchronous dynamic consensus method based on sharding, characterized in that: include: S1. Guide the election nodes to generate the shard committee and enter the first era; S2. Each committee forms a consensus, processes cross-shard transactions, and forms a routing protocol between different committees. S3, all committees are restructured to solve the node storage space problem; among them, In S1, the method for guiding the election node to generate a shard committee includes: S101. Each initial participating node creates a random bipartite graph G(L, R) locally using hard-coded parameters and random seeds, where L is a node and R is a group. The neighbors of each node in R are randomly, independently and uniformly selected from L, changing from level 0 to level 1 to form an initial group. S102. The initial group generates a random string s by running a distributed random number generation algorithm. Each node performs a hash operation on s and the ID of the subset. The nodes whose hash values ​​fall within a certain interval broadcast that the subset is the selected node. The other nodes in the group sign the identity ID and string s to prove that these nodes are the selected nodes. These selected nodes broadcast their identities, including the proof, to the entire network. S103, these selected nodes form a root group; S104. The root group is responsible for selecting members of the reference committee. The reference committee randomly divides the set of all nodes into various shard committees. In S2, the method for reaching consensus within each committee includes: S201. In each round of consensus formation, each committee selects a leader using the epoch random source; S202, the leader collects transactions from the client and packages them into block Bi; S203, the leader uses the IDA-Gossip protocol to broadcast the block Bi and creates a block header Hi containing the merkle tree and the round number; S204. Committee members initiate a consensus protocol for block header Hi; In S3, all committee reconstruction methods include: Based on the cuckoo principle, the reference committee divides the committee into two categories: active committees with a majority of active members, and passive committees with a majority of inactive members. When a new node joins, the reference committee randomly adds the node to any active committee and randomly adds a fixed number of nodes in the committee to different passive committees to complete the reconstruction.

2. According to claim 1, an asynchronous dynamic consensus method based on sharding is characterized in that: In S204, the leader adds a propose tag to the block header Hi and broadcasts Hi through the network; after each node receives the block header Hi, each node responds to the received block header Hi, adds a response tag to Hi, and broadcasts it through the network.

3. According to claim 2, an asynchronous dynamic consensus method based on sharding is characterized in that: If the node receives a different version of the block header Hi, it means that the leader is malicious, and the node rejects the block by broadcasting a pending tag; conversely, if the node only receives one block header Hi and receives more than mf+1 response tags, it means that all committees have reached a consensus on block Bi; once a consensus is reached among the committees, each node that receives at least mf+1 response tags broadcasts the block header again, and adds a receiving tag and a content that proves that the node was selected.

4. According to claim 1, an asynchronous dynamic consensus method based on sharding is characterized in that: For messages between different committees, each sender only sends to the nodes of the target committee he knows. The node that receives the message sends it to other members of the committee through the IDA-gossip protocol. For client users, they only need to submit the transaction to any committee, and the committee that receives the transaction will forward the transaction to the target committee through the Kademlia routing protocol.

5. According to claim 1, an asynchronous dynamic consensus method based on sharding is characterized in that: In the S3, based on the idea of ​​state sharding, the storage space is reduced to 1 / k of the original, where k is the number of shards, thereby greatly reducing the storage space consumption of each shard.

Images