A blockchain dynamic sharding expansion method based on graph flow segmentation

Abstract

The present invention discloses a blockchain dynamic sharding expansion method based on graph flow segmentation, wherein the dynamic sharding expansion means that the blockchain dynamically updates the sharding state according to the current network status; the graph flow segmentation is based on the input and output parties of each transaction as the two end nodes of an edge, and a graph data stream formed by all generated transactions, and according to the graph data stream segmentation algorithm, the entire transaction state graph, i.e., the blockchain node network, is segmented to achieve the purpose of sharding; the method is performed according to the epoch (generation) cycle, and each epoch cycle includes the transaction packaging and state update process. After one epoch, the sharding state will change according to the transaction graph, so as to solve the problems of low efficiency, great security challenges, and low cross-shard processing efficiency in the existing blockchain solution; the present invention improves the blockchain business throughput, reduces the cross-chain ratio, and load balances as much as possible under the premise of ensuring security.

Description

Technical Field

[0001] The present invention belongs to the field of blockchain technology and distributed storage, and more specifically refers to a blockchain dynamic sharding expansion method based on graph stream segmentation. Background Art

[0002] Blockchain technology is based on a peer-to-peer network, but its weak scalability seriously affects the performance improvement of blockchain. Sharding technology is a practical blockchain horizontal expansion solution. By dividing the blockchain network into multiple shards, different shards can process transactions in parallel, which can improve the overall transaction throughput and processing rate of the system; Byzantine consensus usually requires multiple rounds of message propagation and feedback, and its complexity increases exponentially with the number of nodes in the network. The number of nodes in each shard is less than the overall number, which can greatly improve the consensus speed.

[0003] However, the number of shards is not necessarily the more the better. When the probability of malicious nodes is equal, the number of nodes in each shard and security are inversely exponential. In addition, in actual systems, nodes are not stable and unchanging. When nodes are dynamically entering and exiting, the state of the shard cannot always remain unchanged. The change of the shard state is accompanied by the replication and transfer of data, and we hope that nodes that frequently trade with each other can be in the same shard to achieve the effect of reducing cross-shard transactions and load balancing.

[0004] At present, the mainstream processing of cross-shard transactions in blockchain sharding usually includes active client maintenance, trace-based annotation, transaction splitting, etc. The most typical one is the transaction splitting solution represented by Ethereum, which divides the transfer process of a transaction into two processes, sending and receiving, and completes them in different consensus cycles. However, this approach destroys the atomicity of the transaction and is not yet mature. Summary of the invention

[0005] The present invention provides a blockchain dynamic sharding expansion method based on graph flow segmentation to solve the problems of low efficiency, great security challenges, multiple cross-shard transactions and low cross-shard processing efficiency in existing blockchain solutions; the present invention improves the blockchain business throughput, reduces the cross-chain ratio, and balances the load as much as possible while ensuring security.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A method for dynamic sharding expansion of blockchain based on graph flow segmentation, wherein the dynamic sharding expansion means that the blockchain dynamically updates the sharding status according to the current network status; the graph flow segmentation is based on the input and output parties of each transaction as the two end nodes of an edge, and a graph data stream formed by all generated transactions. According to the graph data stream segmentation algorithm, the entire transaction status graph, i.e., the blockchain node network, is segmented to achieve the purpose of sharding; the method is performed according to the epoch (generation) cycle, and each epoch cycle includes the transaction packaging and status update process. After an epoch, the sharding status will change according to the transaction graph to achieve the role of load balancing and reducing security risks. Specifically, the method includes the following steps:

[0008] Step 1: The node calculates the target Nonce value that meets the current Target by running the hash function, and performs a modulo operation based on the current number of shards. The result is the serial number of the shard to which the node belongs. The node needs to initiate an admission transaction. After the transaction is successfully packaged into a block, the node can initiate a formal transaction in the next epoch.

[0009] Step 2: During the transaction process, each shard packages the transaction data in the transaction pool of the shard and generates a block. The first transaction block of each epoch is preceded by the status block of the previous epoch. The block structure of the entire system is a topological graph structure, and each epoch converges to a status block.

[0010] Step 3: Execute the graph flow segmentation algorithm

[0011] Each node continuously reads transaction information from the newly generated block and updates the state graph of all current nodes. When the current epoch ends, the state graph will also be determined. The transaction information is used as the input of the graph flow segmentation algorithm. The transaction information includes the state graph and the transaction account.

[0012] Step 4: After the graph state and account are split in step 3, the blockchain system reaches a consensus on the split result. The consensus in this stage adopts the PBFT algorithm and generates a state block. This state block serves as the convergence block of all current shards, indicating the end of the current epoch.

[0013] Step 5: After the status block of the current epoch reaches consensus, the associated shards are changed after the status transfer according to the block content to start a new round of transactions.

[0014] Preferably, the nodes include ordinary nodes and committee nodes, the ordinary nodes are nodes used to initiate transactions, and the committee nodes form a committee, which are used to process cross-article transactions and calculations of shard state reconstruction respectively.

[0015] Preferably, in the step 2, the ordering of blocks in the topological graph structure adopts the PBFT consensus algorithm, and the number of blocks generated in one epoch depends on the current network status.

[0016] Preferably, in step 2, for cross-shard transactions, the atomicity of cross-shard transactions is ensured by combining transaction locking and double confirmation mechanisms through the committee node as an intermediary.

[0017] Preferably, the committee nodes are formed by staking a specified number of tokens to obtain the qualification for election, and the remaining nodes can vote to elect the committee, and each shard contains at least one committee member.

[0018] Preferably, in step 2, the basis for executing the graph flow segmentation algorithm is: load balancing between shards, security of the blockchain system, and communication cost of replication between shards.

[0019] As a preferred method, the HDRF-P graph flow segmentation algorithm is used to determine which subgraph (sub-shard) the associated edge is assigned to based on the degree information of the node. The optimization process of segmentation is to find the minimum cut of the transaction graph. The minimum cut can reduce the communication cost between nodes and balance the load of multiple nodes in the distributed system to avoid overheating caused by excessive load on a few nodes and shards. In the process of optimizing the optimal solution based on the transaction flow, the boundary conditions, that is, security guarantees, must also be considered. The number of each shard cannot be lower than the threshold, otherwise the shard will be manipulated and security problems will arise.

[0020] Preferably, the HDRF-P algorithm is based on the HDRF image flow segmentation algorithm.

[0021] The input v1 and v2N of the algorithm represent two points on an edge and the total number of nodes respectively. Since the HDRF algorithm needs to give the number of divisions, the security guarantee is based on the number of nodes and the PBFT algorithm:

[0022] n=3f+1

[0023] Where n represents the total number of nodes, and f represents the number of Byzantine nodes;

[0024] The number of partitions k can be obtained, where N represents the total number of nodes and μ represents the proportion of malicious nodes:

[0025]

[0026] When assigning an edge, an objective function value is calculated for each subgraph, and the subgraph with the highest value is selected for assignment. Its objective function consists of two parts:

[0027] Part I

[0028]

[0029] is the penalty term for the size of the subgraph, and its weight is controlled by a coefficient λ;

[0030] Part 2

[0031]

[0032] It represents the inclusion of two points v1 and v2 in the subgraph. If it does not include any of them, it is 0. If it includes one of them, it is in the interval (1, 2). If both are included, it is the maximum value 3. When only one of them is included, according to the calculation method g(v1, i), the point with lower inclusion degree will have a higher value, so as to achieve the priority segmentation of points with high degree.

[0033] In addition, an additional constraint is added to the selection of the segmentation to which the edge belongs:

[0034]

[0035] The present invention has the following characteristics and beneficial effects:

[0036] With the above technical solution, dynamic sharding expansion means that the sharding state will be dynamically adjusted according to the transaction density, total number of nodes and load balancing strategy of the current system; parallel consensus is achieved through sharding to improve the system's transaction processing peak performance; dynamic sharding state adjustment can ensure that the security of the blockchain remains within an allowable range, while pursuing a balance between the replication cost and load balancing between shards; cross-shard transactions refer to different shards to which the accounts involved in a transaction in the blockchain system belong. When dealing with such cross-shard issues, the committee is used as an intermediary, and combined with account locking and other forms, the space-for-time approach is used to improve efficiency; the sharding of the system is implemented through a graph data stream segmentation algorithm, and each transaction is input into the graph stream segmentation algorithm as an edge of the global transaction graph to achieve the conversion of the sharding state, which can effectively improve the performance, throughput and security of the entire system. In addition, it is carried out according to the epoch cycle, and each epoch cycle includes the transaction packaging and status update process. After an epoch, the sharding state will change according to the transaction graph to achieve the role of load balancing and enhanced security. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 12 is a global architecture diagram of an embodiment of the present invention.

[0039] Figure 2 4 is a flow chart of a graph segmentation algorithm used in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0042] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0043] The present invention provides a blockchain dynamic sharding expansion method based on a graph flow segmentation algorithm, such as Figure 1 As shown in the figure, the system is divided into ordinary nodes and committee nodes. Ordinary nodes are nodes that initiate transactions, and committee nodes form committees. The committees have two functions, namely, processing cross-section transactions and calculations for shard state reconstruction. The system takes an epoch as a cycle, and each cycle will dynamically change the shard state; in each epoch, it includes committee elections, transaction packaging (including cross-shard transactions), graph segmentation, and shard state transitions.

[0044] In the committee election, nodes that want to run for the committee can obtain the right to be elected by staking a certain amount of tokens. The other ordinary nodes select committee members through elections. The committee requires that each shard contains at least one committee member.

[0045] The transaction packaging is the process in which miners in the shard sort and block the transactions in the transaction pool of the current epoch. The consensus is based on the PBFT algorithm. For cross-shard transactions in the transaction, a proof is generated and locked by the committee node, and then the proof is sent to another shard to realize step-by-step cross-shard.

[0046] In the process of transaction packaging in all shards, the generated confirmed transactions are used as edges of the transaction graph to generate graph stream input. According to the graph stream segmentation algorithm: the load of the shard with overheated transactions is balanced to the shard with lower heat. According to the principle of locality, transactions are kept within the shard as much as possible to reduce cross-shard transactions. The number of nodes in each shard cannot be lower than the security threshold. After segmentation, the state block of the next epoch is generated and broadcast to all shards.

[0047] The state transition is to update the state according to the new state diagram to obtain the next round of sharding state and committee.

[0048] The specific implementation steps are as follows:

[0049] Step 1: The node gets a valid Nonce by running the POW algorithm, and adds the result as a transaction to the transaction pool. The access information is packaged when the block is generated in this epoch, and it takes effect in the next epoch. The shard to which the node belongs is obtained by taking the Nonce result modulo the current number of shards. Only then can the node conduct transactions normally.

[0050] Step 2: Nodes that want to run for committee membership can obtain the qualification to be elected by staking a specified number of tokens. The remaining nodes can vote to elect the committee. Each shard contains at least one committee member.

[0051] Step 3: Each shard has an independent transaction pool, and blocks are generated in units of shards. Specifically, miners package and sort transactions, and then run the PBFT consensus algorithm to reach a consensus on the sorting results. The first transaction block of each epoch is preceded by the status block, which not only maintains the linear structure of the blockchain, but also increases the cost of tampering.

[0052] For cross-shard transactions, specifically, taking the transaction of transferring money from an account in the first shard to an account in the second shard as an example, it is divided into the following three steps:

[0053] The transaction output party initiates the transaction. If it is a cross-shard transaction, it will be routed to the committee node in the current shard.

[0054] The transaction output committee node generates a signature to prove that there is indeed available money in its account and locks the money;

[0055] The committee node of the transaction output party routes the proof to the committee node of the transaction input party to verify the signature. If the verification is successful, the transaction is completed and the cross-shard transaction is written into the transaction pool of the input party and the output party respectively. If the verification fails, it means that the balance of the transaction output party is insufficient, then the transaction fails, and the failure information is sent back to the committee node of the transaction output party to perform the unlocking operation.

[0056] Step 4: After the transaction is completed, all transactions are taken as edges of the graph and accounts as points to form a transaction graph. Here we need to split the entire transaction graph into multiple shards. Using the HDRF-P graph flow segmentation algorithm, we can determine which subgraph (subshard) the associated edges are assigned to based on the degree information of the nodes. The optimization process of segmentation is to find the minimum cut of the transaction graph. The minimum cut can reduce the communication cost between nodes and balance the load of multiple nodes in the distributed system, avoiding overheating caused by excessive load on a few nodes and shards. In the process of optimizing the optimal solution based on the transaction flow, we must also consider the boundary conditions, that is, security guarantees. The number of each shard cannot be lower than the threshold, otherwise the manipulation of the shards will cause security problems.

[0057] The complete HDRF-P algorithm, such as Figure 2 As shown in the figure, the algorithm is improved based on the HDRF graph stream segmentation algorithm. The HDRF algorithm is mainly used in the fields of big data and distributed storage.

[0058] The input v1 and v2N of the algorithm represent two points on an edge and the total number of nodes respectively. Since the HDRF algorithm needs to give the number of divisions, the security guarantee is based on the number of nodes and the PBFT algorithm:

[0059] n=3f+1

[0060] Where n represents the total number of nodes, and f represents the number of Byzantine nodes;

[0061] The number of partitions k can be obtained, where N represents the total number of nodes and μ represents the proportion of malicious nodes:

[0062]

[0063] When assigning an edge, an objective function value is calculated for each subgraph, and the subgraph with the highest value is selected for assignment. Its objective function consists of two parts:

[0064] Part I

[0065]

[0066] is the penalty term for the size of the subgraph, and its weight is controlled by a coefficient λ;

[0067] Part 2

[0068]

[0069] It represents the inclusion of two points v1 and v2 in the subgraph. If it does not include any of them, it is 0. If it includes one of them, it is in the interval (1, 2). If both are included, it is the maximum value 3. When only one of them is included, according to the calculation method g(v1, i), the point with lower inclusion degree will have a higher value, so as to achieve the priority segmentation of points with high degree.

[0070] In addition, an additional constraint is added to the selection of the segmentation to which the edge belongs:

[0071]

[0072] Under this constraint, security is prioritized when selecting the split, and the weight of the edge is considered secondly. This limits each split to not be infinitely large or too small, and is always within the security range of the PBFT algorithm.

[0073] Step 5: After the network is sharded, the system needs to reach a consensus on the sharding results. The system runs the PBFT consensus algorithm, uses the sharding results as the consensus content, and generates a status block, which is broadcast to all shards. This block is both the convergence block of the current epoch transaction block and the pre-block of the next epoch. The block structure is in a dynamic expansion and contraction state.

[0074] Step 6: After receiving the broadcast shards, the node changes its state, that is, the shards are broken up and then reassembled, and the above steps are repeated.

[0075] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A blockchain dynamic sharding expansion method based on graph flow segmentation, characterized in that: The steps include: Step 1: The node calculates the target Nonce value that meets the current Target by running the hash function, and performs a modulo operation based on the current number of shards. The result is the serial number of the shard to which the node belongs. The node needs to initiate an admission transaction. After the transaction is successfully packaged into a block, the node can initiate a formal transaction in the next epoch. Step 2: During the transaction process, each shard packages the transaction data in the transaction pool of the shard and generates a block. The first transaction block of each epoch is preceded by the status block of the previous epoch. The block structure of the entire system is a topological graph structure, and each epoch converges to a status block. Step 3: Execute the graph flow segmentation algorithm Each node continuously reads transaction information from the newly generated block and updates the state graph of all current nodes. When the current epoch ends, the state graph will also be determined. The transaction information is used as the input of the graph flow segmentation algorithm. The transaction information includes the state graph and the transaction account. The HDRF-P graph flow segmentation algorithm is used to determine which subgraph the associated edges are assigned to based on the degree information of the node. The segmentation optimization process is to find the minimum cut of the transaction graph. The minimum cut can reduce the communication cost between nodes and balance the load of multiple nodes in the distributed system to avoid overheating caused by excessive load on a few nodes and shards. In the process of optimizing the optimal solution based on the transaction flow, boundary conditions must also be considered, that is, security guarantees. The number of each shard cannot be lower than the threshold, otherwise the shards will be manipulated and security problems will arise; The HDRF-P image flow segmentation algorithm is based on the HDRF image flow segmentation algorithm The input v1 and v2 of the algorithm represent two points on an edge respectively, and N represents the total number of nodes. Since the HDRF graph flow segmentation algorithm needs to give the number of divisions, the security guarantee is based on the number of nodes and the PBFT algorithm: N=3f+1 Where N represents the total number of nodes, and f represents the number of Byzantine nodes; The number of partitions k can be obtained, where N represents the total number of nodes and μ represents the proportion of malicious nodes: When assigning an edge, an objective function value is calculated for each subgraph, and the subgraph with the highest value is selected for assignment. Its objective function consists of two parts: Part I is the penalty term for the size of the subgraph, and its weight is controlled by a coefficient λ; Part 2 It represents the inclusion of two points v1 and v2 in the subgraph. If it does not include any of them, it is 0. If it includes one of them, it is in the interval (1, 2). If both are included, it is the maximum value 3. When only one of them is included, according to the calculation method g(v1, i), the point with lower inclusion degree will have a higher value, so as to achieve the priority segmentation of points with higher degree. In addition, an additional constraint is added to the selection of the split to which the edge belongs: Step 4: After the graph state and account are split in step 3, the blockchain system reaches a consensus on the split result. The consensus in this stage adopts the PBFT algorithm and generates a state block. This state block serves as the convergence block of all current shards, indicating the end of the current epoch. Step 5: After the status block of the current epoch reaches consensus, the associated shards are changed after the status transfer according to the block content to start a new round of transactions.

2. The blockchain dynamic sharding expansion method based on graph flow segmentation according to claim 1 is characterized in that: The nodes include ordinary nodes and committee nodes. The ordinary nodes are nodes used to initiate transactions, and the committee nodes form committees, which are used to process cross-shard transactions and shard state reconstruction calculations respectively.

3. According to the blockchain dynamic sharding expansion method based on graph flow segmentation according to claim 1, it is characterized in that: In the step 2, the ordering of blocks in the topological graph structure adopts the PBFT consensus algorithm, and the number of blocks generated in one epoch depends on the current network status.

4. The blockchain dynamic sharding expansion method based on graph flow segmentation according to claim 2 is characterized in that: In step 2, for cross-shard transactions, the atomicity of cross-shard transactions is ensured by combining transaction locking and double confirmation mechanisms through the committee node as an intermediary.

5. The blockchain dynamic sharding expansion method based on graph flow segmentation according to claim 4 is characterized in that: The committee nodes are formed by staking a specified number of tokens to obtain the qualification for election. The remaining nodes can vote to elect the committee. Each shard contains at least one committee member.

6. The blockchain dynamic sharding expansion method based on graph flow segmentation according to claim 1 is characterized in that: In step 2, the basis for executing the graph flow segmentation algorithm is: load balancing between shards, security of the blockchain system, and communication cost of replication between shards.

Images