Network topology consensus implementation method based on Merkel tree and diff transmission

Abstract

The invention discloses a network topology consensus implementation method based on a Merkel tree and diff transmission, a Hash Merkel tree data structure is designed and is used for storing node information of the whole network by each consensus node in the network, data transmitted by each round of consensus is only changed data nodes in the Merkel tree and Hash values related to the changed data nodes, namely, diff transmission, and the data transmitted by each round of consensus are only changed data nodes in the Merkel tree and Hash values related to the changed data nodes in the Merkel tree. And the whole Merkel tree does not need to be transmitted, so that the communication cost is reduced, and the communication security is also improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed data communication, and more specifically, to a method for implementing network topology consensus based on Merkle tree and diff transfer. Background Art

[0002] With the continuous development of Internet technology, in order for nodes in the network to master the topology information of the network, topology information communication is required between nodes in the network. There are many ways of network topology information communication, and the common ones are as follows.

[0003] Specific protocol data units (PDUs) such as ICMP and ARP are used. Among them, ICMP uses error reporting units to transfer network errors, and ARP uses ARP request and response data units to resolve IP addresses; network devices can use broadcast and multicast methods to send topology information to all devices in the network. For example, the DHCP server uses the broadcast method to send IP address information to clients, and routing protocols use the multicast method to send routing update information to routers in the network. Well-known routing protocols include OSPF and RIP; some protocols use neighbor discovery processes to exchange network topology information. For example, IPv6 uses the Neighbor Discovery Protocol (NDP) to discover neighbor nodes and routing information; the Border Gateway Protocol (BGP) obtains routing information through the exchange of protocol messages; some network scanning tools obtain topology information by scanning devices in the network and recording their connection relationships and configuration information.

[0004] When the topology information in the network is transmitted, the amount of transmission is often large and the security is not high. Common topology interaction methods have deficiencies in terms of stability and reliability, and topology data may be tampered with during the transmission process, which is dangerous in networks with security requirements. The network topology consensus algorithm can avoid these deficiencies. Network topology consensus enables each node to reach a consensus on the network topology structure based on information exchange and coordination with each other in a distributed network. This consensus is very important for the stability and reliability of the network because only when all nodes have a common understanding of the network topology can they correctly forward data packets and ensure network connectivity. Summary of the Invention

[0005] In view of this, the present invention provides a method for implementing network topology consensus based on Merkle tree and diff transmission. For the consensus communication between network nodes, a hash Merkle tree data structure is designed. The bottom-level leaf nodes are used to store the information of all network nodes. The content of each round of network consensus is the information of all network nodes. Since most of the node information in the whole network does not change, only a few nodes have dynamic changes, so the data transmitted in each round of consensus is only the changed data nodes and their related hash values in the Merkle tree, that is, diff transmission, and there is no need to transmit the entire Merkle tree, which not only reduces the communication cost but also improves the network security.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: A method for implementing network topology consensus based on Merkle tree and diff transmission, comprising the following specific steps: Step 1: In the Merkle tree, score the stability of the nodes in the network. The more stable network nodes are stored in the leaves closer to the left, and the more unstable network nodes are stored in the leaves closer to the right. Step 2: The Merkle tree leaves store the network information of one or more network nodes, the adjacency relationship with other nodes, and network interface information, etc. Step 3: The number of leaf nodes can be appropriately selected according to the network scale to avoid the Merkle tree having too large a depth due to too many leaf nodes. Step 4: After each round of consensus in the network, each node in the network maintains a copy of the same Merkle tree. Step 5: When the information of any number of network nodes changes, change the content of the corresponding leaf nodes of the Merkle tree, calculate the Hash values and Merkle Root of the corresponding intermediate layers of the Merkle tree, and assemble and broadcast diff messages. Step 6: The diff message content includes the content of the corresponding leaf nodes of the changed Merkle tree and their position information, as well as the updated Merkle Root. Step 7: After receiving the diff information, other nodes will modify the content of the corresponding leaf nodes of their own Merkle tree according to the Merkle tree of the previous round of consensus, and update and calculate the Hash values and Merkle Root of the corresponding intermediate layers. If the Merkle Root calculated by itself is the same as the Merkle Root in the received diff, it proves that the content of the leaf nodes in the received diff is correct.

[0007] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method for implementing network topology consensus based on Merkle tree and diff transmission. For the consensus communication between network nodes, a hash Merkle tree data structure is designed. The leaf nodes at the bottom layer are used to store the node information of the entire network. The content of each round of network consensus is the information of all network nodes. Since most of the node information in the entire network does not change, only a few nodes have dynamic changes, so the data transmitted in each round of consensus is only the changed data nodes in the Merkle tree and the related hash values, that is, diff transmission, and there is no need to transmit the entire Merkle tree, which not only reduces the communication cost but also improves the network security. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0009] Figure 1 The attached drawing is a schematic diagram of the Merkle tree data structure designed by the present invention; Figure 2 The attached drawing is a schematic diagram of the Merkle tree data structure of the embodiment designed by the present invention; MODE OF IMPLEMENTATION

[0010] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0011] The embodiments of the present invention disclose a method for implementing network topology consensus based on Merkle tree and diff transmission. For the consensus communication between network nodes, a hash Merkle tree data structure is designed. The leaf nodes at the bottom layer are used to store the node information of the entire network. The content of each round of network consensus is the information of all network nodes. Since most of the node information in the entire network does not change, only a few nodes have dynamic addition and departure, so the data transmitted in each round of consensus is only the changed data nodes in the Merkle tree and the related hash values, that is, diff transmission, and there is no need to transmit the entire Merkle tree, which not only reduces the communication cost but also improves the security of consensus communication.

[0012] Similar to a hash list, a Merkle tree divides data into small data chunks, each corresponding to a hash. However, as we move up the tree, instead of directly calculating the root hash, we combine two adjacent hashes into a single string and then calculate the hash of this string. In this way, we obtain a "sub-hash" for every two hashes. If the total number of hashes at the bottom layer is odd, there will inevitably be a single hash left at the end. In this case, we simply calculate the hash of this single hash, so we can also obtain its sub-hash. Then, as we continue to move up, we use the same method to get a new set of hashes with a smaller number. Eventually, it will definitely form an upside-down tree. At the root of the tree, there is only one root hash left in this generation, which we call the MerkleRoot.

[0013] In a distributed system, consistency is generally achieved through the principle of a state replication machine. The core idea is that all replicas in the system run the same state machine. As long as all replicas start with the same initial state and execute a set of operations in the same order based on the same initial state, then all states will eventually converge to be the same, that is, the entire system will exhibit consistency externally.

[0014] In a general network, each node conducts consensus communication, and the consensus content includes the state information of all nodes in the network, adjacency relationships, etc. Most of the node information does not change, and only about 10% of the nodes change frequently. Therefore, 90% of the content in each round of consensus is the same as that in the previous round. If the voting and consensus process transmits the complete Merkle tree content, it will waste communication costs and result in low efficiency of the consensus mechanism. Therefore, this solution designs a diff transmission design based on the Merkle tree, enabling each round of voting to only transmit the content that is different from the previous round of consensus, that is, the diff, thereby reducing network communication costs and improving the consensus efficiency.

[0015] The schematic diagram of the Merkle tree storage in this embodiment is as Figure 2 shown. A hash Merkle tree with 8 leaf nodes is designed according to the network node scale of the hundred-level. The part of the entire Merkle tree that occupies the most storage is the leaf nodes, that is, the data nodes that actually store the node information. Figure 2 The leaf nodes of the Merkle tree in

[0016] In each round of consensus, it should be assumed that before a new round of consensus, the state of each node is consistent, that is, everyone's Merkle tree is consistent. When entering a new round of consensus, the slave node will send the change information it has mastered to the master node. Through the broadcast mechanism, the master node collects the diff uploaded by all slave nodes and updates the Merkle tree. When choosing which leaf node in the Merkle tree to store the information of a node, the master node will score the node according to its stability in the past consensus rounds. The node with a larger score proves that it is more stable, so it is stored on the left, and so on. This can minimize the changes in the Merkle tree in each round of consensus.

[0017] When the master node collects all the diff summaries, it will form the latest Merkle tree, such as Figure 2 As shown in the figure, when the nodes in Data7 and Data8 change, Hash4, Hash6, and Merkle Root also change accordingly. Obviously, if the entire Merkle tree is broadcasted to each node as a consensus message at this time, it will cause a waste of communication costs, so in fact, the master node broadcasts the changed Data7, Data8 and their location information, as well as the updated MerkleRoot. After receiving these diff information, the slave node modifies the Data7 and Data8 nodes according to the Merkle tree of the previous round of consensus, and updates the calculation of Hash4. The new Hash6 is obtained by hashing Hash3 and Hash4; the final Merkle Root is obtained by hashing the new Hash6 and Hash5. If the Merkle Root calculated by itself is the same as the Merkle Root transmitted by the master node, this proves that there is no problem with Data7 and Data8, and the slave node recognizes the consensus result this time.

[0018] In this embodiment, the leaf nodes at the bottom level need to contain hundreds of levels of network node information. The leaf nodes have different storage spaces. The leaf nodes closer to the left have larger spaces and can store record information of multiple nodes. Because most of the nodes stored on the left are stable nodes and are not easy to change, these data nodes will not be modified most of the time, thereby reducing the height of the Merkle tree. The data nodes on the right have small storage spaces and store less node information. The multiple data nodes on the far right only store one piece of network node information. Because they store node information that is easy to change and update in the network, these data nodes are often broadcast in the consensus, and they occupy less storage space, which can reduce communication costs.

[0019] like Figure 2As shown, the Data1 and Data2 nodes of this Merkle tree can store the information of 30 network nodes, and the Data3 and Data4 nodes can store the information of 15 network nodes. In this way, the height of the tree can be reduced, thereby reducing the storage cost and calculation cost. The Data6 - Data8 nodes can only store the information of one node to deal with some nodes with frequent changes, preventing the situation where the data volume of a single Data node to be propagated is too large due to the information change of individual network nodes.

[0020] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. The present invention discloses a method for implementing network topology consensus based on Merkle tree and diff transmission, characterized in that It includes the following specific steps: Step 1: In the Merkle tree, score the stability of the nodes in the network. The network nodes stored in the leaves closer to the left are more stable, and the less stable network nodes are stored in the leaves closer to the right; Step 2: The leaves of the Merkle tree store network information of one or more network nodes, adjacency relationships with other nodes, network interfaces and other information; Step 3: The number of leaf nodes can be appropriately selected according to the network scale to avoid the Merkle tree having too large a depth due to too many leaf nodes; Step 4: After each round of consensus in the network, each node in the network maintains an identical Merkle tree; Step 5: When the information of any number of network nodes changes, change the content of the corresponding leaf nodes of the Merkle tree, calculate the Hash values and Merkle Root of the corresponding intermediate layers of the Merkle tree, and assemble and broadcast diff messages; Step 6: The content of the diff message includes the content of the corresponding leaf nodes of the changed Merkle tree and their location information, as well as the updated Merkle Root; Step 7: After other nodes receive the diff information, they will modify the content of the corresponding leaf nodes of their own Merkle tree according to the Merkle tree of the previous round of consensus, and update and calculate the Hash values and Merkle Root of the corresponding intermediate layers. If the Merkle Root calculated by themselves is the same as the Merkle Root in the received diff, it proves that the content of the leaf nodes in the received diff is correct.

Images