A method for generating a distributed hash table based on the Raft consensus algorithm

Abstract

The present invention relates to the field of computer technology, and specifically relates to a method for generating a distributed hash table based on the Raft consensus algorithm, including registering a distributed hash table identifier to obtain a new distributed hash table identifier; directing an old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier; and solving malicious users based on the directed distributed hash table identifier. The present invention provides an improved method for the existing Kademlia DHT algorithm to enhance the long-term stability of the DHT overlay network. By running a third-party auxiliary entity of the Raft algorithm, outside the DHT network, a Raft consensus third-party entity is independently run, which can make up for data consistency, improve the persistent storage of data in the DHT network, and further improve the long-term stability of the DHT network, thus solving the problem of poor stability of the existing distributed hash table generation method.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method for generating a distributed hash table based on the Raft consensus algorithm. Background Art

[0002] Kademlia DHT is a distributed hash table algorithm that is widely used in P2P file transfer systems and distributed content routing systems. This algorithm adopts the idea of content routing, where the stored content itself represents routing information, facilitating users to find the data resources they want in a large-scale distributed network environment.

[0003] The traditional Kademlia DHT algorithm operates in the following manner: The Peer nodes participating in the DHT overlay network randomly generate distributed hash table identifiers, connect to the DHT overlay network through the Bootstrap node, and establish a routing table by continuously querying their own distributed hash table identifiers. In such an algorithm process, the following disadvantages exist:

[0004] There is no restriction on the generation of distributed hash table identifiers. Any node can use a binary number of any preset length as its own distributed hash table identifier, which enables malicious attackers to impersonate others' distributed hash table identifiers or deceive other Peers into believing that they have certain resources.

[0005] The traditional DHT algorithm implementation will regularly replace the new distributed hash table identifier to avoid such problems. However, regularly generating new distributed hash table identifiers will invalidate the routing table on the DHT overlay network, and Peers need to re-establish the routing table.

[0006] Temporary offline and IP changes will cause changes in the distributed hash table identifier. Due to the existence of NAT, the IP of the user may be different each time they connect to the DHT overlay network; and with the development of today's mobile networks, users often switch between different IPs, which will also cause the routing table of the DHT overlay network to become invalid.

[0007] The Bootstrap node may be malicious or suffer from a single point of failure. Users must join the DHT network through a Bootstrap node, but there are not many stable and well-known Bootstrap nodes. Moreover, the network may be partitioned at any time, preventing users from joining the DHT overlay network. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for generating a distributed hash table based on the Raft consensus algorithm, aiming to solve the problem of poor stability of the existing distributed hash table generation method.

[0009] To achieve the above object, the present invention provides a method for generating a distributed hash table based on the Raft consensus algorithm, including the following steps:

[0010] Register the distributed hash table identifier to obtain a new distributed hash table identifier;

[0011] Direct the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier;

[0012] Solve malicious users based on the directed distributed hash table identifier.

[0013] Among them, the registering the distributed hash table identifier to obtain a new distributed hash table identifier includes:

[0014] The participant generates a distributed hash table identifier of a preset length and submits it to the network layer, queries it in the server cluster, and obtains a query result;

[0015] Based on the first query result, if the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain a new distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the corresponding relationship between the participant and the distributed hash table identifier, and obtains a new distributed hash table identifier.

[0016] Among them, the directing the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier includes:

[0017] The participant generates a distributed hash table identifier of a preset length and submits it to the network layer, queries it in the server cluster, and obtains a second query result;

[0018] Based on the second query result, if the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain a directed distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the corresponding relationship between the participant and the distributed hash table identifier, and obtains a directed distributed hash table identifier;

[0019] Direct the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier.

[0020] Among them, the solving malicious users based on the directed distributed hash table identifier includes:

[0021] Identify malicious users who misappropriate the distributed hash table identifier.

[0022] The server cluster uses the term of the algorithm based on the directed distributed hash table identifier to solve malicious users.

[0023] Among them, the algorithm is a consensus algorithm.

[0024] A method for generating a distributed hash table based on the Raft consensus algorithm of the present invention registers the distributed hash table identifier to obtain a new distributed hash table identifier; directs the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier; and solves malicious users based on the directed distributed hash table identifier. The present invention provides an improved method for the existing Kademlia DHT algorithm to enhance the long-term stability of the DHT overlay network. By running a third-party auxiliary entity of the Raft algorithm outside the DHT network and independently running a Raft consensus third-party entity, it can make up for data consistency, improve the persistent storage of data in the DHT network, and thus improve the long-term stability of the DHT network, thereby solving the problem of poor stability of the existing distributed hash table generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 It is the overall structure diagram provided by the present invention.

[0027] Figure 2 It is the flowchart of DHT ID registration.

[0028] Figure 3 It is the flowchart of DHT ID change.

[0029] Figure 4 It is the flowchart of DHT ID conflict.

[0030] Figure 5 It is the flowchart of a method for generating a distributed hash table based on the Raft consensus algorithm provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following details the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present invention and should not be construed as a limitation of the present invention.

[0032] Please refer to Figures 1 to 5 , the present invention provides a method for generating a distributed hash table based on the Raft consensus algorithm, including the following steps:

[0033] S1 Register the distributed hash table identifier to obtain a new distributed hash table identifier;

[0034] S11 The participant generates a distributed hash table identifier of a preset length and submits it to the network layer for query in the server cluster to obtain a first query result;

[0035] Specifically, register the DHT ID: After the participant generates a DHT ID of a preset length, it is submitted to the network layer. The network layer, as a service proxy, queries the DHT ID in ETCD.

[0036] S12 Based on the first query result, if the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain a new distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the corresponding relationship between the participant and the distributed hash table identifier to obtain a new distributed hash table identifier.

[0037] Specifically, if the ID already exists, notify the participant to regenerate a new DHT ID; if the DHT ID does not exist, the Raft algorithm performs a round of state machine update to store the corresponding relationship between the participant and the DHT ID. Through this step, the participant declares itself as the owner of the DHT ID.

[0038] S2 Orient the old distributed hash table identifier to the new distributed hash table identifier to obtain an oriented distributed hash table identifier;

[0039] S21 The participant generates a distributed hash table identifier of a preset length and submits it to the network layer for query in the server cluster to obtain a second query result;

[0040] Specifically, register the DHT ID: After the participant generates a legal-length DHT ID, it is submitted to the network layer. The network layer, as a service proxy, queries the DHT ID in ETCD.

[0041] S22 Based on the second query result, if the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain an oriented distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the corresponding relationship between the participant and the distributed hash table identifier to obtain an oriented distributed hash table identifier;

[0042] Specifically, if the ID already exists, the participant is notified to regenerate a new DHT ID; if the DHT ID does not exist, the Raft algorithm performs a round of state machine updates to store the correspondence between the participant and the DHT ID. Through this step, the participant declares itself as the owner of the DHT ID.

[0043] S23 redirects the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier.

[0044] Specifically, due to situations such as IP changes or network fluctuations, the participant needs to replace its DHT ID. It can register the new ID in ETCD and redirect the old ID.

[0045] S3 resolves malicious users based on the directed distributed hash table identifier.

[0046] S31 identifies malicious users who misappropriate the distributed hash table identifier.

[0047] Specifically, it identifies malicious users who misappropriate the DHT ID.

[0048] S32 The server cluster resolves malicious users based on the directed distributed hash table identifier using the term of the algorithm.

[0049] Specifically, ETCD will save the user information of the earliest declarer of the DHT ID. Therefore, conflicts of DHT IDs can be resolved according to the term of the Raft algorithm. Only the user who registers the ID earliest is the legitimate user. As the Bootstrap node of the DHT overlay network, it provides initial routing information to new participants. New Peers participating in the DHT network can entrust ETCD as a learner of the Raft algorithm to obtain a certain amount of DHT routing information from any participating role in the Raft configuration as Bootstrap information to establish an initial routing table.

[0050] The data table design of ETCD and DHT network participants. ETCD is a distributed K-V database that adopts the Raft consensus algorithm. In the example of the present invention, using ETCD as an external trusted third-party entity for the Kademlia DHT algorithm will not affect the operation of the existing DHT network. The Peer nodes of the DHT network belong to the learner identity in the ETCD system and can only send query requests and write requests to the ETCD system; they do not participate in the operation of the Raft consensus protocol and the verification of data validity.

[0051] The content routing table of the Peer node is shown in Table 1:

[0052] CID1 DHT ID1; DHT ID2 CID2 DHT ID1; DHT ID3 CID3 DHT ID4

[0053] Table 1: Content Routing Table, Network Connection Routing Table of Peer Nodes:

[0054] DHT ID1 IP1 DHT ID2 IP2 DHT ID3 IP1

[0055] Table 2: Network Connection Routing Table

[0056] CID is the content ID, which is calculated based on the file content. The Peer node queries the content routing table through the CID to obtain the corresponding DHT ID, and then the Peer node can know which peer to request data from. Through the network routing table, the specific network connection address of the data request object can be obtained. After the Peer node queries the DHT ID, it can synchronously query ETCD. The query result of ETCD complements the query result of its own routing table, enhancing the availability of the DHT network. If there are discrepancies between the two results, there are several possible reasons: ① The IP address of the corresponding Peer has changed. ② Someone is impersonating the corresponding Peer, and false data has been saved to its own routing table.

[0057] Example data in ETCD:

[0058]

[0059] Table 3: ETCD Database Table

[0060] IP is the IP corresponding to the DHT ID, Term indicates in which term of the Raft consensus this record was added to ETCD. Commitment is the result of encrypting the IP with the Peer's private key H(IP). The verifier of ETCD decrypts it with the Peer's public key. If the obtained result is the same as the IP, the verification passes. If the Peer user needs to change information such as its DHT ID and IP later, it is also verified in the same way, which can make the DHT network routing information more stable.

[0061] The above-disclosed is only a preferred embodiment of a method for generating a distributed hash table based on the Raft consensus algorithm of the present invention. Of course, it cannot be used to limit the scope of the rights of the present invention. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A method for generating a distributed hash table based on the Raft consensus algorithm, characterized in that Including the following steps: Register the distributed hash table identifier to obtain a new distributed hash table identifier, which includes: the participant generates a distributed hash table identifier of a preset length and submits it to the network layer, and queries in the server cluster to obtain a first query result; Judge based on the first query result. If the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain a new distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the correspondence between the participant and the distributed hash table identifier to obtain a new distributed hash table identifier; Direct the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier, which includes: the participant generates a distributed hash table identifier of a preset length and submits it to the network layer, and queries in the server cluster to obtain a second query result; Judge based on the second query result. If the distributed hash table identifier already exists, notify the participant to regenerate a new distributed hash table identifier to obtain a directed distributed hash table identifier; if the distributed hash table identifier does not exist, the Raft algorithm performs a round of state machine update to store the correspondence between the participant and the distributed hash table identifier to obtain a directed distributed hash table identifier; Direct the old distributed hash table identifier to the new distributed hash table identifier to obtain a directed distributed hash table identifier; Solve malicious users based on the directed distributed hash table identifier.

2. The method for generating a distributed hash table based on the Raft consensus algorithm according to claim 1, wherein The solving of malicious users based on the directed distributed hash table identifier includes: Identifying malicious users who misappropriate the distributed hash table identifier; The server cluster uses the term of the algorithm based on the directed distributed hash table identifier to solve malicious users.

3. The method for generating a distributed hash table based on the Raft consensus algorithm according to claim 2, wherein The algorithm is a consensus algorithm.

Images