An improved consensus method based on Raft and related equipment

Abstract

The present invention discloses a consensus method and related equipment based on Raft improvement, the method comprising: when the Raft consensus system is started, all nodes are slave nodes, and the master node election is automatically performed. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence; the number of slave nodes is calculated according to an adaptive strategy, and slave nodes are generated according to the node sorting sequence, and a network topology is generated according to a sharding strategy, and the network topology is broadcast and all nodes except the master node are sharded; the consensus service is executed, and if the master node fails, the slave node with the first order in the node sorting sequence is inherited as the master node through a crash recovery mechanism, so that the Raft consensus system can resume normal service. The present invention improves the availability of the consensus system through a pre-sorting method and a crash recovery mechanism, and improves the throughput performance and scalability of the consensus system through adaptive network sharding.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an improved consensus method, system, terminal and computer-readable storage medium based on Raft. Background Art

[0002] Blockchain technology is a decentralized architecture and computing paradigm that uses block chain data structures to verify and store data, distributed node consensus algorithms to generate and update data, cryptography to ensure the security of data transmission and access, and smart contracts composed of automated script codes to program and operate data.

[0003] Consensus: Multiple server nodes reach a consensus on a transaction, even in the event of partial node failure, network delay, or network segmentation.

[0004] RAFT: Raft is a consensus algorithm designed to replace Paxos. It makes it easier to understand through logical separation, but it is also formally proven to be secure and provides some additional features. Raft provides a general method for distributing state machines in a cluster of computing systems, ensuring that every node in the cluster agrees on the same series of state transitions. Consensus algorithms are the cornerstone of the development of modern distributed systems and blockchain technology. Discrete server nodes reach consistency in providing services through consensus.

[0005] Among them, the leader node provides services such as log read and write access to clients outside the consensus system, synchronizes log requests to followers internally, tells followers to submit logs when logs are synchronized to more than 1 / 2 of the nodes, and tries to keep the log status of all nodes consistent with the leader node. It is precisely because all content in the consensus system is consistent with the leader node that it will cause great problems when the leader node is lost. Even if the role of a secondary node is added, it only helps the master node share the network pressure of sending logs to the slave nodes, and the decision-making power of the log content still lies with the master node.

[0006] Among them, candidate node: When the system has no master node (or when initialized), the slave node will think that the master node is lost because it cannot receive the heartbeat message from the master node, and then enter the master node election. At this time, the slave node that times out first (thinks that the master node is lost) will become a candidate node and send a voting request to other slave nodes to perform the voting link in the Raft consensus algorithm. Because it does not involve content modification, this part is not mentioned in this document. Elections and voting are inherent business functions of the Raft consensus algorithm. A temporary role in the Leader election process.

[0007] Follower: Receives and persists the logs synchronized by the Leader. After the Leader tells it that the logs can be submitted, it submits the logs. It is the node that completes the consensus process. It only receives and responds to requests, and does not actively send them. (Raft consensus process: The master node sends logs to all slave nodes. The logs on the slave nodes should be as consistent as possible with the master node. This log replication process is the Raft consensus process, that is, how to make all nodes copy and save the same logs and reach the same state as much as possible, that is, to achieve consistency).

[0008] Vice-Leader: This is a node identity that is improved and added by the present invention, and is used to implement the network sharding of technical point 2. It is to adaptively select the first k slave nodes with strong communication capabilities or high hardware configuration strategies from the pre-sorted order, and then transform them into vice-leaders. Its function is to share the communication pressure of the master node. The master node sends logs to the vice-leader, and the vice-leader forwards them to the slave nodes corresponding to the shards.

[0009] Raft is a consensus mechanism that is widely used today. It uses a powerful leader node to make all nodes in the business reach a consensus. In a Raft instance with multiple member nodes, when the leader node crashes, the process of re-electing the leader node and the failure of the election will cause the business to be continuously unavailable. Secondly, with the increase in the number of nodes, since a single leader node becomes a communication bottleneck, the throughput of the entire business will be affected by the bandwidth of the leader node, which also restricts the scalability of the consensus system.

[0010] For the decline in system availability caused by the crash of the master node in the Raft cluster, the more common improvement technical solution is real-time monitoring, that is, by setting up an additional loop monitoring node to judge the availability of the consensus system in real time, and through a series of operations to designate a new master node and restore the cluster system to normal when the master node crashes.

[0011] For the master node throughput bottleneck caused by the increase in the number of nodes in Raft, the more common improvement technical solution is network sharding, that is, by grouping and combining the nodes in the original Raft instance, setting up multiple Raft instances, and limiting the number of nodes in each Raft instance, and using Raft or other consensus protocols to re-establish the master nodes of multiple consensus instances into a consensus cluster, so as to alleviate the master node throughput bottleneck problem caused by the increase in the number of nodes in Raft.

[0012] The real-time monitoring solution is implemented based on the method of adding additional monitoring nodes. The additional monitoring nodes are used to cyclically monitor all nodes in the Raft cluster to determine the availability of the consensus system in real time. Since the monitoring node needs to determine whether a node failure has occurred and whether the master node needs to be replaced based on the availability of the consensus system, it is necessary to continuously communicate with the system nodes for monitoring. However, in actual situations, the frequency of master node crashes is low, and the time of failure is unpredictable. If the monitoring node detects at a frequency with a timing interval that is too large, it may cause the system failure time to be longer; if the monitoring node monitors at a frequency with a timing interval that is too small, it will communicate with the network too frequently, causing additional network overhead; if the monitoring node uses a feedback adjustment mechanism, that is, it gets the message of the master node downtime from other nodes and re-assigns the master node after checking, this is compared with the situation where the master node election mechanism of the Raft consensus itself succeeds once. Since the Raft consensus algorithm provides a recovery mechanism when the master node crashes, it is no longer worthwhile to set up monitoring nodes when the system node failure rate is extremely low and the benefits are minimal.

[0013] The network sharding scheme is implemented based on the grouping method of system nodes. By changing the Raft network architecture, the Raft network is divided into one or more parts, so as to reduce the number of slave nodes that the master node is responsible for communicating, thereby reducing the bandwidth pressure of the master node network communication and improving the system throughput. A single simple network sharding is just a rough division of the network structure into two parts. In the other part, a vice-leader node (Vice-Leader Node) is specified to communicate with the master node, and then broadcast to the general network nodes that it is responsible for. Since the network shards only rely on the connection between the master node and the vice node, when the master node or the vice node crashes, some network nodes will be isolated, resulting in network isolation, which greatly reduces the system availability. The network sharding scheme using multiple Raft groups divides all nodes evenly into multiple groups, each of which is a Raft consensus instance. The groups rely on Raft consensus or routing broadcast methods to make the master nodes of all groups reach consensus. Since the multi-Raft grouping method abstracts the system network into two or more layers, its communication delay will increase. In addition, if a separate routing node is set up, the routing node may also become a system performance bottleneck. Multiple Raft groups require pre-configuration of the nodes, which is not conducive to the automatic and smooth expansion of system nodes.

[0014] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0015] The main purpose of the present invention is to provide a consensus method, system, terminal and computer-readable storage medium based on Raft improvement, aiming to solve the problem in the prior art that due to the increase in the number of nodes, a single master node becomes a communication bottleneck, the throughput of the entire business is affected by the master node bandwidth, and the scalability of the consensus system is also restricted.

[0016] To achieve the above object, the present invention provides a consensus method based on Raft improvement, and the consensus method based on Raft improvement comprises the following steps:

[0017] When the Raft consensus system is started, all nodes are slave nodes. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence.

[0018] After obtaining the node sorting sequence, the number of secondary nodes is calculated according to the adaptive strategy, secondary nodes are generated according to the node sorting sequence, a network topology is generated according to the sharding strategy, the network topology is broadcasted and all nodes except the primary node are sharded;

[0019] After the adaptive network sharding is completed, the consensus service of the Raft consensus system is executed. If the master node fails, the crash recovery mechanism is used to make the secondary node ranked first in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

[0020] Optionally, in the consensus method based on Raft improvement, the step of counting node information and processing it to obtain a node sorting sequence specifically includes:

[0021] The master node collects and counts relevant information of all slave nodes in the Raft consensus system, including log index, hardware performance and network status;

[0022] Determine whether the log index changes;

[0023] If the log index changes, the log index information of the slave node is updated according to the statistical results;

[0024] If the log index has not changed, determining whether the hardware performance has changed;

[0025] If the hardware performance changes, the hardware performance information of the slave node is updated according to the statistical results;

[0026] If the hardware performance has not changed, determining whether the network status has changed;

[0027] If the network status changes, the network status information of the slave node is updated according to the statistical results;

[0028] If the network state has not changed, the nodes are sorted according to the statistical information and the sorting strategy to obtain a node sorting sequence;

[0029] Update the node sorting sequence in the Raft consensus system.

[0030] Optionally, the consensus method based on Raft improvement, wherein the crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the primary node so that the Raft consensus system can resume normal service, specifically includes:

[0031] Determine whether the master node is lost;

[0032] If the primary node is lost, find the first secondary node according to the node sorting sequence;

[0033] Other secondary nodes in the node sorting sequence send a primary node loss message to the first secondary node in the succession order, so as to request the first secondary node to become the new primary node;

[0034] Determine whether the first secondary node has received more than 1 / 2 of the secondary node messages;

[0035] If the first secondary node receives more than 1 / 2 of the secondary node messages, a new primary node is generated, and the term number of the primary node is increased by one and the message is broadcast, so that the entire Raft consensus system enters the next term;

[0036] If the first secondary node does not receive more than 1 / 2 of the secondary node messages, it considers the first secondary node in the inheritance sequence to be unavailable, and searches for the next secondary node to try again;

[0037] If the primary node is not lost, determine whether the secondary node is lost;

[0038] If the secondary node is lost, the primary node searches for the first secondary node based on the node sorting sequence after sharding;

[0039] The master node sends a secondary node loss message to the node in the succession order, requesting the node in the succession order to become the new secondary node;

[0040] After a new secondary node is generated, it takes over the network sharding of the original secondary node.

[0041] Optionally, the consensus method based on Raft improvement, wherein the number of secondary nodes is calculated according to the adaptive strategy, the secondary nodes are generated according to the node sorting sequence, the network topology is generated according to the sharding strategy, the network topology is broadcasted and all nodes except the primary node are sharded, specifically includes:

[0042] The master node calculates the number of slave nodes required for network sharding according to the set adaptive strategy;

[0043] According to the sorting order of the node sorting sequence, the secondary node numbers are determined from the front to the back;

[0044] According to the sharding strategy, the remaining slave nodes are divided using the statistical node information, and the slave nodes are matched one by one according to the rules to generate the network topology structure;

[0045] The master node sends the network topology structure to the slave node through the heartbeat message. After receiving the heartbeat message, the slave node automatically adjusts the message sending and receiving object or transforms into a slave node.

[0046] Optionally, in the consensus method based on the improvement of Raft, the sharding strategy includes: selecting and dividing to generate a network topology structure according to node number hash mapping, network conditions and hardware configuration.

[0047] Optionally, in the consensus method based on Raft improvement, the secondary node message indicates that the primary node has lost the message.

[0048] Optionally, the consensus method based on Raft improvement further includes:

[0049] When an election is held, the candidate nodes that are not elected will become slave nodes after the master node is generated or enters the next term;

[0050] When an election is held, if the election timer times out and the election is still unsuccessful, the election will be re-held;

[0051] When the master node is changed, the remaining slave nodes are changed to slave nodes and wait for re-screening.

[0052] In addition, to achieve the above purpose, the present invention also provides a consensus system based on Raft improvement, wherein the consensus system based on Raft improvement includes:

[0053] The node pre-sorting module is used when all nodes are slave nodes when the Raft consensus system is started. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence.

[0054] An adaptive network sharding module is used to calculate the number of secondary nodes according to an adaptive strategy after obtaining the node sorting sequence, generate secondary nodes according to the node sorting sequence, generate a network topology structure according to the sharding strategy, broadcast the network topology structure and shard all nodes except the primary node;

[0055] The crash recovery module is used to execute the consensus business of the Raft consensus system after completing the adaptive network sharding. If the master node fails, the crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

[0056] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a Raft-improved consensus program stored in the memory and executable on the processor, wherein the Raft-improved consensus program implements the steps of the Raft-improved consensus method as described above when executed by the processor.

[0057] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a consensus program based on Raft improvement, and when the consensus program based on Raft improvement is executed by a processor, the steps of the consensus method based on Raft improvement as described above are implemented.

[0058] In the present invention, when the Raft consensus system is started, all nodes are slave nodes, and the Raft consensus system automatically elects a master node. If the master node is successfully elected, the node information is counted and processed to obtain a node sorting sequence; after obtaining the node sorting sequence, the number of slave nodes is calculated according to an adaptive strategy, and slave nodes are generated according to the node sorting sequence. A network topology is generated according to a sharding strategy, and the network topology is broadcast and all nodes except the master node are sharded; after completing the adaptive network sharding, the consensus service of the Raft consensus system is executed. If the master node fails, the crash recovery mechanism makes the slave node with the first order in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service. The present invention is improved based on the Raft consensus algorithm, improves the availability of the consensus system through a pre-sorting method and a crash recovery mechanism, improves the throughput performance and scalability of the consensus system through adaptive network sharding, and realizes a great improvement in the availability, throughput performance and scalability of the Raft consensus system. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0060] Figure 2 It is a flowchart of the entire implementation process of the consensus method improved based on Raft in a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0061] Figure 3It is a schematic diagram of node status in a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0062] Figure 4 It is a flowchart of a node pre-sorting method in a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0063] Figure 5 It is a flowchart of a crash recovery mechanism in a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0064] Figure 6 It is a schematic diagram of the adaptive network allocation process in a preferred embodiment of the consensus method improved based on Raft of the present invention;

[0065] Figure 7 It is a schematic diagram of the principle of a preferred embodiment of the consensus system improved based on Raft of the present invention;

[0066] Figure 8 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0068] The present invention is based on the improvement of the Raft consensus algorithm. The availability of the consensus system is improved through a pre-sorting method and a crash recovery mechanism. The throughput performance and scalability of the consensus system are improved through adaptive network sharding. The availability, throughput performance and scalability of the Raft consensus system are greatly improved. The technological breakthrough will contribute to the further development of the performance of traditional distributed systems and blockchain systems.

[0069] Aiming at improving the availability of Raft consensus, the present invention proposes a pre-sorting method, which utilizes information such as log synchronization progress, network conditions, and node configuration of all nodes to generate a node sorting order, that is, a crash inheritance order, and realizes automatic and rapid recovery of the master node in Raft when a crash error occurs through a crash recovery mechanism.

[0070] Aiming at improving the throughput and scalability of Raft consensus, the present invention proposes an adaptive network sharding method, which calculates the number of secondary nodes and generates secondary nodes according to an adaptive strategy, and generates a network topology according to the sharding strategy, thereby reducing the network communication pressure of the primary node and enabling the consensus system to automatically perform network sharding.

[0071] The consensus method based on Raft improvement described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the consensus method based on Raft improvement includes the following steps:

[0072] Step S10: When the Raft consensus system is started, all nodes are slave nodes. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence.

[0073] Step S20: after obtaining the node sorting sequence, the number of secondary nodes is calculated according to the adaptive strategy, secondary nodes are generated according to the node sorting sequence, a network topology is generated according to the sharding strategy, the network topology is broadcasted, and all nodes except the primary node are sharded;

[0074] Step S30: After the adaptive network sharding is completed, the consensus service of the Raft consensus system is executed. If the master node fails, the crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

[0075] Specifically, based on the original functions of the Raft consensus, the present invention adds two major innovations: a pre-sorting method and a crash recovery mechanism, as well as adaptive network sharding. The pre-sorting method and the crash recovery mechanism enable the consensus system to achieve the function of rapid recovery of the master node failure, thereby improving the system availability; the adaptive network sharding enables the consensus system to automatically perform network sharding on all nodes according to the adaptive strategy, thereby reducing the network communication pressure of the master node and improving the throughput and scalability of the consensus system. Figure 2 As shown in the figure, the detailed operation steps of the Raft improved consensus system are as follows:

[0076] start;

[0077] Step S1, master node election: when the Raft consensus system is started, all nodes are slave nodes, and the Raft consensus system automatically performs master node election;

[0078] Step S2, determine whether the master node is successfully elected; if it is, proceed to step S3 to perform node pre-sorting; if it is not, proceed to step S1 to re-elect the master node;

[0079] Step S3, node pre-sorting: if the master node election is successful, the nodes are pre-sorted, and the node pre-sorting is to obtain a node sorting sequence by counting and processing the node information;

[0080] Step S4, adaptive network sharding: calculating the number of secondary nodes according to the adaptive strategy, generating secondary nodes according to the node sorting sequence, generating a network topology structure according to the sharding strategy, broadcasting the network topology structure and sharding all nodes except the primary node;

[0081] Step S5, consensus business: execute other parts of the Raft consensus system until the Raft consensus system exits;

[0082] Step S6: Determine whether the master node fails; if it is, proceed to step S7 to execute the crash recovery mechanism; if it is not, proceed to step S5 to continue to execute the consensus service;

[0083] Step S7, crash recovery mechanism: If the master node fails, the Raft consensus system is restored to normal service through the crash recovery mechanism.

[0084] Finish.

[0085] The node status is as follows: Figure 3 As shown in FIG, a Raft consensus system is composed of many consensus nodes. The nodes in the Raft consensus system have three states: slave node, candidate node, and master node. In the present invention, a slave node is added to realize subsequent functions. Figure 3 As shown in the figure, the detailed changes of node status are as follows:

[0086] Scenario 1: The Raft consensus system is initialized and all nodes are slave nodes.

[0087] Scenario 2: The slave node does not receive a message from the master node due to timeout, and is converted to a candidate node and begins an election.

[0088] Scenario 3: The candidate node obtains more than half of the votes, becomes the primary node, and starts consensus operations.

[0089] Case 4: The master node finds a node with a higher term and becomes a slave node.

[0090] Scenario 5: When an election is held, the candidate nodes that are not elected will become slave nodes after the master node is generated or enters the next term.

[0091] Case 6: When an election is held, if the election timer times out and the election is not successful, the election is held again.

[0092] Scenario 7: When the primary node changes or the secondary node is lost, the node pre-sorting and adaptive network sharding process will be performed according to the predetermined strategy, and the secondary node will be selected from the slave node.

[0093] Scenario 8: When the primary node fails, the secondary node ranked first in the sorting order will inherit the primary node.

[0094] Scenario 9: When the master node is changed, the remaining slave nodes will be changed to slave nodes and wait for re-screening.

[0095] Specifically, the node pre-sorting method sorts all slave nodes by counting node information, evaluating the node's log index, hardware performance, network status or other custom policy conditions, and finally obtains an ordered node sequence. Figure 4 As shown, the detailed workflow of node pre-sorting is as follows:

[0096] start;

[0097] S101. Counting slave node information: The master node collects and counts relevant information of all slave nodes in the Raft consensus system, including log index, hardware performance, and network status;

[0098] S102, determine whether the log index has changed; if it is determined to be no, execute step S104; if it is determined to be yes, execute step S103 to update the node log index information;

[0099] S103, updating the log index information of the slave node: if the log index changes, the log index information of the slave node is updated according to the statistical result;

[0100] S104, if the log index has not changed, determine whether the hardware performance has changed; if it is determined to be no, execute step S106; if it is determined to be yes, execute step S105 to update the node hardware performance information;

[0101] S105, updating the hardware performance information of the slave node: if the hardware performance changes, the hardware performance information of the slave node is updated according to the statistical results;

[0102] S106, if the hardware performance has not changed, determine whether the network status has changed; if it is determined to be no, execute step S108; if it is determined to be yes, execute step S107 to update the node network status information;

[0103] S107, updating the network status information of the slave node: if the network status changes, updating the network status information of the slave node according to the statistical results;

[0104] S108, pre-sorting the slave nodes: if the network state has not changed, sorting the nodes according to the statistical information and the sorting strategy to obtain a node sorting sequence;

[0105] S109, updating the node sorting sequence in the Raft consensus system;

[0106] Finish.

[0107] Specifically, through the existing slave node sorting sequence, we can equivalently obtain the probability order of nodes that can win the election if the Raft consensus system is re-elected. Therefore, the crash recovery mechanism uses this order as the inheritance order when the master node fails, so as to quickly restore the function of the consensus system. Similarly, the node sorting sequence after adaptive network sharding is also effective, so the crash recovery mechanism can also handle the failure of the secondary node loss. Figure 5 As shown, the detailed workflow of the crash recovery mechanism is as follows:

[0108] start;

[0109] S301, determine whether the master node is lost; if it is, execute step S302; if it is not, execute step S307;

[0110] S302, searching for the first secondary node in the sequence: if the primary node is lost, searching for the first secondary node according to the node sorting sequence. In terms of sorting strategy, the first node in the node sorting sequence is the node with the best sorting result, and the others are sorted in order;

[0111] S303, other secondary nodes send a master node loss message to them: other secondary nodes in the node sorting sequence send a master node loss message to the first secondary node in the inheritance order, so as to request the first secondary node to become the new master node;

[0112] S304, determining whether the first secondary node has received more than 1 / 2 of the secondary node messages; if determined to be yes, executing step S305; if determined to be no, executing step S306;

[0113] S305. Generate a new master node and enter the next term: If the first slave node receives more than 1 / 2 of the slave node messages, a new master node is generated, and the term number of the master node is increased by one and the message is broadcast, so that the entire Raft consensus system enters the next term;

[0114] S306. If the first secondary node does not receive more than 1 / 2 of the secondary node messages, it is considered that the first secondary node in the inheritance sequence is unavailable, and the next secondary node is searched for and tried again. That is, when the first secondary node in the node sorting sequence does not receive the primary node loss message equal to or greater than 1 / 2 of the secondary nodes within the specified time, it is considered that the first secondary node in the inheritance sequence is unavailable, so the next secondary node is searched for and tried again.

[0115] S307, if the primary node is not lost, determine whether the secondary node is lost; if the determination is yes, execute step S308; if the determination is no, execute step S311;

[0116] S308, search for the first slave node in the shard: if the secondary node is lost, the master node searches for the first slave node according to the node sorting sequence after sharding. As far as the sharding strategy is concerned, the node sorting sequence after sharding is still valid, and the first node is the node with the best sorting result, and the others are sorted in order;

[0117] S309, the master node sends a secondary node loss message to the node: the master node sends a secondary node loss message to the node in the succession order, requesting the node in the succession order to become a new secondary node;

[0118] S310, generating a new secondary node and replacing the original secondary node: after the new secondary node is generated, it takes over the network sharding of the original secondary node;

[0119] S311, slave node loss processing service: when a slave node is lost, the slave node loss processing service in the Raft consensus system is executed;

[0120] Finish.

[0121] Specifically, through the node pre-sorting method, a set of node sorting sequences according to certain priority rules can be obtained. Here, it is considered that the node sorting sequence meets the priority conditions of the current business logic. Adaptive network sharding generates a secondary node sequence based on the existing node sorting sequence combined with the adaptive strategy calculation, and then generates a topological network structure according to the sharding strategy, and completes network sharding through broadcasting. Figure 6 As shown in the figure, the detailed workflow of adaptive network slicing is as follows:

[0122] start;

[0123] S201, calculating the number of secondary nodes according to the adaptive strategy: the master node calculates the number of secondary nodes required for network sharding according to the set adaptive strategy;

[0124] S202, generating secondary nodes according to the pre-sorted order of nodes: determining secondary node numbers from front to back according to the sorted order of the node sorting sequence;

[0125] S203, generating a network topology structure according to a sharding strategy: according to the sharding strategy, using the statistical node information, the remaining slave nodes are divided, and the slave nodes are matched one by one according to the rules to generate a network topology structure; wherein the sharding strategy is not unique, and the network topology structure can be generated by selecting and dividing according to the node number hash mapping, network conditions, and hardware configuration;

[0126] S204, sharding the nodes: the master node sends the network topology structure to the slave node via a heartbeat message, and the slave node automatically adjusts the message receiving and sending object or transforms into a slave node after receiving the message.

[0127] Finish.

[0128] (1) The present invention provides a node pre-ordering and crash inheritance mechanism, which greatly improves the availability of the Raft consensus system when a crash error occurs without adding too much system overhead. There is no need to consider the specific consensus scheme and form. The present invention can be adapted to other consensus algorithms and Raft improvement technologies and methods, such as multi-Raft grouping.

[0129] (2) The present invention provides an adaptive network sharding solution, which determines the number of secondary nodes through an adaptive algorithm, and then performs network sharding based on conditional information such as numbering hash, network conditions, and geographic location, thereby alleviating the network communication pressure brought by the increase in the number of nodes on the main node and improving system throughput and scalability.

[0130] The evaluation strategies used in the above two technologies can be reorganized or changed according to specific usage and needs, and their process steps can also be customized and combined.

[0131] The improved consensus method based on Raft proposed in this invention can provide the following advantages compared with the general Raft consensus system:

[0132] (1) Compared with the real-time monitoring solution, the inheritance sequence when the crash error occurs is obtained through the pre-sorting method. When a crash error occurs, the master node can be quickly restored and the log security can be guaranteed as much as possible, thereby improving the system availability.

[0133] (2) Compared with the network sharding solution, adaptive network sharding eliminates the need to pre-configure the consensus system. The number of secondary nodes can be adaptively determined and the sharding of network nodes can be automatically completed, so that a good balance between system latency and throughput is achieved, thereby improving system performance and scalability.

[0134] In summary, the present invention greatly improves the availability and throughput of the consensus system based on the existing technology. In addition, the present invention has been subjected to multiple simulation tests, and the experiments have proved that the present invention can meet the expected design indicators.

[0135] Furthermore, if Figure 7 As shown, based on the above consensus method based on Raft improvement, the present invention also provides a consensus system based on Raft improvement, wherein the consensus system based on Raft improvement includes:

[0136] The node pre-sorting module 51 is used for when all nodes are slave nodes when the Raft consensus system is started, the Raft consensus system automatically elects a master node, and if the master node election is successful, the node information is counted and processed to obtain a node sorting sequence;

[0137] The adaptive network sharding module 52 is used to calculate the number of secondary nodes according to the adaptive strategy after obtaining the node sorting sequence, generate secondary nodes according to the node sorting sequence, generate a network topology structure according to the sharding strategy, broadcast the network topology structure and shard all nodes except the primary node;

[0138] The crash recovery module 53 is used to execute the consensus business of the Raft consensus system after completing the adaptive network sharding. If the master node fails, the crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

[0139] Furthermore, if Figure 8 As shown, based on the above-mentioned consensus method and system based on Raft improvement, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 8 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0140] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a consensus program 40 based on Raft improvement is stored on the memory 20, and the consensus program 40 based on Raft improvement can be executed by the processor 10, thereby realizing the consensus method based on Raft improvement in this application.

[0141] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the improved consensus method based on Raft.

[0142] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0143] In one embodiment, when the processor 10 executes the consensus program 40 based on Raft improvement in the memory 20, the steps of the consensus method based on Raft improvement as described above are implemented.

[0144] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a consensus program based on Raft improvement, and when the consensus program based on Raft improvement is executed by a processor, the steps of the consensus method based on Raft improvement are implemented as follows:

[0145] When the Raft consensus system is started, all nodes are slave nodes. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence.

[0146] After obtaining the node sorting sequence, the number of secondary nodes is calculated according to the adaptive strategy, secondary nodes are generated according to the node sorting sequence, a network topology is generated according to the sharding strategy, the network topology is broadcasted and all nodes except the primary node are sharded;

[0147] After the adaptive network sharding is completed, the consensus service of the Raft consensus system is executed. If the master node fails, the crash recovery mechanism is used to make the secondary node ranked first in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

[0148] The step of collecting and processing the node information to obtain a node sorting sequence specifically includes:

[0149] The master node collects and counts relevant information of all slave nodes in the Raft consensus system, including log index, hardware performance and network status;

[0150] Determine whether the log index changes;

[0151] If the log index changes, the log index information of the slave node is updated according to the statistical results;

[0152] If the log index has not changed, determining whether the hardware performance has changed;

[0153] If the hardware performance changes, the hardware performance information of the slave node is updated according to the statistical results;

[0154] If the hardware performance has not changed, determining whether the network status has changed;

[0155] If the network status changes, the network status information of the slave node is updated according to the statistical results;

[0156] If the network state has not changed, the nodes are sorted according to the statistical information and the sorting strategy to obtain a node sorting sequence;

[0157] Update the node sorting sequence in the Raft consensus system.

[0158] The crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the primary node, so that the Raft consensus system can resume normal service, specifically including:

[0159] Determine whether the master node is lost;

[0160] If the primary node is lost, find the first secondary node according to the node sorting sequence;

[0161] Other secondary nodes in the node sorting sequence send a primary node loss message to the first secondary node in the succession order, so as to request the first secondary node to become the new primary node;

[0162] Determine whether the first secondary node has received more than 1 / 2 of the secondary node messages;

[0163] If the first secondary node receives more than 1 / 2 of the secondary node messages, a new primary node is generated, and the term number of the primary node is increased by one and the message is broadcast, so that the entire Raft consensus system enters the next term;

[0164] If the first secondary node does not receive more than 1 / 2 of the secondary node messages, it considers the first secondary node in the inheritance sequence to be unavailable, and searches for the next secondary node to try again;

[0165] If the primary node is not lost, determine whether the secondary node is lost;

[0166] If the secondary node is lost, the primary node searches for the first secondary node based on the node sorting sequence after sharding;

[0167] The master node sends a secondary node loss message to the node in the succession order, requesting the node in the succession order to become the new secondary node;

[0168] After a new secondary node is generated, it takes over the network sharding of the original secondary node.

[0169] The method of calculating the number of secondary nodes according to the adaptive strategy, generating secondary nodes according to the node sorting sequence, generating a network topology structure according to the sharding strategy, broadcasting the network topology structure and sharding all nodes except the primary node specifically includes:

[0170] The master node calculates the number of slave nodes required for network sharding according to the set adaptive strategy;

[0171] According to the sorting order of the node sorting sequence, the secondary node numbers are determined from the front to the back;

[0172] According to the sharding strategy, the remaining slave nodes are divided using the statistical node information, and the slave nodes are matched one by one according to the rules to generate the network topology structure;

[0173] The master node sends the network topology structure to the slave node through the heartbeat message. After receiving the heartbeat message, the slave node automatically adjusts the message sending and receiving object or transforms into a slave node.

[0174] The sharding strategy includes: selecting and dividing to generate a network topology structure according to node number hash mapping, network conditions and hardware configuration.

[0175] The secondary node message indicates that the primary node has lost the message.

[0176] The consensus method based on Raft improvement also includes:

[0177] When an election is held, the candidate nodes that are not elected will become slave nodes after the master node is generated or enters the next term;

[0178] When an election is held, if the election timer times out and the election is still unsuccessful, the election will be re-held;

[0179] When the master node is changed, the remaining slave nodes are changed to slave nodes and wait for re-screening.

[0180] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a consensus program based on Raft improvement, and when the consensus program based on Raft improvement is executed by a processor, the steps of the consensus method based on Raft improvement as described above are implemented.

[0181] In summary, the present invention provides a consensus method and related equipment based on Raft improvement, the method comprising: when the Raft consensus system is started, all nodes are slave nodes, the Raft consensus system automatically performs master node election, if the master node election is successful, the node information is counted and processed to obtain a node sorting sequence; after obtaining the node sorting sequence, the number of secondary nodes is calculated according to an adaptive strategy, secondary nodes are generated according to the node sorting sequence, a network topology is generated according to a sharding strategy, the network topology is broadcasted and all nodes except the master node are sharded; after completing the adaptive network sharding, the consensus service of the Raft consensus system is executed, if the master node fails, the secondary node with the first order in the node sorting sequence is inherited as the master node through a crash recovery mechanism, so that the Raft consensus system can resume normal service. The present invention is improved based on the Raft consensus algorithm, improves the availability of the consensus system through a pre-sorting method and a crash recovery mechanism, improves the throughput performance and scalability of the consensus system through adaptive network sharding, and realizes a great improvement in the availability, throughput performance and scalability of the Raft consensus system.

[0182] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0183] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0184] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A consensus method based on Raft improvement, characterized in that: The consensus method based on Raft improvement includes: When the Raft consensus system is started, all nodes are slave nodes. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence. After obtaining the node sorting sequence, the number of secondary nodes is calculated according to the adaptive strategy, secondary nodes are generated according to the node sorting sequence, a network topology is generated according to the sharding strategy, the network topology is broadcasted and all nodes except the primary node are sharded; After the adaptive network sharding is completed, the consensus service of the Raft consensus system is executed. If the master node fails, the crash recovery mechanism is used to make the secondary node ranked first in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service; The crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the primary node, so that the Raft consensus system can resume normal service, specifically including: Determine whether the master node is lost; If the primary node is lost, find the first secondary node according to the node sorting sequence; Other secondary nodes in the node sorting sequence send a primary node loss message to the first secondary node in the succession order, so as to request the first secondary node to become the new primary node; Determine whether the first secondary node has received more than 1 / 2 of the secondary node messages; If the first secondary node receives more than 1 / 2 of the secondary node messages, a new primary node is generated, and the term number of the primary node is increased by one and the message is broadcast, so that the entire Raft consensus system enters the next term; If the first secondary node does not receive more than 1 / 2 of the secondary node messages, it considers the first secondary node in the inheritance sequence to be unavailable, and searches for the next secondary node to try again; If the primary node is not lost, determine whether the secondary node is lost; If the secondary node is lost, the primary node searches for the first secondary node based on the node sorting sequence after sharding; The master node sends a secondary node loss message to the node in the succession order, requesting the node in the succession order to become the new secondary node; After a new secondary node is generated, it takes over the network sharding of the original secondary node.

2. The consensus method based on Raft improvement according to claim 1 is characterized in that: The counting of node information and processing to obtain a node sorting sequence specifically includes: The master node collects and counts relevant information of all slave nodes in the Raft consensus system, including log index, hardware performance and network status; Determine whether the log index changes; If the log index changes, the log index information of the slave node is updated according to the statistical results; If the log index has not changed, determining whether the hardware performance has changed; If the hardware performance changes, the hardware performance information of the slave node is updated according to the statistical results; If the hardware performance has not changed, determining whether the network status has changed; If the network status changes, the network status information of the slave node is updated according to the statistical results; If the network state has not changed, the nodes are sorted according to the statistical information and the sorting strategy to obtain a node sorting sequence; Update the node sorting sequence in the Raft consensus system.

3. The consensus method based on Raft improvement according to claim 1 is characterized in that: The method of calculating the number of secondary nodes according to the adaptive strategy, generating secondary nodes according to the node sorting sequence, generating a network topology structure according to the sharding strategy, broadcasting the network topology structure and sharding all nodes except the primary node specifically includes: The master node calculates the number of slave nodes required for network sharding according to the set adaptive strategy; According to the sorting order of the node sorting sequence, the secondary node numbers are determined from the front to the back; According to the sharding strategy, the remaining slave nodes are divided using the statistical node information, and the slave nodes are matched one by one according to the rules to generate the network topology structure; The master node sends the network topology structure to the slave node through the heartbeat message. After receiving the heartbeat message, the slave node automatically adjusts the message sending and receiving object or transforms into a slave node.

4. The consensus method based on Raft improvement according to claim 3 is characterized in that: The sharding strategy includes: selecting and dividing to generate a network topology structure according to node number hash mapping, network conditions and hardware configuration.

5. The consensus method based on Raft improvement according to claim 1 is characterized in that: The secondary node message indicates that the primary node has lost the message.

6. The consensus method based on Raft improvement according to claim 1 is characterized in that: The consensus method based on Raft improvement also includes: When an election is held, the candidate nodes that are not elected will become slave nodes after the master node is generated or enters the next term; When an election is held, if the election timer times out and the election is still unsuccessful, the election will be re-held; When the master node is changed, the remaining slave nodes are changed to slave nodes and wait for re-screening.

7. A consensus system based on Raft improvement, wherein the consensus system based on Raft improvement is applied to the consensus method based on Raft improvement according to any one of claims 1 to 6, characterized in that: The consensus system based on Raft improvement includes: The node pre-sorting module is used when all nodes are slave nodes when the Raft consensus system is started. The Raft consensus system automatically elects a master node. If the master node election is successful, the node information is counted and processed to obtain a node sorting sequence. An adaptive network sharding module is used to calculate the number of secondary nodes according to an adaptive strategy after obtaining the node sorting sequence, generate secondary nodes according to the node sorting sequence, generate a network topology structure according to the sharding strategy, broadcast the network topology structure and shard all nodes except the primary node; The crash recovery module is used to execute the consensus business of the Raft consensus system after completing the adaptive network sharding. If the master node fails, the crash recovery mechanism is used to make the secondary node with the first order in the node sorting sequence inherit as the master node, so that the Raft consensus system can resume normal service.

8. A terminal, characterized in that: The terminal includes: a memory, a processor, and a Raft-based improved consensus program stored in the memory and executable on the processor. When the Raft-based improved consensus program is executed by the processor, the steps of the Raft-based improved consensus method as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a consensus program based on Raft improvement, and when the consensus program based on Raft improvement is executed by a processor, the steps of the consensus method based on Raft improvement are implemented as described in any one of claims 1-6.

Images