A Raft distributed system and Raft consensus method

Abstract

The present invention proposes a Raft distributed system and a Raft consensus method, which relate to the field of distributed storage and are used to solve the computing and storage overhead problems caused by storing data on each node in the prior art. The system includes a master node, a slave node, a candidate node, and a backup node. The master node executes the client's data request and sends the data request log to the slave node and the backup node. The slave node receives the master node's data request log and executes the data request associated with the data request log. The candidate node initiates the master node election voting process in the intermediate state during the master node election process. The backup node has no state machine and does not store a complete copy of the data. A set number of slave nodes are randomly selected and configured as backup nodes. The present invention utilizes the fault tolerance capability of the Raft algorithm, introduces a new backup node role, and eliminates redundant copies of data on the backup node by optimizing the Raft consensus method. While ensuring consistency, the system can greatly reduce storage and computing overhead.

Description

Technical Field

[0001] The present invention belongs to the field of distributed storage, and in particular relates to a Raft distributed system and a Raft consensus method. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of the Internet of Things, e-commerce, and social networks, global big data reserves have grown rapidly, becoming the foundation for the development of the big data industry. In 2017, global big data reserves were 21.6ZB, and in 2018, global big data reserves reached 33ZB, a growth rate of 52.8%. With the continuous advancement of information technology, the China Business Industry Research Institute predicts that global big data reserves will reach 61.2ZB in 2022. Faced with massive amounts of data, the processing capacity of a single node cannot meet the growing computing and storage tasks, and distributed systems can only be adopted.

[0004] A distributed system is composed of a group of computer nodes that communicate through a network and work in coordination to complete a common task. It can use cheap, ordinary machines to complete computing and storage tasks that a single computer cannot complete. To improve availability, distributed systems usually introduce replicas, which brings about data consistency issues. To maintain data consistency between nodes, a consistency algorithm must be used.

[0005] Consensus algorithms allow multiple machines to work together as a cluster. Even if some machines fail, the cluster can still function normally, ensuring data consistency across multiple nodes. Among consistency algorithms, the Raft algorithm is widely used in distributed systems due to its easy understandability and engineering implementation. For example, ETCD, TIKV, etc. all use the Raft algorithm to solve consistency problems in distributed systems.

[0006] The Raft algorithm primarily includes a leader election module and a log synchronization module. At any given moment, a node can only be one of three roles: leader, follower, or candidate. When the cluster starts, all nodes are slaves. After waiting for a random heartbeat timeout (Election Timeout), the slave increases its current term by one, becoming a candidate node. It then initiates an election, votes for itself, and sends a vote request to all other nodes. If a candidate receives votes from more than half of the nodes (including itself), it becomes the leader. The candidate node that initiates the election first is generally most likely to become the leader. The leader needs to periodically send heartbeats to other nodes to maintain its status and prevent unnecessary elections. The Raft algorithm can tolerate non-Byzantine node failures. For a cluster with N = 2F + 1 nodes, as long as no more than F nodes experience failures, the cluster can function normally.

[0007] In Raft, all read and write requests from clients are processed by the master node. The master node first serializes the request into a Raft log and then synchronizes it to all other nodes. Log synchronization requires three steps: writing the log (Append), committing the log (Commit), and applying the log (Apply). The maximum index of the committed log is called CommitIndex. Figure 1 Taking a 3-node Raft cluster as an example, the typical approach is as follows:

[0008] 1) The client initiates a data write request to the master node a.

[0009] 2) The master node a serializes the request into a Raft log and appends it to the local log.

[0010] 3) Send the log to Follower b and Follower c via RPC.

[0011] 4) Follower b and Follower c nodes receive the log and append it to their local logs.

[0012] 5) Follower b and Follower c reply to the master node a that the node log has been successfully received.

[0013] 6) After half of the nodes successfully reply to the master node a, the master node a commits the log.

[0014] 7) The master node a applies the log to the state machine.

[0015] 8) The master node a responds to the client.

[0016] 9) The master node a broadcasts the latest CommitIndex.

[0017] 10) Follower b and Follower c update CommitIndex and commit the log.

[0018] 11) Follower b and Follower c apply the log to the state machine.

[0019] There is currently a lot of work being done to improve the performance of the Raft algorithm. In engineering, batching and pipeline methods are used to send batches of logs instead of individual logs at a time, and logs can be sent continuously without waiting for responses from followers, thereby improving system throughput. In algorithms, to reduce the time consumption of the Raft consensus method, some work has improved performance by breaking some of Raft's limitations, such as Commit Return and Immediate Read. To reduce the network cost of log transmission, Craft uses erasure codes to shard the log. The master node sends the corresponding shards instead of the entire log to the follower, reducing network overhead but sacrificing read performance.

[0020] The existing technology has at least the following deficiencies:

[0021] In the existing Raft consensus method, client requests to the master node are serialized into a log by the master node, which is then synchronized to all slave nodes and ultimately applied to the state machine. Assuming a cluster has N = 2F + 1 nodes, to ensure availability, a copy of the data must be stored on each node, for a total of 2F + 1 copies. If F + 1 nodes in the cluster fail, even if the remaining F nodes have complete copies of the data, the cluster will be unable to provide external services. As long as at least F + 1 nodes are functioning properly, the cluster can provide external services. Therefore, existing technologies store data on each node, resulting in at least F redundant copies, which leads to unnecessary computational and storage overhead. Summary of the Invention

[0022] To overcome the above-mentioned deficiencies in the prior art, the present invention provides a Raft distributed system and a Raft consensus method. By utilizing the fault-tolerance capability of the Raft algorithm, a new backup node Recorder role is introduced. By optimizing the Raft consensus method, redundant copies of data on the backup nodes are eliminated, which can greatly reduce storage and computing overhead while ensuring consistency.

[0023] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0024] A first aspect of the present invention provides a Raft distributed system;

[0025] A Raft distributed system includes a plurality of nodes, wherein the nodes are divided into a master node, a slave node, a candidate node, and a backup node;

[0026] The master node is configured to: execute the data request of the client and send the data request log to the slave node and the backup node;

[0027] The slave node is configured to: receive the data request log of the master node and execute the data request associated with the data request log;

[0028] The candidate node is configured to: initiate the election voting process for the master node during the intermediate state of the master node election process;

[0029] The backup node is configured as follows: it has no state machine and does not save a complete copy of the data. After the system generates a master node, it randomly selects a set number of slave nodes to be configured as backup nodes.

[0030] Furthermore, the randomly selected set number of slave nodes are configured as backup nodes, specifically:

[0031] The newly generated master node detects and configures the backup nodes in the cluster;

[0032] Assume that the number of cluster nodes is N=2F+1, where F is a non-negative integer. F nodes are randomly selected from the slave nodes and converted into backup nodes.

[0033] A second aspect of the present invention provides a Raft consensus method.

[0034] A Raft consensus method, based on a Raft distributed system provided in the first aspect, includes:

[0035] When the client sends a data request, the Raft log synchronization process is performed;

[0036] When the node meets the compression conditions, the Raft log compression process is performed;

[0037] When the system meets the heartbeat timeout condition, the Raft leader election process begins.

[0038] The Raft log synchronization process: The master node serializes the data request sent by the client into a Raft log and sends this Raft log to all slave nodes and backup nodes; after receiving the Raft log, the slave node performs log replication, log submission, and log application operations; after receiving the Raft log, the backup node performs log replication and log submission.

[0039] The Raft log compression process: All nodes in the cluster determine whether they meet the compression conditions; the master node and slave nodes that meet the compression conditions obtain snapshots from the state machine and perform log compression; the backup nodes that meet the compression conditions request snapshot data from the master node and perform log compression;

[0040] The Raft master election process: After a slave node or backup node finds that it meets the timeout heartbeat condition, it votes to become the new master node; if the new master node does not have a complete data copy, the master node transfer operation is performed; after the master node transfer, the node without a complete data copy is transformed into a slave node or backup node.

[0041] Furthermore, the Raft log synchronization process further includes: after the log replication is completed, the slave node and the backup node reply a message to the master node;

[0042] Furthermore, the Raft log synchronization process further includes: after the master node receives the reply messages from the slave node and the backup node, it performs log submission, log application, and reply to the client.

[0043] Furthermore, the election voting in the Raft leader selection process is specifically as follows:

[0044] When a node finds that it meets the heartbeat timeout condition, it increases its term by one, transforms itself into a candidate node, initiates an election, and sends a voting request to other nodes.

[0045] After receiving the voting request message, other nodes perform the voting operation and return the voting response message containing the voting results to the candidate node;

[0046] The candidate node determines whether the received voting results meet the preset conditions. If so, it is successfully elected as the master node and sends heartbeats to other nodes.

[0047] Furthermore, the master node transfer operation in the Raft master election process is specifically as follows:

[0048] Send logs to randomly selected slave nodes to match the log progress of the master node;

[0049] The selected slave node initiates a new round of election voting;

[0050] When the voting results meet the preset conditions, the selected slave node becomes the new master node.

[0051] Furthermore, the compression condition in the Raft log compression process is specifically: whether the length of the log itself exceeds a threshold.

[0052] In a third aspect, the present invention further provides an electronic device, comprising:

[0053] a memory for non-transitory storage of computer-readable instructions; and

[0054] a processor for executing said computer-readable instructions,

[0055] When the computer-readable instructions are executed by the processor, the method described in the second aspect is executed.

[0056] In a fourth aspect, the present invention further provides a storage medium that non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the second aspect are executed.

[0057] One or more of the above technical solutions have the following beneficial effects:

[0058] The present invention introduces a new backup node Recorder role into the Raft algorithm, modifies the Raft algorithm log synchronization process, and cancels the log application part of the backup node, thereby achieving the effect of reducing redundant copies of data.

[0059] The present invention adjusts the node's heartbeat timeout (Election Timeout) according to the Raft role to which the node belongs, thereby increasing the probability that the node with a complete data copy becomes the master node.

[0060] If the new master node generated by the cluster of the present invention does not have a complete data copy, the master node is automatically transferred to a node with a complete data copy.

[0061] The present invention designs a log compression method for a backup node, obtains a data snapshot from a master node, completes log compression, and solves the problem that the backup node does not have a complete data copy and cannot perform log compression.

[0062] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0064] Figure 1 This is a flowchart of Raft log synchronization in the prior art;

[0065] Figure 2 This is the Raft role diagram of the first embodiment;

[0066] Figure 3 Node selection flowchart for the first embodiment.

[0067] Figure 4 This is a log synchronization flowchart of the second embodiment.

[0068] Figure 5 This is a log compression flow chart of the third embodiment.

[0069] Figure 6 This is a main selection flow chart of the fourth embodiment. DETAILED DESCRIPTION

[0070] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] The overall idea proposed by the present invention is:

[0072] The new backup node Recorder role is introduced into the Raft algorithm. F nodes are selected from the cluster's N = 2F + 1 nodes to become backup nodes. Backup nodes are similar to slave nodes and can participate in voting and elections. The only difference is that backup nodes do not have a state machine and do not save a complete copy of the data. By eliminating the log application step of the backup node and reducing redundant copies of data, storage and computing overhead can be greatly reduced.

[0073] Example 1

[0074] This embodiment discloses a Raft distributed system;

[0075] like Figure 2 As shown, a Raft distributed system includes multiple nodes, which are divided into master nodes, slave nodes, candidate nodes and backup nodes;

[0076] Specifically, based on the three roles of the master node Leader, the slave node Follower, and the candidate node Candidate, a new role is added: the backup node Recorder, and a set number of slave nodes are randomly selected and configured as backup nodes;

[0077] Among them, the master node is configured to: execute the client's data request and send the data request log to the slave node and backup node; the slave node is configured to: receive the data request log of the master node and execute the data request associated with the data request log; the candidate node is configured to: be in the intermediate state during the election of the master node and initiate the election voting process of the master node; the backup node is configured to: have no state machine and do not save a complete copy of the data.

[0078] The existing Raft algorithm has three roles: the leader node, the follower node, and the candidate node. On this basis, the role of the recorder node is added, so that the system roles are divided into four types: the leader node, the follower node, the candidate node, and the recorder node. On the node in the recorder role, there is no state machine, does not save a complete copy of the data, does not need to apply logs, but can elect and vote.

[0079] When Raft is running normally, all nodes are in the follower state. After the follower votes in the election, it will change itself into a candidate node. The candidate node sends a request for votes to other nodes through remote procedure call (RPC). At this time, there are two situations:

[0080] If a candidate node receives votes from a majority of nodes, it will be elected as the leader node and send heartbeat messages to other nodes to prevent unnecessary elections of other nodes;

[0081] If the candidate node does not receive votes from more than half of the nodes after the timeout, a new round of election will be triggered, and it will continue to send messages requesting votes to other nodes;

[0082] After the leader node is generated in the cluster, the leader node will detect the number of recorder nodes in the cluster. Assuming the number of cluster nodes is N = 2F + 1, there are three situations at this time:

[0083] If the number of recorder nodes is less than F, assumed to be a (0 <= a < F), the leader node will randomly select F - a nodes from the follower nodes and change these follower nodes into recorder nodes through configuration changes. [[ID=!7]]

[0084] If the number of recorder nodes is greater than F, assumed to be a (a > F), the leader node will randomly select a - F nodes from the recorder nodes and change these recorder nodes into follower nodes through configuration changes.

[0085] If the number of recorder nodes is equal to F, no modification is required.

[0086] Steps for configuration changes:

[0087] (1) The leader node serializes the configuration change information into a Raft log.

[0088] The configuration change information is the self - state of the node. For example, if follower node B is to be changed into a recorder node, leader node A can create a request for configuration change information, serialize it into a log and send it to other nodes. The specific configuration change information ChangeState is as follows:

[0089] ChangeState{

[0090] Id: The identity of the node whose state is to be changed

[0091] State: new state, Follower or Recorder

[0092] }

[0093] (2) The master node sends the Raft log to other nodes.

[0094] (3) After receiving replies from most nodes, the master node commits the log and the configuration change is successful.

[0095] Take a distributed cluster with three nodes as an example. When the cluster starts, all nodes are slave nodes. The three nodes are Follower a, Follower b, and Follower c, and their term is 1.

[0096] like Figure 3 As shown in the figure, the specific steps for selecting a backup node are:

[0097] (1) Follower a times out first, increments its term by one, becomes a Candidate, initiates an election, and sends a voting request to Follower b and Follower c.

[0098] (2) Follower b and Follower c nodes receive the voting request information sent by Candidate a node, determine that the voting information is legal, and cast their votes for Candidate a.

[0099] (3) Candidate a receives the voting messages from Follower b and Follower c, transforms itself into the master node, and sends heartbeats to Follower b and Follower c.

[0100] (4) Follower b and Follower c nodes receive the heartbeat message sent by the master node a and set the current Term and master node.

[0101] (5) The master node a detects that there are less than one backup node, selects the follower c node as the recorder node, packages this command into a log, and sends it to the follower b and follower c nodes.

[0102] (6) After receiving the log, Follower b and Follower c nodes append it to the local log and reply to the master node a.

[0103] (7) After receiving the reply from Follower b or Follower c, the master node a submits the log and applies it to the state machine.

[0104] (8) The master node a broadcasts the latest CommitIndex.

[0105] (9) Follower b and Follower c nodes submit logs.

[0106] (10) Follower b node applies the log to the state machine, and Follower c node transforms itself into a backup node.

[0107] Example 2

[0108] This embodiment discloses a Raft consensus method, based on a Raft distributed system disclosed in the first embodiment, including:

[0109] When the client sends a data request, the Raft log synchronization process is performed;

[0110] When the node meets the compression conditions, the Raft log compression process is performed;

[0111] When the system meets the heartbeat timeout condition, the Raft leader election process begins.

[0112] like Figure 4 As shown in the figure, the Raft log synchronization process is that the master node serializes the data request sent by the client into a Raft log and sends this Raft log to all slave nodes and backup nodes. After receiving the Raft log, the slave node performs log replication, log submission, and log application operations. After receiving the Raft log, the backup node performs log replication and log submission. Specifically:

[0113] Step S401: The master node serializes the data request sent by the client into a Raft log and sends this Raft log to all slave nodes and backup nodes;

[0114] Step S402: After receiving the Raft log, the slave node performs log replication, log submission, and log application operations;

[0115] Step S403: After receiving the Raft log, the backup node replicates and commits the log.

[0116] The backup node does not have a state machine and does not perform the operation of applying the log to the state machine.

[0117] Specifically, in the original Raft algorithm, client requests are serialized into logs by the master node. After three steps: log replication, log submission, and log application, they are ultimately applied to the state machines of all nodes. In a cluster with N = 2F + 1 nodes, a piece of data needs to be stored in N copies. By introducing the Recorder role and eliminating the log application portion of the backup node, only F + 1 copies of a piece of data are ultimately stored, reducing the number of redundant copies by F, significantly reducing computational and storage overhead.

[0118] Taking the distributed cluster in Example 1 as an example, after the backup node is selected, the three nodes in the cluster are the master node a, the follower node b, and the recorder node c. The log synchronization method after introducing the recorder role is as follows:

[0119] (1) The client initiates a data request to the master node a.

[0120] (2) The master node a serializes the request into a Raft log and appends it to the local log, which is log replication.

[0121] (3) The master node a sends the log to the follower node b and the recorder node c via RPC.

[0122] (4) Follower b node and Recorder c node receive the log and append it to the local log, which is log replication.

[0123] (5) Follower node b and Recorder node c reply to master node a that the log has been successfully received.

[0124] (6) After Follower b or Recorder c successfully replies to Master a, Master a commits the log.

[0125] (7) Master node a applies the log to the state machine.

[0126] (8) The master node a responds to the client.

[0127] (9) The master node a broadcasts the latest CommitIndex.

[0128] (10) Follower b node and Recorder c node update CommitIndex and commit the log.

[0129] (11) Follower b node applies the log to the state machine.

[0130] like Figure 5 As shown in the figure, the Raft log compression process is that all nodes in the cluster determine whether they meet the compression conditions. The master node and slave nodes that meet the compression conditions obtain snapshots from the state machine and perform log compression. The backup nodes that meet the compression conditions request snapshot data from the master node and perform log compression. Specifically:

[0131] Step S501: All nodes in the cluster detect whether logs need to be compressed;

[0132] Step S502: The master node and the slave node that meet the compression conditions obtain snapshots from the state machine and perform log compression;

[0133] Step S503: The backup node that meets the compression condition requests snapshot data from the master node and performs log compression.

[0134] Specifically, as clients continue to issue requests, the Raft log will become longer and longer. However, in practice, it is impossible to have infinite space to store Raft logs. Moreover, when new nodes join, the excessively long logs that need to be synchronized will also affect the availability of the system. Raft provides snapshot technology to solve this problem. In the original Raft algorithm, each node takes a snapshot independently. However, after the introduction of backup nodes, because backup nodes do not save a complete copy of the data, they cannot take snapshots independently. However, the master node or slave node has a complete copy of the data and can request snapshot data from the master node or slave node. In this embodiment, when the backup node performs log compression, it needs to request snapshot data from the master node. After the master node sends the snapshot data, the backup node can save the snapshot and compress the log.

[0135] Taking the distributed cluster in Example 1 as an example, as the number of requests received by the master node a continues to increase, the Raft log becomes longer and longer, and the log needs to be compressed. Taking the log compression of the Recorder node c as an example, the log compression process is as follows:

[0136] (1) Recorder c node detects whether the length of its own log exceeds the threshold.

[0137] (2) Recorder c node does not have a complete copy of the data. If the threshold is exceeded, Recorder c node requests snapshot data from the master node a node.

[0138] (3) The master node a sends the snapshot to the recorder node c.

[0139] (4) Recorder c receives the snapshot data sent by master a and persists it.

[0140] (5) Recorder c node compresses the log.

[0141] like Figure 6 As shown in the figure, the Raft master election process is that after a slave or backup node finds that it meets the timeout heartbeat condition, it votes to become the new master node. If the new master node does not have a complete data copy, it will perform the master node transfer operation. After the master node transfer, the node without a complete data copy will be transformed into a slave node or backup node. Specifically:

[0142] Step S601: After a slave node or backup node finds that it meets the timeout heartbeat condition, it becomes the new master node through election voting;

[0143] Step S601: If the new master node does not have a complete data copy, it performs a master node transfer operation;

[0144] Step S601: After the master node is transferred, the node that does not have a complete data copy is transformed into a slave node or a backup node.

[0145] Specifically, the election of the master node in Raft is triggered by a heartbeat mechanism. When Raft is running, nodes are in one of three states: follower, master, or recorder. The master node periodically sends heartbeats to other nodes to maintain its status. If a follower or backup node does not receive a heartbeat message from the master node within a period of time, it will assume that the current master node is down or a network partition has occurred and will initiate an election. The node that initiates the election earlier has a greater chance of becoming the master node. Because backup nodes do not store a complete copy of the data and cannot read the data, two principles are set:

[0146] (1) Try to let the slave node initiate the election to become the master node.

[0147] (2) If the backup node initiates an election to become the master node due to incomplete logs or other reasons, the master node must be transferred to a node with a complete data copy.

[0148] Because the node that initiates the election earliest has the highest probability of becoming the master node, the early or late node election can be controlled by modifying the node's heartbeat timeout. In this embodiment, the probability of the backup node initiating an election to become the master node is reduced by setting the backup node's heartbeat timeout to 1.2 times the slave node's heartbeat timeout.

[0149] The complete process of selecting a new leader after the introduction of the Recorder role is as follows:

[0150] (1) Check whether the heartbeat has timed out. If not, the node remains unchanged.

[0151] (2) If a node's heartbeat times out, the node increases its term by one, transforms itself into a candidate, initiates an election, and sends a voting request to other nodes. Because the slave node's heartbeat timeout (Election Timeout) is smaller, the slave node initiates the election earlier than the backup node.

[0152] (3) Determine whether the voting results meet the preset conditions, such as whether the majority of nodes vote. If so, the master node is successfully elected, sends heartbeats to other nodes, and checks whether it has a complete data copy. If it has a complete data copy, it ends.

[0153] (4) If there is no complete copy of the data, the master node needs to be transferred.

[0154] (5) If the voting result does not meet the preset conditions, check whether a master node is generated in this round of election. If a new master node is generated, transform itself into a slave node or backup node.

[0155] (6) If no new master node is generated, a new round of election is initiated.

[0156] The specific steps for transferring the master node are as follows:

[0157] (1) Stop accepting client requests.

[0158] (2) Select a slave node.

[0159] (3) Send logs to the slave nodes to match the log progress of the master node.

[0160] (4) Notify the node to initiate a new round of elections.

[0161] (5) The node increases its term by one and sends a message to other nodes requesting a vote.

[0162] (6) The node receives votes from the majority of nodes and becomes the new master node.

[0163] Example 3

[0164] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the method described in the above-mentioned embodiment 2.

[0165] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0166] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0167] During implementation, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software.

[0168] The method in Example 2 can be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0169] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0170] Example 4

[0171] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the second embodiment is performed.

[0172] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A Raft distributed system, comprising a plurality of nodes, characterized in that: The nodes are divided into master nodes, slave nodes, candidate nodes and backup nodes; The master node is configured to: execute the data request of the client and send the data request log to the slave node and the backup node; The slave node is configured to: receive the data request log of the master node and execute the data request associated with the data request log; The candidate node is configured to: be in an intermediate state during the process of electing the master node and initiate the voting process for the election of the master node; The backup node is configured as follows: it does not have a state machine and does not store a complete copy of the data. After the system generates a master node, it randomly selects a set number of slave nodes to be configured as backup nodes; The randomly selected set number of slave nodes are configured as backup nodes, specifically: The newly generated master node detects and configures the backup nodes in the cluster; Assume that the number of cluster nodes is N=2F+1, where F is a non-negative integer. F nodes are randomly selected from the slave nodes and converted into backup nodes.

2. A Raft consensus method, characterized in that: A Raft distributed system according to claim 1, comprising: When the client sends a data request, the Raft log synchronization process is performed; When the node meets the compression conditions, the Raft log compression process is performed; When the system meets the heartbeat timeout condition, the Raft leader election process begins. The Raft log synchronization process: The master node serializes the data request sent by the client into a Raft log and sends this Raft log to all slave nodes and backup nodes; after receiving the Raft log, the slave node performs log replication, log submission, and log application operations; after receiving the Raft log, the backup node performs log replication and log submission. The Raft log compression process: All nodes in the cluster determine whether they meet the compression conditions; the master node and slave nodes that meet the compression conditions obtain snapshots from the state machine and perform log compression; the backup nodes that meet the compression conditions request snapshot data from the master node and perform log compression; The Raft master election process: After a slave node or backup node finds that it meets the timeout heartbeat condition, it votes to become the new master node; if the new master node does not have a complete data copy, the master node transfer operation is performed; after the master node transfer, the node without a complete data copy is transformed into a slave node or backup node.

3. A Raft consensus method according to claim 2, characterized in that: The Raft log synchronization process further includes: after the log replication is completed, the slave node and the backup node reply a message to the master node.

4. A Raft consensus method according to claim 2, characterized in that: The Raft log synchronization process also includes: after the master node receives the reply messages from the slave node and the backup node, it performs log submission, log application, and reply to the client operation.

5. A Raft consensus method according to claim 2, characterized in that: The election voting in the Raft leader election process is specifically as follows: When a node finds that it meets the heartbeat timeout condition, it increases its term by one, transforms itself into a candidate node, initiates an election, and sends a voting request to other nodes. After receiving the voting request message, other nodes perform the voting operation and return the voting response message containing the voting results to the candidate node; The candidate node determines whether the received voting results meet the preset conditions. If so, it is successfully elected as the master node and sends heartbeats to other nodes.

6. A Raft consensus method according to claim 2, characterized in that: The master node transfer operation in the Raft master election process is specifically as follows: Send logs to randomly selected slave nodes to match the log progress of the master node; The selected slave node initiates a new round of election voting; When the voting results meet the preset conditions, the selected slave node becomes the new master node.

7. A Raft consensus method according to claim 2, characterized in that: The compression condition in the Raft log compression process is specifically whether the length of the log itself exceeds a threshold.

8. An electronic device, comprising: a memory for non-transitory storage of computer-readable instructions; as well as a processor for executing said computer-readable instructions, When the computer-readable instructions are executed by the processor, the method according to any one of claims 2 to 7 is executed.

9. A storage medium, characterized by non-transitory storage of computer-readable instructions, wherein: When the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 2 to 7 are executed.

Images