A cluster processing method and storage medium

Abstract

The present application proposes a cluster processing method and storage medium, wherein the cluster includes multiple nodes, and the types of these multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on a consistency protocol. The follower node can receive heartbeat messages sent by the leader node, and after determining that the client node with a heartbeat timeout is a node to be isolated based on the heartbeat detection information of each client node, a heartbeat response message containing isolation prompt information for the node to be isolated can be sent to the leader node to prompt the leader node to quickly and accurately determine whether the node to be isolated is in an offline state, and to isolate the offline node grid in a timely manner, thereby avoiding the adverse impact of such client nodes in an offline state on the original cluster and improving business response and fault recovery efficiency.

Description

Technical Field

[0001] The present application mainly relates to the field of computers, and more specifically to a cluster processing method and storage medium. Background Art

[0002] Raft is a distributed consistency protocol that has been widely used in many fields. It can maintain data state consistency among multiple nodes in a cluster and achieve high availability of the cluster.

[0003] In cluster processing based on the Raft protocol, since the cluster consists of a leader node and multiple follower nodes, in application scenarios such as log data persistence, the leader node must confirm that more than half of the follower nodes have successfully persisted the data before it can be considered a successful log data persistence. For cluster processing involving a large number of nodes, this can lead to technical issues such as slow service response and slow fault recovery in the cluster system. Summary of the Invention

[0004] To solve the above technical problems, the present application proposes a cluster processing method, wherein the cluster includes multiple nodes, and the types of the multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on a consensus protocol. The cluster processing method is applied to the follower nodes, including:

[0005] Receive a heartbeat message sent by the leader node;

[0006] Determining, based on the heartbeat detection information for each of the client nodes, that the client node with a heartbeat timeout is a node to be isolated; wherein the heartbeat detection information can be updated based on a heartbeat message sent by the client node;

[0007] A heartbeat response message is sent to the leader node; the heartbeat response message includes at least isolation prompt information for the node to be isolated, so as to prompt the leader node to determine whether the node to be isolated is in an offline state.

[0008] Optionally, also include:

[0009] Determining that the heartbeat message from the leader node includes a node isolation identifier, prohibiting the sending of service messages to the client node corresponding to the node isolation identifier; the node isolation identifier is used to identify the client node in an offline state; and / or,

[0010] Determining that the heartbeat message from the client node includes a to-be-recovered identifier, adding status recovery information for the client node to a heartbeat response message and sending it to the leader node;

[0011] Determine that the heartbeat message from the leader node includes a node recovery identifier, delete the node isolation identifier corresponding to the client node, and resume sending business messages to the client node corresponding to the node recovery identifier; the node recovery identifier is used to identify the client node that has recovered from the offline state to the normal operating state.

[0012] Optionally, the method further includes:

[0013] Receive a voting request message sent by the candidate node; the voting request message includes a first offline node number and a log submission parameter; the first offline node number is the number of client nodes in an offline state in the candidate node view, and the log submission parameter can represent the local persistent log data of the candidate node;

[0014] Determine the number of second offline nodes and local log submission parameters recorded locally; the second offline node number is the number of client nodes in the offline state in the follower node view, and the local log submission parameters can represent the local persistent log data of the follower node;

[0015] Detecting whether the candidate node meets a master election condition based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the log submission parameter;

[0016] Sending a voting response message corresponding to the detection result to the candidate node.

[0017] Optionally, detecting, based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the submitted log parameter, whether the candidate node meets a master election condition, and sending a voting response message corresponding to the detection result to the candidate node includes:

[0018] Detecting that the number of the first offline nodes is greater than the number of the second offline nodes, and the local log submission parameter is the same as the submitted log parameter, and determining that the candidate node does not meet the master election condition;

[0019] Sending a voting response message to the candidate node to reject the vote, triggering the candidate node to switch to a follower node, and the follower node itself switches to a new candidate node;

[0020] Sending a voting request message to each follower node in the current cluster; the voting request message includes the second offline node number and the local log submission parameter recorded by the new candidate node.

[0021] Optionally, detecting, based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the submitted log parameter, that the candidate node does not meet the master election condition, and sending a voting response message corresponding to the detection result to the candidate node includes:

[0022] Detecting that the number of the first offline nodes is less than or equal to the number of the second offline nodes, and / or the local log submission parameter is different from the submitted log parameter, and determining that the candidate node meets the master election condition;

[0023] Sending a voting response message to the candidate node to confirm the vote;

[0024] Receive a voting result message sent by a new leader node; the new leader node is obtained by switching when the candidate node determines that the number of confirmed votes exceeds half of the total number of follower nodes based on the voting response messages from each follower node.

[0025] The present application also proposes a cluster processing method, wherein the cluster includes multiple nodes, and the types of the multiple nodes include a leader node, a follower node, a candidate node, and a client node defined based on a consensus protocol. The cluster processing method is applied to the leader node, comprising:

[0026] Sending a heartbeat message to each of the following nodes;

[0027] Receiving a heartbeat response message sent by the follower node; the heartbeat response message includes at least isolation prompt information for the node to be isolated, and the node to be isolated is the client node with a heartbeat timeout determined by the follower node based on the heartbeat detection information for each of the client nodes;

[0028] Determine whether the node to be isolated is in an offline state based on the isolation prompt information from different follower nodes.

[0029] Optionally, determining whether the node to be isolated is in an offline state based on the isolation prompt information from different follower nodes includes:

[0030] Obtaining the number of prompts of the isolation prompt information for the same node to be isolated;

[0031] Detecting that the number of prompts is greater than a preset number, determining that the corresponding node to be isolated is in an offline state; the preset number is half of the total number of the leader node and the follower nodes;

[0032] It is detected that the number of prompts is less than or equal to the preset number, and it is determined that the corresponding node to be isolated is in a normal operating state.

[0033] Optionally, also include:

[0034] A node isolation identifier for the node to be isolated that is in an offline state is configured, the node isolation identifier is added to a next heartbeat message, and the next heartbeat message is sent to each of the follower nodes.

[0035] The present application also proposes a cluster processing method, wherein the cluster includes multiple nodes, and the types of the multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on a consensus protocol. The cluster processing method is applied to the client nodes, including:

[0036] Sending a heartbeat message to the leader node and the follower node;

[0037] receiving a heartbeat response message from the leader node and / or the follower node;

[0038] Determining the number of received heartbeat response messages;

[0039] It is detected that the received number is less than or equal to a preset number, and it is determined that the client node itself is in an offline state, where the preset number is half of the total number of the leader node and the follower nodes.

[0040] Optionally, also include:

[0041] When it is determined that the client node itself is in an offline state, configuring a corresponding node isolation flag to prohibit sending messages to the leader node and the follower node;

[0042] detecting a failure recovery that caused the client node to enter an offline state, and configuring a pending recovery flag for the client node; the pending recovery flag is used to instruct the leader node to determine whether the client node has recovered from the offline state to a normal operating state;

[0043] The to-be-recovered identifier is added to a next heartbeat message, and the next heartbeat message is sent to each of the follower nodes and the leader node.

[0044] Optionally, also include:

[0045] When it is determined that the client node itself is in an offline state, triggering the client node to switch to a new follower node and receive a heartbeat message sent by the leader node; the heartbeat message includes the log data to be replicated and the log submission parameters of the local persistent log data of the leader node;

[0046] Performing persistence processing on the log data to be copied, executing operations corresponding to the persistent log data, and updating local log submission parameters of the local persistent log data;

[0047] Determine that the updated local log submission parameter is the same as the log submission parameter from the leader node, maintain the new follower node unchanged or trigger the new follower node to recover as the client node.

[0048] Optionally, also include:

[0049] Sending an access request message to the leader node;

[0050] Receive a confirmation access message sent by the leader node; the confirmation access message includes log submission parameters of the current persistent log data of the cluster and view information of the leader node; the view information can represent the current status of each node in the cluster;

[0051] Sending a heartbeat message to the follower node and the leader node in the cluster; the heartbeat message includes log acquisition request information, and the log acquisition request information is determined according to the log submission parameter of the persisted log data;

[0052] Receive the heartbeat response message sent by the follower node and the leader node, and execute the operation corresponding to the requested log data contained in the heartbeat response message.

[0053] Optionally, also include:

[0054] Determining, based at least on the states of the nodes, a secondary node for the client node itself in the cluster; the secondary node being the leader node or the follower node;

[0055] The heartbeat message contains log acquisition request information, including:

[0056] The heartbeat message sent to the auxiliary node includes log acquisition request information;

[0057] The executing the operation corresponding to the requested log data included in the heartbeat response message includes:

[0058] Execute the operation corresponding to the requested log data; the requested log data is included in the heartbeat response message from the auxiliary node and is obtained by the auxiliary node according to the log acquisition request information.

[0059] The present application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the cluster processing method as described above;

[0060] The cluster includes multiple nodes, and the types of the multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on the consistency protocol.

[0061] It can be seen that the present application provides a cluster processing method and storage medium, which includes multiple nodes. The types of these multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on the consistency protocol. The follower node can receive the heartbeat message sent by the leader node. According to the heartbeat detection information of each client node, after determining that the client node with heartbeat timeout is the node to be isolated, it can send a heartbeat response message containing isolation prompt information for the node to be isolated to the leader node, so as to prompt the leader node to quickly and accurately determine whether the node to be isolated is in an offline state, and isolate the offline node grid in time, thereby avoiding the adverse effects of such offline client nodes on the original cluster and improving business response and fault recovery efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0063] Figure 1 This is a schematic diagram of an improvement for defining node types in a cluster based on a consensus protocol in the cluster processing method proposed in this application;

[0064] Figure 2 A flowchart of an optional example of the cluster processing method proposed in this application;

[0065] Figure 3 A network link diagram between different types of nodes in a cluster applicable to the cluster processing method proposed in this application;

[0066] Figure 4 A flowchart of another optional example of the cluster processing method proposed in this application;

[0067] Figure 5 A flowchart of another optional example of the cluster processing method proposed in this application;

[0068] Figure 6 A flowchart of another optional example of the cluster processing method proposed in this application;

[0069] Figure 7 A flowchart of another optional example of the cluster processing method proposed in this application;

[0070] Figure 8 A flowchart of another optional example of the cluster processing method proposed in this application;

[0071] Figure 9 A flowchart of another optional example of the cluster processing method proposed in this application;

[0072] Figure 10 A schematic diagram of a scenario in which a client node is converted to a follower node for failure recovery in the cluster processing method proposed in this application;

[0073] Figure 11 A flowchart of another optional example of the cluster processing method proposed in this application;

[0074] Figure 12 A schematic structural diagram of an optional example of a cluster processing device proposed in this application;

[0075] Figure 13 This is a structural diagram of another optional example of the cluster processing device proposed in this application;

[0076] Figure 14 This is a structural diagram of another optional example of the cluster processing device proposed in this application;

[0077] Figure 15 The diagram is a structural diagram of an optional example of a computer device suitable for the cluster processing method proposed in this application. DETAILED DESCRIPTION

[0078] In response to the technical problems described in the background technology section, the consensus protocol based on which multiple nodes in a cluster achieve consistency, such as the Raft protocol, defines node types (i.e., roles in the cluster), such as leader nodes, follower nodes, and candidate nodes. This application improves the consensus protocol, such as Figure 1 As shown, a newly defined node type, the client node, is added. In this way, when the number of nodes contained in the cluster is determined, the number of follower nodes contained in the cluster is reduced based on the improved consistency protocol proposed in this application. In this way, when processing business according to the principle of minority obeys majority, the number of response messages that the leading node needs to obtain from the follower nodes is reduced, and feedback from more than half of the follower nodes can be obtained more quickly, thereby improving business response speed, fault recovery speed, etc.

[0079] Furthermore, the client nodes newly defined in this application are typically data nodes. Compared to server nodes, they do not need to persist log data locally, reducing the hardware cost and resource usage of the cluster. Furthermore, for service nodes such as the leader and follower nodes mentioned above, this application newly defines a node monitoring function. These nodes can act as monitoring nodes and monitor the status of other nodes under their view. This allows for the timely isolation of a faulty node upon discovery, preventing it from participating in subsequent business processing. This avoids the inability to process business and feedback response messages in a timely manner due to the node failure, which in turn reduces business response speed.

[0080] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0081] Reference Figure 2 , is a flow chart of an optional example of the cluster processing method proposed in this application. Combined with the above description of the technical solution of this application, the cluster may include multiple nodes. The types of these multiple nodes may include leader nodes, follower nodes, candidate nodes, and client nodes defined based on a consensus protocol (such as the improved raft protocol mentioned above). The embodiment of this application can describe the implementation process of the cluster processing method from the follower node side, such as Figure 2 As shown, the method may include but is not limited to the following steps:

[0082] Step S21, receiving a heartbeat message sent by the leader node;

[0083] For clusters such as distributed storage systems, the improved Raft protocol (a distributed consistency protocol) can be used to maintain consistency among multiple nodes in the cluster. For example, each node can maintain consistency on shared storage at the same time. When a few follower nodes fail, the normal operation of the cluster is not affected and business services can still be provided normally.

[0084] In the process of cluster processing business, refer to Figure 3 The network link diagram between different types of nodes in the cluster shown is based on the heartbeat mechanism for communication between the leader node and each follower node. That is, the leader node can send heartbeat messages (that is, heartbeat packets / messages obtained based on the consistency protocol) to each follower node at a preset time interval (that is, the heartbeat cycle). The heartbeat message can be sent in a broadcast manner but is not limited to it. This application does not elaborate on the implementation method of the leader node sending the heartbeat message.

[0085] Among them, the raft protocol can be based on replicated state machines, that is, each node in the cluster can start from the same state, go through the same sequence of operations (such as log entries), and finally reach the same state, such as achieving log consistency of each node. The embodiment of the present application does not elaborate on how each node in the cluster maintains consistency based on the improved raft protocol.

[0086] Based on this, the heartbeat message sent by the leader node may include: the leader node's local persistent log data (such as log entries used to record client operations such as reading / writing data), commit value (that is, the log submission parameter of the persisted log data, which can also be called the log number), term number (Term id) and other view information. This application does not limit the content of the heartbeat message sent by the leader node, which can be determined according to the situation.

[0087] Step S22, determining the client node with a heartbeat timeout as a node to be isolated based on the heartbeat detection information for each client node; the heartbeat detection information can be updated based on the heartbeat message sent by the client node;

[0088] In an embodiment of the present application, based on the consistency protocol, the newly added client node can actively send a heartbeat message to the follower node based on the heartbeat mechanism. The follower node can use the received heartbeat message to update the heartbeat detection information for the client node, so that it can determine whether the communication with the corresponding client node is abnormal based on the updated heartbeat detection information (this application does not limit the content contained therein, and it can be determined as the case may be). If no heartbeat message is received from the client node within a period of time (such as the heartbeat period determined by the heartbeat mechanism for the client node and the follower node), it means that the heartbeat of the client node has timed out, and it can be considered that the client node may be faulty. For the convenience of description, such client nodes can be recorded as nodes to be isolated.

[0089] During the consensus protocol-based communication process between a follower node and a client node, the client node performs operations corresponding to the log data it obtains (which can come from the follower node and / or the leader node), obtains the corresponding memory value, and includes this value in a heartbeat message sent to the follower node and the leader node. Thus, upon receiving the heartbeat message from the client node, the follower node can promptly update its own memory value of the client node to represent the latest heartbeat message from the client node, thereby determining whether the heartbeat message exchanged with the client node has timed out.

[0090] Among them, the memory value corresponding to each client node recorded by the follower node can have a corresponding update time. In this way, the follower node can monitor the time difference between the update time of the memory value and the current time, and determine whether the heartbeat message exchanged between the follower node and the client node has timed out by detecting whether the time difference reaches the corresponding time interval (such as the heartbeat period determined by the heartbeat mechanism of the client node and the follower node). In this way, the client node with a heartbeat timeout is determined, that is, after the last heartbeat message from the client node is received, the heartbeat message sent by the client node has not been received for more than a corresponding time interval. Such client nodes can be recorded as nodes to be isolated. It can be seen that in this embodiment, the above-mentioned heartbeat detection information may include but is not limited to the memory value of the corresponding client node and its update time, etc.

[0091] Optionally, in actual applications, the follower node defined based on the consistency protocol can obtain the isolated node determined during the heartbeat interaction with the client node within a preset time window according to the method described above. The duration of the preset time window is less than or equal to the term duration of the current leader node, that is, the effective duration of the leader node as the cluster leader. This application does not impose any restrictions on the preset time window or the term duration of the leader node, which can be determined according to the circumstances.

[0092] It should be noted that the two communication processes, such as the heartbeat interaction between the follower node and the client node, and the heartbeat interaction between the follower node and the leader node, do not affect each other. That is to say, these two heartbeat interaction processes can be executed independently. After the follower node determines that there are nodes to be isolated under its view according to the above method, it can add it to the heartbeat response message sent to the leader node when sending a heartbeat response message to the leader node, so that the leader node can further determine whether each node to be isolated is offline based on the heartbeat response messages fed back by multiple follower nodes.

[0093] Of course, if the follower node analyzes the heartbeat detection information of each client node and the node to be isolated whose heartbeat timeout is not determined, the follower node regularly receives the heartbeat messages sent by each client node during the heartbeat cycle with the leader node. The follower node does not need to feed back the heartbeat timeout detection result (i.e., abnormality detection result) of the client node obtained this time to the leader node. If the heartbeat message received from the leader node includes log data to be persisted, the follower node can execute the operation of recording the log data and complete the persistence operation of the local log data. This application does not elaborate on the log data consistency implementation process between the follower node and the leader node. The follower node will feed back the execution result of the log data from the leader node to the leader node to determine whether the persistence of the log data, i.e., commit, is successfully achieved.

[0094] Step S23: Send a heartbeat response message to the leader node; the heartbeat response message at least includes isolation prompt information for the node to be isolated, so as to prompt the leader node to determine whether the node to be isolated is in an offline state.

[0095] Following the above analysis, for any follower node in the cluster, in the process of responding to the heartbeat message from the leader node, if the received log data that needs to be committed is locally persisted and the leader node is waiting for the log data to be committed; or when the heartbeat message does not contain log data, the follower node directly responds to the heartbeat to determine whether the communication between the follower node and the leader node is normal, the follower node determines that there is a client node to be isolated (recorded as a node to be isolated) with a heartbeat timeout according to the method described above, and can generate an isolation prompt message for the node to be isolated to inform the leader node that the communication between the node to be isolated and the follower node is abnormal, and the node to be isolated may have been offline, so that the leader node can determine that more than half of the follower nodes have reported the same node to be isolated, and the node to be isolated can be considered to be offline, and a node isolation identifier can be configured for it. Regarding the cluster processing process of the leader node, please refer to the description of the corresponding embodiment below, and this embodiment will not be described in detail here.

[0096] It should be noted that this application does not limit the content and representation of the above-mentioned isolation prompt information. It may include the node identifier of the node to be isolated and the heartbeat timeout identifier configured for the node to be isolated, etc. According to actual needs, it may also include the memory value generated by the operation corresponding to the log data executed by the node to be isolated, etc. This application does not list them one by one here.

[0097] In summary, for the follower nodes defined based on the consistency protocol (such as the improved raft protocol) in the cluster, on the basis of achieving log data consistency between them and the leader node, they can also realize fault monitoring of the client nodes by detecting whether the heartbeat messages sent periodically by the client nodes have timed out, and promptly report the nodes to be isolated with the monitored heartbeat timeout to the leader node, so that the leader node can promptly mark the client nodes in the offline state for isolation, and promptly inform each follower node of the monitoring result that one or more client nodes are in the offline state by sending heartbeat messages, so that the follower nodes no longer need to send business messages to the client nodes in the offline state, that is, the network isolation of the client nodes in the offline state is realized, thereby avoiding the adverse effects of such client nodes in the offline state on the original cluster.

[0098] Among them, the present application defines a type of node, the client node, in the cluster based on the consistency protocol, so that the number of follower nodes defined based on the consistency protocol is reduced. In this way, in the process of cluster processing business, such as the fault monitoring business of the client node, the number of monitoring results of whether the heartbeat messages sent periodically by the follower nodes to the leader node to determine whether the client nodes have timed out is reduced, and it can more quickly and reliably determine whether the client node is offline, thereby improving the efficiency of fault monitoring.

[0099] Similarly, for log data that needs to be persisted, the leader node needs to determine fewer feedback results on the local log persistence status of the follower nodes, allowing it to more quickly determine whether the log data has been successfully persisted, that is, whether more than half of the follower nodes (i.e., half of the total number of follower nodes and the leader node) have successfully persisted the log data locally, thereby improving business response efficiency. Other types of task request messages (including) sent by the leader node to the follower nodes can also be determined using this minority-subordinates-majority processing method, thereby improving the business processing performance of the cluster.

[0100] Reference Figure 4 , which is a flow chart of another optional example of the cluster processing method proposed in this application. This embodiment can be applied to the leader node of the cluster. The node types of the multiple nodes contained in the cluster can include the definition of the leader node, follower node, candidate node and client node based on the consistency protocol, and the definition implementation process is not limited. The embodiment of this application can be the interactive implementation process of the cluster processing method executed by the above-mentioned follower node. Regarding the execution steps of the follower node in this method, you can refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail. Figure 4 As shown, the cluster processing method performed by the leader node proposed in this embodiment may include:

[0101] Step S41, sending a heartbeat message to each follower node;

[0102] Step S42: receiving a heartbeat response message sent by the follower node; the heartbeat response message may include at least isolation prompt information for the node to be isolated;

[0103] For the leader node in the current cluster, heartbeat messages can be sent regularly to each follower node and client node to maintain communication between the leader node and each follower node and client node. By monitoring whether these nodes can regularly feedback heartbeat response messages, it is determined whether the corresponding node is faulty, thereby realizing fault monitoring of large-scale nodes in the cluster, so as to promptly restore the faulty nodes, etc. The present application does not impose any restrictions on the implementation method of the leader node sending heartbeat messages to other nodes in the cluster. Under normal circumstances, the heartbeat message can include the local persistent log data of the leader node, that is, the log data that needs to be copied by the follower node, the log submission parameter (commit value), the view information of each follower node and client node connected to the leader node network, etc., so as to achieve consistency between each node that receives the heartbeat message and the leader node. The present application does not impose any restrictions on the content contained in the heartbeat message and its data format (such as the message format determined based on the communication protocol), which can be determined according to the circumstances.

[0104] According to the above description of the heartbeat message, the content of the heartbeat response message fed back by the follower node to the leader node can be determined based on the content of the heartbeat message received by the follower node from the leader node. For example, when the heartbeat message contains the log data to be persisted, the commit value, the term number and other contents, the follower node can persist the log data, update the commit value of the local record to the commit value from the leader node, etc., and include the successful execution result of the log data persistence in the heartbeat response message, etc. It can be seen that the follower node can passively receive the log data sent by the leader node and persist it, realize the two-stage application of the log, and ensure the log consistency between the leader node and the follower node. The data format of the heartbeat response message can be determined based on the requirements of the communication protocol. This application does not limit the generation and transmission implementation methods of each message (i.e., data packet / message).

[0105] Among them, during the heartbeat interaction processing between the leading node and the follower node, the follower node can also perform heartbeat interaction processing with the client node to determine the client node with heartbeat timeout, that is, the node to be isolated, that is, the follower node can serve as a monitoring node for the client node to realize fault monitoring of the client node. Combined with the above description from the follower node side, it can determine the client node with heartbeat timeout based on the heartbeat detection information of each client node. The implementation process of this application embodiment will not be described in detail here.

[0106] Step S43: Determine whether the node to be isolated is in an offline state based on the isolation prompt information from different follower nodes.

[0107] After the leader node of the cluster obtains the heartbeat response information sent by each follower node, it can obtain the isolation prompt information, log data persistence execution results and other contents contained in the heartbeat response message based on the principle of minority obeys majority (the nodes participating in the counting are the follower nodes and leader nodes of the cluster). Based on the content contained in the isolation prompt information, it can determine which client node’s heartbeat timeout is monitored by the follower node, that is, determine the number of nodes to be isolated reported by each follower node during the term of the leader node. Afterwards, the number of reports of multiple follower nodes on the same node to be isolated can be determined. If it exceeds more than half of the total number of the leader node and the follower node (recorded as the preset number), the node to be isolated can be considered to be offline.

[0108] If it is determined that the number of isolation prompt messages reported for the same node to be isolated does not exceed the preset number, the leader node may assume that the node to be isolated is in normal operation, or continue to analyze the heartbeat response messages fed back by the follower nodes. If, after a preset time, the number of isolation prompt messages reported for the node to be isolated still does not exceed the preset number, the leader node is determined to be in normal operation. Thereafter, the leader node may include the detection result of whether the node to be isolated is in an offline state in the heartbeat message to be sent. By sending this heartbeat message to the follower node, the follower node is promptly notified to isolate the offline client node, thereby preventing the adverse impact of offline client nodes (such as isolated nodes and split-brain nodes) on cluster performance.

[0109] In summary, based on the client nodes defined based on the consistency protocol added to the cluster, this application can ensure that the leader node monitors and manages all nodes in the entire cluster in accordance with the processing method described above, and can timely monitor the faulty nodes with brain split (that is, the determined client nodes that are offline), and perform fault processing in a timely manner to ensure the business processing performance of the cluster.

[0110] Reference Figure 5 , is a flow chart of another optional example of the cluster processing method proposed in this application. This embodiment can still be described from the follower node side, and can describe the application scenario of the cluster processing method described above, in which the follower node isolates the client node network in an offline state. Regarding the heartbeat interaction processing process between the follower node and the leader node, reference can be made to the description of the corresponding part of the above embodiment, which will not be described in detail in this embodiment. Figure 5 As shown, the cluster processing method proposed in this embodiment may include:

[0111] Step S51, receiving a heartbeat message; the heartbeat message comes from a leader node and / or a client node;

[0112] Step S52: determining that the heartbeat message from the leader node includes a node isolation identifier, prohibiting the sending of service messages to the client node corresponding to the node isolation identifier;

[0113] In combination with the description of the corresponding part of the above embodiment, the node isolation identifier can be used to identify the client node in the offline state. As for whether the client node is in the offline state, the implementation process of configuring the node isolation identifier for the client node in the offline state can refer to the cluster processing method described in the context from the leader node side. The embodiment of this application will not be described in detail here.

[0114] It can be seen that the follower nodes in the cluster of this application can monitor the faults of the client nodes. After reporting the monitoring results to the leader node according to but not limited to the above method, the leader node can reliably determine the client nodes that are offline based on the principle of minority obeys majority, and configure node isolation identifiers for them to inform the follower nodes which client nodes are isolated by the network. There is no need to send business messages to such client nodes, thereby saving resource consumption.

[0115] Step S53, determining that the heartbeat message from the client node includes a pending recovery flag, adding the status recovery information for the client node to the heartbeat response message and sending it to the leader node;

[0116] The client nodes in the cluster of this application have capabilities such as self-network monitoring and diagnosis and two-stage log application. Therefore, according to the method described above, the client nodes that are offline can be determined. The client nodes can use their own network monitoring and diagnosis capabilities to monitor whether the fault of the client node has been restored. If restored, a pending recovery flag can be configured for the client node, and based on the heartbeat mechanism, it can be actively reported to the follower node and the leader node. It can be seen that the pending recovery flag can indicate that the fault of the corresponding client node configured with the node isolation flag may have been restored. It can instruct the leader node to further determine the fault recovery status of the client node, thereby improving the efficiency and reliability of fault recovery.

[0117] Regarding the heartbeat interaction processing process between the follower node and the client node, please refer to the description of the corresponding part of the context. This application does not elaborate on the heartbeat interaction process between these two types of nodes, and does not limit the content of each message transmitted between the two and the communication protocol followed, which can be determined according to the situation.

[0118] In some other embodiments, after the client node determines that the fault has been recovered based on its own network monitoring and diagnosis capabilities, it can directly configure a node recovery identifier for the client node, include it in the next heartbeat message to be sent, and report it to the follower node and the leader node to restore normal business communications between the follower node and the leader node and the client node.

[0119] It should be noted that the self-network monitoring and diagnosis process of the client node is not limited to being completed within one heartbeat cycle of the client node. It can be determined based on the actual situation. After the fault is recovered, it continues to send heartbeat messages to the leader node and the client node based on the heartbeat mechanism. The heartbeat message may include but is not limited to a pending recovery identifier or a node recovery identifier, etc. This application does not limit the content of the heartbeat message sent by the client node. The content of the heartbeat message sent at different stages may be different, depending on the situation.

[0120] Step S54, receiving the heartbeat message sent by the leader node;

[0121] Step S55: Determine that the heartbeat message includes a node recovery identifier, delete the node isolation identifier corresponding to the client node, and resume sending service messages to the client node corresponding to the node recovery identifier.

[0122] Similar to the implementation process of the leader node determining whether the node to be isolated reported by the follower node is offline, after the leader node receives the heartbeat response message containing the to-be-recovered identifier reported by the follower node, it can determine whether the number of reported heartbeat response messages containing the same to-be-recovered identifier (from different follower nodes) exceeds the preset number. If it exceeds, it can be considered that the fault of the client node with the to-be-recovered identifier has been recovered; otherwise, it can be considered that the fault of the client node has not been recovered, or after waiting for a certain period of time, the number of follower nodes reporting the recovery of the fault of the client node still does not exceed the preset number, and then it is determined that the fault of the client node has not been recovered. This application does not limit the implementation method.

[0123] For the client node that the leader node previously determined to be isolated from the network, that is, the client node configured with a node isolation identifier, it can be determined during the heartbeat interaction between the leader node and the follower node, and the obtained node isolation identifier can be included in the heartbeat message to be sent. In the heartbeat interaction process, the leader node obtains the report from the client node configured with the node isolation identifier, and / or the fault recovery information reported by the follower node for the client node with the node isolation identifier, such as the above-mentioned identifier to be recovered. When the heartbeat interaction between the nodes in the cluster is uninterrupted, it is determined that the client node has recovered, and the obtained node recovery identifier of the corresponding client node is added to the heartbeat message and then sent.

[0124] In this way, the follower node can promptly learn that the client node in the offline state has resumed normal operation. In order to realize communication between the follower node and the client node and meet business processing requirements, the follower node can delete the node isolation identifier of the client node previously received from the leader node, so that it can resume sending business messages to the client node.

[0125] Reference Figure 6, which is a flow chart of another optional example of the cluster processing method proposed in this application. This embodiment can describe an optional detailed implementation of the cluster processing method described above, such as Figure 6 As shown, the method may include:

[0126] Step S61: The leader node sends a heartbeat message to the follower node;

[0127] Step S62: The follower node determines, based on the heartbeat detection information for each client node, that the client node with a heartbeat timeout is a node to be isolated; the heartbeat detection information can be updated based on the heartbeat message sent by the client node;

[0128] Step S63: The follower node sends a heartbeat response message to the leader node; the heartbeat response message includes at least isolation prompt information for the node to be isolated;

[0129] Step S64: The leader node receives the heartbeat response message and obtains isolation prompt information from different follower nodes;

[0130] Regarding the implementation process of step S61 to step S64, reference may be made to the description of the cluster processing method performed by the corresponding side in the above embodiment, and the embodiment of the present application will not be described in detail here.

[0131] Step S65: The leader node obtains the number of isolation prompts for the same node to be isolated.

[0132] Step S66: The leader node detects that the number of prompts is greater than a preset number, and determines that the corresponding node to be isolated is in an offline state; the preset number is half of the total number of the leader node and the follower node;

[0133] This application defines a leader node based on the consistency protocol as a cluster manager, a log data persistence initiator, and implements basic functions such as log two-stage application. It broadcasts and submits persisted log numbers, view information, etc. at regular intervals. It can also serve as a large-scale node monitoring node to monitor the status of follower nodes and client nodes; follower nodes can also serve as large-scale node monitoring nodes to monitor the status of client nodes. The monitoring implementation process can refer to the description of the corresponding part of the context embodiment.

[0134] Based on this, the follower node can process the heartbeat interaction with the client node and report the client node with a determined heartbeat timeout to the leader node, so that the leader node can determine whether the corresponding client node is offline based on the feedback results of the majority obeying the minority of follower nodes. Therefore, the leader node can analyze the content of the heartbeat response information received from different follower nodes during the term of office to determine whether the number of follower nodes reporting the same node to be isolated (i.e., the number of prompts mentioned above) exceeds the preset number. If so, it is determined that the node to be isolated is offline; otherwise, that is, if the number of prompts is less than or equal to the preset number, it can be determined that the corresponding node to be isolated is in normal operation, but it is not limited to this detection implementation method.

[0135] Step S67: The leader node configures a node isolation identifier for the node to be isolated that is in an offline state, and adds the node isolation identifier to the next heartbeat message;

[0136] It should be understood that, since the heartbeat interaction between the leader node and the follower nodes is not interrupted during the implementation process of the leader node determining the offline node, the leader node can continuously send heartbeat messages to the follower nodes at preset time intervals during its term of office. The content of the heartbeat messages sent at different times can be different, including but not limited to the message content described above. If the leader node has not obtained the log data sent by the client, there is currently no log data that needs to be persisted, and it is determined that each client node is currently in normal operation, the heartbeat message sent by the leader node can be an empty heartbeat packet used to maintain communication between the leader node and the follower nodes. This application does not provide detailed examples one by one here.

[0137] Therefore, the next heartbeat message in step S67 may be the heartbeat message that the leader node will send to the follower node after the leader node configures the node isolation identifier. Since it takes a certain amount of time for the leader node to determine whether the node to be isolated is offline, the next heartbeat message is often not the next adjacent heartbeat message of the heartbeat message sent by the leader node in step S61, that is, the time difference between the next heartbeat message sent by the leader node in step S67 and the heartbeat message sent by the leader node in step S61 can be n times the preset time interval of the heartbeat mechanism for the leader node, n can be an integer greater than 1, and the time difference is less than the term length of the leader node, and the specific length can be determined according to the circumstances.

[0138] Step S68, the leader node sends the next heartbeat message to the follower node;

[0139] Step S69: The follower node determines that the next heartbeat message includes a node isolation identifier, and prohibits sending a service message to the client node corresponding to the node identifier.

[0140] Regarding the implementation process of step S68 and step S69, please refer to the description of the corresponding part of the above embodiment, which will not be repeated in this embodiment. After receiving the heartbeat message from the leader node, the follower node can repeat the operation described above to complete the normal business operation in the consistency protocol.

[0141] Among them, the node isolation identifier can be used to identify the client node in the offline state. In this application, the contents of the above-mentioned node isolation identifier, pending recovery identifier, and node recovery identifier can be specific strings, numbers, etc. that represent the corresponding meanings. This application does not limit the content of these identifiers.

[0142] Reference Figure 7 , which is another optional example of the flow chart of the cluster processing method proposed in this application. The embodiment of this application can describe how to implement the election operation based on the consistency protocol for multiple nodes in the cluster and determine the implementation process of the leader node. This embodiment can be described from the follower node side, such as Figure 7 As shown, the method may include:

[0143] Step S71: Receive a voting request message sent by a candidate node; the voting request message includes the number of the first offline node and log submission parameters;

[0144] In actual applications, for a follower node in a cluster, if it does not receive a heartbeat message from the leader node within a preset time interval (such as the heartbeat period of the leader node), the roles of the follower node and the leader node in the cluster will change. The leader node can switch to a follower node, and the follower node can switch to a candidate node and then initiate the leader election process, that is, send a voting request message to each follower node in the current cluster, requesting the follower node to determine whether the candidate node can switch to the leader node in the next term of the cluster. It should be noted that the conditions for triggering a follower node to switch to a candidate node include but are not limited to the timeout of the heartbeat message sent by the leader node,

[0145] In the election mechanism proposed in this application, the status of the client node under the node's own view, such as whether it is offline, may have an adverse effect on the election result. Therefore, in order to select a leader node with stronger performance and better meet business processing requirements, the follower node can first detect whether the candidate node has the ability to become a leader node before voting for the current candidate node. To this end, when the candidate node sends a voting request message to each follower node in the cluster, in addition to obtaining the commit value of the local persistent log data, it can also obtain the first offline node number of the offline nodes (i.e., the nodes with heartbeat timeout) under the candidate node's own view. Based on the communication protocol, a voting request message containing at least the commit value and the first offline node number is obtained.

[0146] It can be seen that the above-mentioned first number of offline nodes can be the number of client nodes in offline state under the candidate node view, and the log submission parameter (that is, the latest commit value recorded locally by the candidate node) can represent the local persistent log data of the candidate node. Regarding the offline status detection process of the client node, please refer to the description of the corresponding part of the above embodiment, which will not be described in detail in this embodiment.

[0147] Step S72, determining the number of second offline nodes recorded locally and the local log submission parameters;

[0148] In an embodiment of the present application, the second number of offline nodes can be the number of client nodes that are offline in the follower node view. Regarding the implementation process of abnormality / fault monitoring of client nodes, reference can be made to the description of the corresponding part of the above embodiment. The follower node local log submission parameter (i.e., the commit value of the local record) can represent the local persistent log data of the follower node. This application does not elaborate on the process of obtaining the second number of offline nodes and the commit value.

[0149] Afterwards, the follower node can detect whether the candidate node meets the leader election conditions, that is, whether the candidate node has the ability to become a leader node, based on the number of second offline nodes, local log submission parameters, the number of first offline nodes, and log submission parameters. The follower node can then send a voting response message corresponding to the detection result to the candidate node, so that the candidate node can determine whether to receive the follower node's vote based on the content of the voting response message. This application does not restrict the content of the leader election conditions, including but not limited to the implementation method described in the steps below.

[0150] Step S73, detecting whether the number of the first offline nodes is greater than the number of the second offline nodes; if yes, proceeding to step S74; if not, executing step S78;

[0151] Step S74: Check whether the local log submission parameters are the same as the submission log parameters from the candidate node; if yes, proceed to step S75; if not, proceed to step S78;

[0152] Step S75: Determine that the candidate node does not meet the master election conditions, and send a voting response message to the candidate node to reject the vote;

[0153] After the follower node receives the voting request message sent by the candidate node, based on the consistency protocol, in addition to checking whether the follower node's own commit value is the same as the candidate node's commit value, that is, detecting whether the candidate node's persistent log data is the latest log, it can also determine whether the number of offline nodes under the follower node's own view is greater than the number of offline nodes under the candidate node's view. If so, it can be considered that the candidate node does not have the attributes to become a leader node, that is, the candidate node does not meet the master election conditions. The follower node will not vote for the candidate node, and can feed back a voting response message of the rejection vote to the candidate node. As needed, the voting results of the rejection vote can also be published to other nodes in the cluster.

[0154] Step S76: trigger the follower node to switch itself to a new candidate node, and the original candidate node to switch to a follower node;

[0155] Step S77: Send a voting request message to each follower node in the current cluster;

[0156] When a follower node refuses to vote for a candidate node, it can apply to become a candidate node itself. The application implementation process can be determined according to the consensus protocol. After that, a new candidate node, multiple follower nodes, and client nodes will appear in the cluster. After the follower node switches to a new candidate node, it can continue to send voting request messages to each follower node in the cluster according to the method described above. The message may include the number of the second offline node and the local log submission parameter (i.e., the local commit value) recorded by the new candidate node. The above election process is repeated until the node with the largest commit value and the number of offline nodes in its view is less than the number of offline nodes in the views of other follower nodes is determined to become the new leader node.

[0157] Step S78: Determine that the candidate node meets the master election conditions, and send a voting response message to the candidate node to confirm the vote;

[0158] Following the method described above, the follower node determines through the above-mentioned detection method that the current candidate node has the ability to be called a leader node, and can vote for the candidate node to become a leader node. The obtained voting response message containing the confirmation vote is sent to the candidate node, so that the candidate node can determine whether the number of confirmed votes exceeds half of the total number of follower nodes based on the voting response messages from each follower node, that is, whether more than half of the follower nodes agree that the candidate node can become the leader node. The candidate node can switch to the leader node.

[0159] Step S79: Receive the voting result message sent by the new leader node.

[0160] As can be seen from the above analysis, the new leader node can be a candidate node that switches when the number of confirmed votes from each follower node exceeds half of the total number of follower nodes based on voting response messages from each follower node. After the candidate node switches roles and becomes the new leader node, it can publish to each follower node and client node in the cluster. Afterwards, the business processing method described above can be used to meet the business processing requirements of the cluster and ensure data consistency between nodes. The implementation process of this application embodiment is not detailed here.

[0161] In summary, in the election process of multiple nodes in a cluster based on the consistency protocol, the status of the newly added client node in the cluster, whether it is offline, will be considered, which improves the election efficiency and ensures that the majority of service nodes can immediately execute business operations after the cluster selects a new leader node, thereby improving the performance of the cluster in processing business.

[0162] Reference Figure 8 , which is a flow chart of another optional example of the cluster processing method proposed in this application. This embodiment can be described from the client node side of the cluster. Regarding the node type of the cluster, please refer to the description of the corresponding part of the above implementation, such as Figure 8 As shown, the method may include:

[0163] Step S81, sending a heartbeat message to the leader node and the follower node;

[0164] Step S82, receiving a heartbeat response message from the leader node and / or the follower node;

[0165] Step S83, determining the number of received heartbeat response messages;

[0166] Step S84: Detecting that the number of received nodes is less than or equal to a preset number, determining that the client node itself is in an offline state; the preset number is half of the total number of the leader node and the follower nodes;

[0167] Step S85: Configure the corresponding node isolation flag to prohibit sending messages to the leader node and follower nodes;

[0168] As mentioned above, the client node newly defined based on the consistency protocol in the cluster of this application, as a lightweight data node, can realize the two-stage application of logs and self-network monitoring and diagnosis. During this implementation process, the client node can handle business according to the principle of minority obeys majority. If the number of heartbeat response messages (from the leader node and / or follower node) not received by the client node is greater than the above-mentioned preset number (i.e., half of the total number of leader nodes and follower nodes), the client node can be considered to be offline, and the node isolation identifier can be configured to achieve self-network isolation.

[0169] In the self-network monitoring method of the client node described above, if it is determined that the received heartbeat response messages are less than or equal to the preset number, it can be considered that the client node's own operating status is in normal operating status, and the client node can continue to send heartbeat messages to the follower node and the leader node according to the method described above.

[0170] Step S86: Detecting the recovery of the failure that caused the client node to enter the offline state, and configuring a pending recovery flag for the client node; the pending recovery flag is used to instruct the leader node to determine whether the client node has recovered from the offline state to a normal operating state;

[0171] Step S87: Add the to-be-recovered identifier to the next heartbeat message, and send the next heartbeat message to each follower node and the leader node.

[0172] Continuing from the above description, after the client node self-isolates itself based on the network monitoring results, it can continue to monitor whether the fault of the client node is restored. If the fault is restored, in order to restore the communication between the client node and the follower node and the leader node, the client node can directly delete the node isolation identifier of the client node, configure the node recovery identifier, and add it to the next heartbeat message to be sent in the new heartbeat cycle, that is, the heartbeat message to be sent after the client node returns to normal. Regarding the relationship between the next heartbeat message and the heartbeat message sent in the above step S81, please refer to the description of the corresponding part of the above method.

[0173] In some other embodiments, as described in the steps above, in order to improve the reliability of fault recovery detection of client nodes isolated by the network, after the client node detects that its own fault has been recovered, it can send a heartbeat message carrying a to-be-recovered identifier to the leader node and the follower node. In this way, the leader node can determine that more than half of the follower nodes report the recovery of the same client node fault according to the method described above, and can send a heartbeat message carrying a node recovery identifier to the follower node and the client node, so that the client node can delete the corresponding node isolation identifier and resume normal business communication.

[0174] Reference Figure 9 , is a flow chart of another optional example of the cluster processing method proposed in this application. In the embodiment of this application, it is possible to describe how to implement the role attribute change processing process based on the consistency protocol, such as describing an optional implementation method of the client node failure recovery process in the cluster, such as Figure 9 As shown, the cluster processing method executed by the client nodes in the cluster may further include:

[0175] Step S91: When it is determined that the client node is offline, trigger the client node to switch to a new follower node;

[0176] Step S92: Receive a heartbeat message sent by the leader node; the heartbeat message includes the log data to be replicated and the log submission parameters of the leader node's local persistent log data;

[0177] Step S93, performing persistence processing on the log data to be copied, executing operations corresponding to the persistent log data, and updating the local log submission parameters of the local persistent log data;

[0178] Step S94: Determine that the updated local log submission parameter is the same as the log submission parameter from the leader node, maintain the new follower node unchanged or trigger the new follower node to recover as a client node.

[0179] In the embodiment of this application, combined with Figure 10 The diagram shows a scenario in which a client node is converted to a follower node for fault recovery after a failure. Based on the consistency protocol, each follower node in the cluster needs to obtain the log data to be replicated from the leader node, that is, the latest persisted log data of the leader node, perform persistence processing on the log data received from the leader node, and send the log data to the client node. The client node is responsible for the log apply process and performs the operations corresponding to the log data.

[0180] Therefore, the process of applying the logs by the client node and the process of persisting the log data from the leader node by the follower node can be executed on different links. As long as the client node can obtain valid log data, it can continue to apply the logs. Based on this, in an embodiment of the present application, according to any of the methods described above, after determining that the client node fails and enters an offline state, the client node can be triggered to convert into a follower node, that is, the role attribute of the client node is changed so that it retains the log apply function of the client node itself, and can also obtain the log data to be recovered from the leader node. Since the log data acquisition process and the log apply process can be executed on different links, the client node can be recovered without affecting the current business, and it can also be applied to node expansion scenarios.

[0181] According to the above-mentioned log recovery processing, it is determined that the updated local log submission parameters are the same as the log submission parameters from the leader node, indicating that the log of the follower node is consistent with that of the leader node, and its fault has been recovered. The follower node (that is, converted from the client node) can be used as a follower node of the cluster, and its functions can refer to the content described above from the follower node side; alternatively, the follower node can also be restored to be used as a client node. This application does not impose any restrictions on this, and it can be determined according to the circumstances.

[0182] In some other embodiments, Figure 10As shown, after a client node fails, in order to restore log data and keep it consistent with the log of the leader node, when the heartbeat message sent by the current leader node of the cluster times out, the client node can also be converted into a leader node and used. By interacting with the follower nodes through heartbeats, the log consistency of each node is guaranteed. The implementation process is not described in detail in this application.

[0183] Reference Figure 11 , which is a flow chart of another optional example of the cluster processing method proposed in this application, the embodiment of this application can describe the creation of a new client node defined based on the consistency protocol in the cluster, and the implementation process of synchronizing cluster log data, such as Figure 11 As shown, the cluster processing method executed by the client node of the cluster may further include:

[0184] Step S111, sending an access request message to the leader node;

[0185] Step S112, receiving an access confirmation message sent by the leader node;

[0186] In an embodiment of the present application, the newly defined client node based on the consistency protocol is a lightweight role that does not require local persistence of log data. In this way, the leader node confirms that the local log data has been submitted, and only needs to obtain whether the log data has been successfully persisted by each follower node, thereby improving business response efficiency and fault recovery efficiency.

[0187] If any node wishes to join the cluster as a client node, it can send an access request message to the leader node to request the leader node's various view information, cluster persistent log data, and other information, including the latest leader node in the cluster, its term number, and which follower nodes it belongs to. The leader node can determine whether it can serve as a client node in the cluster based on information such as node configuration parameters. The leader node's administrator can also determine whether the cluster joins the client node. If approved, the leader node can send a confirmation access message to the node to create the client node.

[0188] The access confirmation message may include log submission parameters of the current cluster's persistent log data and the view information of the leader node. This application does not limit the content of the access confirmation message. The view information can represent the status of each node in the current cluster.

[0189] Step S113, determining an auxiliary node for the client node in the cluster based on at least the status of each node; the auxiliary node is a leader node or a follower node;

[0190] Step S114, sending a heartbeat message to the follower node and the leader node in the cluster; the heartbeat message sent to the auxiliary node includes log acquisition request information, and the log acquisition request information is determined according to the log submission parameter of the persisted log data;

[0191] In some embodiments, based on the cluster's consistency protocol, when each client node is created, it needs to obtain persistent log data from the leader node and follower node, and perform operations corresponding to the log data to ensure log consistency of each node in the cluster. Therefore, after the client node is created, it can send heartbeat messages to each follower node and leader node of the cluster to inform the corresponding node that the cluster has created the client node and maintain communication between the two corresponding nodes. After the client node is created, it needs to first obtain the cluster's persistent log data. Therefore, the heartbeat message sent by the client node can include log acquisition request information.

[0192] Optionally, in order to reduce the pressure of a large number of client nodes obtaining log data from the same node, a follower node or leader node can be determined as the corresponding auxiliary node for different client nodes, so that the client node sends a heartbeat message containing log acquisition request information to the auxiliary node. The heartbeat messages sent to other follower nodes and leader nodes (i.e., non-auxiliary nodes) do not need to include the log acquisition request information.

[0193] Step S115, receiving the heartbeat response message sent by the follower node and the leader node, and executing the operation corresponding to the requested log data; the requested log data is included in the heartbeat response message from the auxiliary node, and is obtained by the auxiliary node according to the log acquisition request information.

[0194] Following the above analysis, after the follower node and the leader node receive the heartbeat message from the client node, they can provide feedback based on the content contained in the heartbeat message. For the heartbeat message received by the auxiliary node, which contains log acquisition request information, the auxiliary node can query the corresponding log data based on this, such as the log data within the corresponding request interval, include it in the heartbeat response message, and feedback it to the client node, so that the client node executes the operation corresponding to the log data fed back by the auxiliary node, that is, executes the operation corresponding to the requested log data contained in the heartbeat response message. The client node can execute the log apply at the business layer, and the implementation process will not be described in detail.

[0195] During the subsequent log consistency processing between the nodes of the cluster, the client node can still send a heartbeat message containing log acquisition request information to the auxiliary node according to the method described above, so that the client node can obtain the latest log data of the cluster. Of course, in the case where the auxiliary node of the client node is not specified, a heartbeat message containing log acquisition request information can be sent to the leader node and / or any one or more follower nodes to obtain the latest log data in a timely manner and execute log apply. This application does not limit the interaction implementation method between the leader node and follower nodes of the cluster and the client node, which can be determined according to the situation.

[0196] Reference Figure 12 , is a schematic diagram of an optional example of the cluster processing device proposed in this application, combined with the above Figure 1 and Figure 3 The cluster structure shown in FIG. 1 may include multiple nodes. The types of these multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on the consensus protocol. This embodiment can be described from the perspective of follower nodes, such as Figure 12 As shown, the device may include:

[0197] A first receiving module 121 is configured to receive a heartbeat message sent by the leader node;

[0198] A node-to-be-isolated determination module 122 is configured to determine, based on heartbeat detection information for each client node, that the client node with a heartbeat timeout is a node to be isolated; the heartbeat detection information can be updated based on a heartbeat message sent by the client node;

[0199] The first sending module 123 is configured to send a heartbeat response message to the leader node; the heartbeat response message includes at least isolation prompt information for the node to be isolated, so as to prompt the leader node to determine whether the node to be isolated is in an offline state.

[0200] Optionally, the device may further include:

[0201] a service message prohibition sending module, configured to determine that the heartbeat message from the leader node includes a node isolation identifier, and prohibit sending service messages to the client node corresponding to the node isolation identifier; the node isolation identifier is used to identify a client node in an offline state; and / or,

[0202] A second sending module is configured to determine that the heartbeat message from the client node includes a to-be-recovered identifier, and add status recovery information for the client node to a heartbeat response message and send the message to the leader node;

[0203] The third sending module is used to determine that the heartbeat message from the leading node includes a node recovery identifier, delete the node isolation identifier corresponding to the client node, and resume sending business messages to the client node corresponding to the node recovery identifier; the node recovery identifier is used to identify the client node that has recovered from the offline state to the normal operating state.

[0204] Optionally, the device may further include:

[0205] A voting request message receiving module, configured to receive a voting request message sent by the candidate node; the voting request message includes a first offline node number and a log submission parameter; the first offline node number is the number of client nodes in an offline state in the candidate node view; the log submission parameter can represent the local persistent log data of the candidate node;

[0206] A first determination module is configured to determine a locally recorded second offline node number and a local log submission parameter; the second offline node number is the number of client nodes in an offline state in the follower node view, and the local log submission parameter can represent the local persistent log data of the follower node;

[0207] a first detection module, configured to detect whether the candidate node meets a master election condition based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the log submission parameter;

[0208] The voting response message sending module is used to send a voting response message corresponding to the detection result to the candidate node.

[0209] In some embodiments, the first detection module may include:

[0210] a first determining unit, configured to detect that the number of the first offline nodes is greater than the number of the second offline nodes, and that the local log submission parameter is the same as the submitted log parameter, and determine that the candidate node does not meet the master election condition;

[0211] A first sending unit, configured to send a voting response message to the candidate node indicating that the candidate node has rejected the vote;

[0212] A node type switching unit, configured to trigger the candidate node to switch to a follower node, and the follower node itself to switch to a new candidate node;

[0213] The second sending unit is configured to send a voting request message to each follower node in the current cluster; the voting request message includes the second offline node number and the local log submission parameter recorded by the new candidate node.

[0214] The second determining unit is configured to detect that the number of the first offline nodes is less than or equal to the number of the second offline nodes, and / or that the local log submission parameter is different from the submission log parameter, and determine that the candidate node meets the master election condition.

[0215] A third sending unit is configured to send a voting response message to the candidate node to confirm the vote;

[0216] The voting result message receiving unit is used to receive the voting result message sent by the new leader node; the new leader node is switched when the candidate node determines that the number of confirmed votes exceeds half of the total number of follower nodes based on the voting response messages from each follower node.

[0217] Reference Figure 13 , is a structural diagram of another optional example of the cluster processing device proposed in this application. The cluster may include multiple nodes. The types of these multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on the consistency protocol. This embodiment can be described from the perspective of the leader node side of the cluster. Figure 13 As shown, the device may include:

[0218] A first sending module 131 is configured to send a heartbeat message to each of the following nodes;

[0219] A first receiving module 132 is configured to receive a heartbeat response message sent by the follower node; the heartbeat response message includes at least isolation prompt information for the node to be isolated, and the node to be isolated is the client node whose heartbeat timeout is determined by the follower node based on the heartbeat detection information for each client node;

[0220] The first determining module 133 is configured to determine whether the node to be isolated is in an offline state according to the isolation prompt information from different following nodes.

[0221] Optionally, the first determining module 133 may include:

[0222] a prompt number obtaining unit, configured to obtain the number of prompts of the isolation prompt information for the same node to be isolated;

[0223] An offline state determining unit, configured to detect that the number of prompts is greater than a preset number and determine that the corresponding node to be isolated is in an offline state; the preset number is half of the total number of the leader node and the follower nodes;

[0224] The normal operating state determining unit is configured to detect that the number of prompts is less than or equal to the preset number, and determine that the corresponding node to be isolated is in a normal operating state.

[0225] Optionally, the above device may further include:

[0226] The node isolation identifier configuration module is used to configure the node isolation identifier for the node to be isolated in the offline state, add the node isolation identifier to the next heartbeat message, and send the next heartbeat message to each follower node.

[0227] Reference Figure 14 , is a structural diagram of another optional example of the cluster processing device proposed in this application. The cluster may include multiple nodes. The types of these multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on the consistency protocol. This embodiment can be described from the perspective of the client node side of the cluster. Figure 14 As shown, the device may include:

[0228] A first sending module 141 is configured to send a heartbeat message to the leader node and the follower node;

[0229] A first receiving module 142 is configured to receive a heartbeat response message from the leader node and / or the follower node;

[0230] A reception quantity determination module 143 is configured to determine the number of received heartbeat response messages;

[0231] The offline state determination module 144 is configured to detect that the received number is less than or equal to a preset number and determine that the client node itself is in an offline state, where the preset number is half of the total number of the leader node and the follower nodes.

[0232] Optionally, the device may further include:

[0233] A node isolation flag configuration module, configured to configure a corresponding node isolation flag when it is determined that the client node itself is in an offline state, and prohibit sending messages to the leader node and the follower node;

[0234] a pending recovery flag configuration module, configured to detect the failure recovery that caused the client node to enter an offline state and configure a pending recovery flag for the client node; the pending recovery flag is used to instruct the leader node to determine whether the client node has recovered from the offline state to a normal operating state;

[0235] The second sending module is configured to add the to-be-recovered identifier to a next heartbeat message, and send the next heartbeat message to each of the follower nodes and the leader node.

[0236] Optionally, the above device may further include:

[0237] A second receiving module is configured to trigger the client node to switch to a new follower node when it is determined that the client node itself is in an offline state, and receive a heartbeat message sent by the leader node; the heartbeat message includes the log data to be replicated and the log submission parameters of the local persistent log data of the leader node;

[0238] A persistence processing module, configured to perform persistence processing on the log data to be copied, execute operations corresponding to the persistent log data, and update local log submission parameters of the local persistent log data;

[0239] The node processing module is used to determine that the updated local log submission parameter is the same as the log submission parameter from the leader node, maintain the new follower node unchanged or trigger the new follower node to recover as the client node.

[0240] Optionally, the above device may further include:

[0241] An access request message sending module, configured to send an access request message to the leader node;

[0242] A confirmation access message receiving module is configured to receive a confirmation access message sent by the leader node; the confirmation access message includes log submission parameters of the current persistent log data of the cluster and view information of the leader node; the view information can represent the current status of each node in the cluster;

[0243] A third sending module is configured to send a heartbeat message to the follower node and the leader node in the cluster; the heartbeat message includes log acquisition request information, and the log acquisition request information is determined according to the log submission parameter of the persisted log data;

[0244] A third receiving module is used to receive heartbeat response messages sent by the follower node and the leader node;

[0245] The execution module is used to execute the operation corresponding to the requested log data contained in the heartbeat response message.

[0246] Optionally, the above device may further include:

[0247] an auxiliary node determination module, configured to determine an auxiliary node for the client node itself in the cluster based at least on the states of the nodes; the auxiliary node being the leader node or the follower node;

[0248] The heartbeat message sent to the auxiliary node includes log acquisition request information, and the execution module may include:

[0249] The execution unit is used to execute the operation corresponding to the requested log data; the requested log data is included in the heartbeat response message from the auxiliary node and is obtained by the auxiliary node according to the log acquisition request information.

[0250] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory of the corresponding node as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. Regarding the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0251] The present application also provides a computer-readable storage medium on which a computer program can be stored. The computer program can be called and loaded by a processor to implement the various steps of the cluster processing method described on the corresponding type node side of the above embodiment, wherein the cluster includes multiple nodes, and the types of the multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on the consistency protocol.

[0252] Reference Figure 15 , is a schematic diagram of an optional example of a computer device applicable to the cluster processing method proposed in this application. The computer device can be a leader node, a follower node, or a client node defined based on a consensus protocol in a cluster, such as Figure 15 As shown, the computer device may include but is not limited to: a communication module 151, a memory 152 and a processor 153, wherein:

[0253] The communication module 151 may include a communication module that can use a wireless communication network to realize data interaction, such as a WIFI module, a 5G / 6G (fifth-generation mobile communication network / sixth-generation mobile communication network) module, a GPRS module, etc., to realize communication with other nodes in the cluster; the communication module 151 may also include a communication interface for realizing data interaction between internal components of the computer device, such as a USB interface, a serial / parallel port, an I / O port, etc. This application does not limit the specific content contained in the communication module 151.

[0254] The memory 152 can be used to store a program for implementing the cluster processing method described in the method embodiment of the corresponding node side above; the processor 153 can load and execute the program stored in the memory to implement the various steps of the cluster processing method described in the method embodiment of the corresponding node side above. The specific implementation process can refer to the description of the corresponding part of the above embodiment and will not be repeated here.

[0255] In the embodiment of the present application, the memory 152 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. The processor 153 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices.

[0256] It should be understood that Figure 15 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiment of the present application. In actual applications, the computer device may include Figure 15 More components shown, or combinations of certain components, can be determined based on the device types of different types of nodes, and this application does not list them one by one here.

[0257] Finally, it should be noted that, in the above embodiments, unless the context clearly indicates an exception, the terms "a," "an," "an," and / or "the" do not specifically refer to the singular but also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements explicitly identified, and these steps and elements do not constitute an exclusive list. A method or device may also include other steps or elements. The phrase "comprises a..." does not preclude the presence of other identical elements in the process, method, product, or device that includes the elements.

[0258] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0259] Terms such as "first" and "second" used in this application are used for descriptive purposes only to distinguish one operation, unit, or module from another operation, unit, or module, and do not necessarily require or imply any actual relationship or order between these units, operations, or modules. They should not be understood as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of such features.

[0260] In addition, the various embodiments in this specification are described in a progressive or parallel manner. Each embodiment focuses on the differences from other embodiments, and reference can be made to the common and similar parts between the various embodiments. For the devices, clusters, and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For relevant parts, refer to the description of the methods.

[0261] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cluster processing method, wherein the cluster includes multiple nodes, wherein the types of the multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on an improved consensus protocol. The client nodes are data nodes that do not locally persist log data. When the number of nodes in the cluster is fixed, adding a newly defined client node reduces the number of follower nodes in the cluster. The cluster processing method is applied to the follower nodes, comprising: Receive a heartbeat message sent by the leader node; Determining, based on the heartbeat detection information for each of the client nodes, the client node with a heartbeat timeout as a node to be isolated; The heartbeat detection information is updated according to the heartbeat message sent by the client node; Sending a heartbeat response message to the leader node; The heartbeat response message includes at least isolation prompt information for the node to be isolated, so as to prompt the leader node to determine whether the node to be isolated is in an offline state; A heartbeat message including a detection result of whether the node to be isolated is in an offline state is received from the leader node, and the client node in the offline state is isolated.

2. The method according to claim 1, further comprising: Determining that the heartbeat message from the leader node includes a node isolation identifier, and prohibiting the sending of service messages to the client node corresponding to the node isolation identifier; The node isolation identifier is used to identify a client node that is in an offline state; and / or, Determining that the heartbeat message from the client node includes a to-be-recovered identifier, adding status recovery information for the client node to a heartbeat response message and sending it to the leader node; Determine that the heartbeat message from the leader node includes a node recovery identifier, delete the node isolation identifier corresponding to the client node, and resume sending business messages to the client node corresponding to the node recovery identifier; the node recovery identifier is used to identify the client node that has recovered from the offline state to the normal operating state.

3. The method according to claim 1, further comprising: Receiving a voting request message sent by the candidate node; The voting request message includes the number of the first offline node and log submission parameters; The first number of offline nodes is the number of client nodes in an offline state in the candidate node view, and the log submission parameter represents the local persistent log data of the candidate node; Determine the number of second offline nodes for local records and local log submission parameters; The second number of offline nodes is the number of client nodes in an offline state in the follower node view, and the local log submission parameter represents the local persistent log data of the follower node; Detecting whether the candidate node meets a master election condition based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the log submission parameter; Sending a voting response message corresponding to the detection result to the candidate node.

4. The method according to claim 3, wherein detecting whether the candidate node meets the master election condition based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the log submission parameter, and sending a voting response message corresponding to the detection result to the candidate node comprises: Detecting that the number of the first offline nodes is greater than the number of the second offline nodes, and the local log submission parameter is the same as the log submission parameter, and determining that the candidate node does not meet the master election condition; Sending a voting response message to the candidate node to reject the vote, triggering the candidate node to switch to a follower node, and the follower node itself switches to a new candidate node; Sending a voting request message to each follower node in the current cluster; the voting request message includes the second offline node number and the local log submission parameter recorded by the new candidate node.

5. The method according to claim 3, wherein detecting whether the candidate node meets the master election condition based on the second number of offline nodes, the local log submission parameter, the first number of offline nodes, and the log submission parameter, and sending a voting response message corresponding to the detection result to the candidate node comprises: Detecting that the number of the first offline nodes is less than or equal to the number of the second offline nodes, and / or the local log submission parameter is different from the log submission parameter, and determining that the candidate node meets the master election condition; Sending a voting response message to the candidate node to confirm the vote; Receive a voting result message sent by a new leader node; the new leader node is obtained by switching when the candidate node determines that the number of confirmed votes exceeds half of the total number of follower nodes based on the voting response messages from each follower node.

6. A cluster processing method, wherein the cluster includes multiple nodes, wherein the types of the multiple nodes include a leader node, a follower node, a candidate node, and a client node defined based on an improved consistency protocol. The client node is a data node that does not locally persist log data. When the number of nodes in the cluster is fixed, adding a newly defined client node reduces the number of follower nodes in the cluster. The cluster processing method is applied to the leader node, comprising: Sending a heartbeat message to each of the following nodes; Receive a heartbeat response message sent by the follower node; The heartbeat response message includes at least isolation prompt information for the node to be isolated, and the node to be isolated is the client node with a heartbeat timeout determined by the follower node based on the heartbeat detection information for each client node; Determining whether the node to be isolated is in an offline state according to the isolation prompt information from different follower nodes; The detection result of whether the node to be isolated is in an offline state is included in the heartbeat message to be sent, and the heartbeat message is sent to the follower node to promptly notify the follower node to isolate the client node in the offline state.

7. The method according to claim 6, wherein determining whether the node to be isolated is in an offline state based on the isolation prompt information from different follower nodes comprises: Obtaining the number of prompts of the isolation prompt information for the same node to be isolated; Detecting that the number of prompts is greater than a preset number, determining that the corresponding node to be isolated is in an offline state; The preset number is half of the total number of the leader node and the follower node; It is detected that the number of prompts is less than or equal to the preset number, and it is determined that the corresponding node to be isolated is in a normal operating state.

8. The method according to claim 6 or 7, further comprising: A node isolation identifier for the node to be isolated that is in an offline state is configured, the node isolation identifier is added to a next heartbeat message, and the next heartbeat message is sent to each of the follower nodes.

9. A cluster processing method, wherein the cluster includes multiple nodes, wherein the types of the multiple nodes include leader nodes, follower nodes, candidate nodes, and client nodes defined based on an improved consensus protocol. The client nodes are data nodes that do not locally persist log data. When the number of nodes in the cluster is fixed, adding a newly defined client node reduces the number of follower nodes in the cluster. The cluster processing method is applied to the client node, comprising: Sending a heartbeat message to the leader node and the follower node; receiving a heartbeat response message from the leader node and / or the follower node; Determining the number of received heartbeat response messages; Detecting that the number of received nodes is less than or equal to a preset number, determining that the client node itself is in an offline state, wherein the preset number is half of the total number of the leader node and the follower nodes; In the case where it is determined that the client node itself is in an offline state, a corresponding node isolation flag is configured to prohibit sending messages to the leader node and the follower node.

10. The method according to claim 9, further comprising: Detecting a failure recovery that caused the client node to enter an offline state, and configuring a pending recovery flag for the client node; The pending recovery flag is used to instruct the leader node to determine whether the client node has recovered from an offline state to a normal operating state; The to-be-recovered identifier is added to a next heartbeat message, and the next heartbeat message is sent to each of the follower nodes and the leader node.

11. The method according to claim 9 or 10, further comprising: In the case of determining that the client node itself is in an offline state, triggering the client node to switch to a new follower node and receive a heartbeat message sent by the leader node; The heartbeat message includes the log data to be replicated and the log submission parameters of the local persistent log data of the leader node; Performing persistence processing on the log data to be copied, executing operations corresponding to the persistent log data, and updating local log submission parameters of the local persistent log data; Determine that the updated local log submission parameter is the same as the log submission parameter from the leader node, maintain the new follower node unchanged or trigger the new follower node to recover as the client node.

12. The method according to claim 9 or 10, further comprising: Sending an access request message to the leader node; receiving an access confirmation message sent by the leader node; The access confirmation message includes the log submission parameters of the current persistent log data of the cluster and the view information of the leader node; the view information represents the current status of each node in the cluster; Sending heartbeat messages to the follower nodes and the leader node in the cluster; The heartbeat message includes log acquisition request information, and the log acquisition request information is determined according to the log submission parameter of the persisted log data; Receive the heartbeat response message sent by the follower node and the leader node, and execute the operation corresponding to the requested log data contained in the heartbeat response message.

13. The method according to claim 12, further comprising: Determining a secondary node in the cluster for the client node itself based at least on the states of the nodes; The auxiliary node is the leader node or the follower node; The heartbeat message contains log acquisition request information, including: The heartbeat message sent to the auxiliary node includes log acquisition request information; The executing the operation corresponding to the requested log data included in the heartbeat response message includes: Execute the operation corresponding to the requested log data; the requested log data is included in the heartbeat response message from the auxiliary node and is obtained by the auxiliary node according to the log acquisition request information.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the cluster processing method according to any one of claims 1 to 13; in, The cluster includes multiple nodes, and the types of the multiple nodes include leader nodes, follower nodes, candidate nodes and client nodes defined based on an improved consistency protocol. The client nodes are data nodes that do not need to persist log data locally. When the number of nodes included in the cluster is determined, adding newly defined client nodes reduces the number of follower nodes included in the cluster.

Images