Cluster management method and related apparatus

Abstract

The application discloses a cluster management method and related devices, which are applied to a cluster, the cluster comprising at least one master node and at least one slave node, the at least one slave node comprising a first slave node; the method comprising: when the first slave node detects that the communication with the master node is abnormal, the first slave node sends a first message to the master node, the first message being used for requesting the master node to indicate how to handle the communication abnormality; if the first slave node receives a handling strategy of the master node about the first message, the first slave node executes a first operation according to the handling strategy. By the master node flexibly and accurately providing a corresponding handling strategy for a slave node that may cause a brain split according to the information of the nodes in the cluster, the brain split of the cluster can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a cluster management method and related device. BACKGROUND

[0002] Cluster refers to a group of independent computers, which are connected through high-speed network to form a computer system that highly cooperates to complete computing tasks. Each cluster node is an independent server that runs its own process. High-availability cluster technology is widely used in the field of storage technology. When a node in the high-availability cluster fails, the services, resources and IP of the node can be transferred to another node, thereby meeting the continuity of the business. In order to ensure the normal operation of the high-availability cluster, each node in the cluster needs to have normal activity when providing external services, thereby ensuring stable services to the outside.

[0003] In the process of providing services by the cluster, due to changes in the network environment, the node may fail to detect the existence of the other nodes in the cluster through heartbeat information or lease information due to various network failures. When a node in the cluster fails, the other nodes in the cluster and the failed node cannot detect the existence of each other, and both sides determine that the other side is inactive. These nodes may access the same resource, thereby causing mutual trampling and covering of data, resulting in data damage or data incompleteness. This phenomenon is called split brain.

[0004] Currently, in the general cluster management method, the nodes in the cluster confirm the survival of each other through the heartbeat mechanism or the lease mechanism. If the heartbeat is lost or the lease is invalid, it is determined that the other node is dead, and arbitration is introduced to prevent split brain processing. After the arbitration node or arbitration device determines the inactive node, the business of the inactive node is interrupted, and the access permission of the corresponding resource is allocated.

[0005] However, the management network and the business network in the cluster system are isolated from each other. When a node is inactive in the management network, but the business function in the business network can continue to develop, the introduction of arbitration processing will forcibly interrupt the business. This processing method is fixed and is easy to cause negative impact on the business. SUMMARY

[0006] The present application provides a cluster management method and related device. The survival state of each slave node is managed by the master node, and when the communication between the slave node and the master node is abnormal, the slave node is flexibly and accurately provided with a processing strategy according to the business situation of the slave node, which can effectively avoid the occurrence of split brain in the cluster, and can also reduce the negative impact caused by interrupting the business of the slave node.

[0007] The first aspect of the present application provides a cluster management method, applied to a cluster, the cluster comprising at least one master node and at least one slave node, the at least one slave node comprising a first slave node; the method comprising:

[0008] When the first slave node detects that the communication between itself and the master node is abnormal, the first slave node sends a first message to the master node, the first message being used to request the master node to indicate how to handle the communication abnormality;

[0009] If the first slave node receives a handling strategy of the master node about the first message, the first slave node performs a first operation according to the handling strategy.

[0010] In the present application, when the slave node of the cluster detects that the communication between itself and the master node is abnormal, the slave node first sends a first message to the master node, and then performs a corresponding handling operation according to the handling strategy of the feedback of the master node to the first message; by providing the handling strategy for the slave node that may cause a brain split flexibly and accurately according to the information of the nodes in the cluster by the master node, the brain split of the cluster can be avoided, and the negative influence caused by interrupting the service of the slave node can be reduced.

[0011] In a possible implementation, the at least one slave node further comprises a second slave node; after the first slave node sends the first message to the master node, the method further comprises: if the handling strategy is not received within a second preset time after the first message is sent, the first slave node sends a second message to the second slave node, the second message being used to request the second slave node to forward the first message to the master node; if a feedback message of the second slave node about the second message is received, the first slave node performs a second operation according to the feedback message.

[0012] In the present application, when the first slave node determines that the alive message between itself and the master node is unreachable and the first message is unreachable, the first slave node communicates with the master node through the second slave node, which can reduce the misjudgment of the master node or the first slave node caused by the communication abnormality, so as to better maintain the continuity of the service running on the first slave node.

[0013] In a possible implementation, after the first slave node sends the second message to the second slave node, the method further comprises: if the feedback message is not received within a third preset time after the second message is sent, the first slave node performs a self-restart operation.

[0014] In the present application, the first slave node can determine whether it is in the communication failure state by whether the feedback message is received; after determining that it is in the communication failure state, the first slave node can interrupt the service and release the corresponding resources as soon as possible by performing the preset self-restart operation, so that other slave nodes in the cluster can continue to develop the service running on the first slave node, and reduce the negative impact on the service.

[0015] In a possible implementation, the feedback message includes a second feedback message, and the second feedback message is used to indicate that the second slave node does not receive the processing strategy; the first slave node performs a second operation according to the feedback message, including: the first slave node sends a re-election request to the second slave node according to the second feedback message.

[0016] In the present application, when the first slave node confirms that the master node is down or in the communication failure state according to the second feedback message of the second slave node, the first slave node can send a re-election request to each second slave node in the cluster, so as to recover the survival management of the cluster to each slave node as soon as possible, and avoid the brain split of the cluster.

[0017] In a possible implementation, before the first slave node sends the first message to the master node, the method further includes: the first slave node acquires a first time of receiving a survival message and a second time of receiving a management message most recently, the survival message and the management message being sent by the master node; if a difference between the first time and a current time is greater than a first threshold value, and a difference between the second time and the current time is greater than a second threshold value, the first slave node determines that the communication between the first slave node and the master node is abnormal.

[0018] In the present application, by adding the management message as a way to judge the survival of the slave node parallel to the survival message, the cluster stability and fault tolerance can be improved, the misjudgment of the master node can be reduced, and the continuity of the running service can be better maintained.

[0019] In a possible implementation, before the first slave node sends the first message to the master node, the method further includes: the first slave node acquires a first time of receiving a survival message most recently, the survival message being sent by the master node; if a difference between the first time and a current time is greater than a first threshold value, the first slave node determines that the communication between the first slave node and the master node is abnormal.

[0020] In the present application, the first slave node can actively detect whether the communication state with the master node is abnormal according to the first time recorded by the first slave node, so that the problem can be found faster when the communication state is abnormal, and the subsequent processing process can be triggered.

[0021] The second aspect of the present application provides another cluster management method, applied to a cluster, the cluster including at least one master node and at least one slave node; the method includes:

[0022] The master node receives a first message sent by the slave node, the first message being used to request the master node to indicate how to handle the communication exception of the slave node with the master node;

[0023] The master node determines a handling strategy according to the service information of the slave node;

[0024] The master node sends the handling strategy to the slave node, so that the slave node performs a first operation according to the handling strategy.

[0025] The third aspect of the present application provides another cluster management method, applied to a cluster, the cluster including at least one master node and at least one slave node, the at least one slave node including a third slave node; the method includes:

[0026] The master node determines the third slave node with a communication exception with the master node itself;

[0027] The master node performs a third operation according to the number of the third slave node.

[0028] In the present application, the master node can find and handle the third slave node with a communication exception more quickly by determining the third slave node with a communication state exception with itself and performing a third operation according to the number of the third slave node, thereby maintaining the stability of the cluster and better maintaining the continuity and security of the service in the cluster.

[0029] In a possible implementation, the master node performs a third operation according to the number of the third slave node, including:

[0030] When the number of the third slave node is less than the number of the at least one slave node, and the first message sent by the third slave node is not received within a fifth preset time after the third slave node is determined, the master node determines a handling strategy according to the service information of the third slave node and sends the handling strategy to the third slave node; wherein the first message is used to request the master node to indicate how to handle the communication exception; when the number of the third slave node is equal to the number of the at least one slave node, the master node performs a self-restart operation.

[0031] In the present application, the master node determines whether it is in a communication exception state by the number of the third slave node; when it is determined that it is in a communication exception state, the master node performs a preset self-restart operation, which can save system resources for determining a handling strategy.

[0032] The fourth aspect of the present application provides a first computing device, the first computing device and a second computing device forming a cluster; the first computing device includes a processor and a transmitter;

[0033] The transmitter is configured to send, to the second computing device, a first message for requesting the second computing device to indicate how to handle the communication anomaly when the first computing device detects that the communication with the second computing device is abnormal;

[0034] The processor is configured to perform a first operation according to the processing strategy when the second computing device receives the processing strategy of the first message.

[0035] The fifth aspect of the present application provides a second computing device, the first computing device and the second computing device form a cluster; the second computing device comprises a processor, a receiver and a transmitter;

[0036] The receiver is configured to receive a first message sent by the first computing device, the first message being used for requesting the second computing device to indicate how to handle a communication anomaly between the second computing device and the first computing device;

[0037] The processor is configured to determine a processing strategy according to the service information of the first computing device;

[0038] The transmitter is configured to send the processing strategy to the first computing device, so that the first computing device performs a first operation according to the processing strategy.

[0039] The sixth aspect of the present application provides another second computing device, the second computing device and the first computing device form a cluster; the second computing device comprises a processor;

[0040] The processor is configured to determine a first computing device which has a communication anomaly with the second computing device itself;

[0041] The processor is further configured to perform a third operation according to the number of the first computing devices.

[0042] It should be understood that the implementation and beneficial effects of the above aspects can be referred to each other. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a distributed system architecture provided by an embodiment of the present application;

[0044] Figure 2 is an implementation environment schematic diagram related to a cluster management method provided by an embodiment of the present application;

[0045] Figure 3 is another cluster system framework schematic diagram provided by an embodiment of the present application;

[0046] Figure 4 is a flow schematic diagram of a cluster management method provided by an embodiment of the present application;

[0047] Figure 5is a cluster-in-cluster information pushing schematic diagram provided by an embodiment of the present application;

[0048] Figure 6 is a cluster-in-message receiving time updating schematic diagram provided by an embodiment of the present application;

[0049] Figure 7 is a flow schematic diagram of another cluster management method provided by an embodiment of the present application;

[0050] Figure 8 is a flow schematic diagram of another cluster management method provided by an embodiment of the present application;

[0051] Figure 9 is a flow schematic diagram of another cluster management method provided by an embodiment of the present application;

[0052] Figure 10 is a flow schematic diagram of another cluster management method provided by an embodiment of the present application;

[0053] Figure 11 is a flow schematic diagram of another cluster management method provided by an embodiment of the present application;

[0054] Figure 12 is a structure schematic diagram of a first computing device provided by an embodiment of the present application;

[0055] Figure 13 is a structure schematic diagram of a second computing device provided by an embodiment of the present application;

[0056] Figure 14 is a structure schematic diagram of a cluster provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] The embodiments of the present application are described below in conjunction with the drawings, obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Those skilled in the art can know that, with the development of technology and the appearance of new scenes, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0058] The terms "first", "second", "third", etc., and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of such terms is not made to limit the scope of the embodiments described herein and that the embodiments described herein are capable of functioning in the absence of any such terms. Moreover, the terms "comprising", "having", "including", and the like, are used in their open-ended, conventional sense, that is, to mean including but not limited to. Any use of such terms is not intended to restrict the scope of embodiments described herein to the specific examples described herein.

[0059] Throughout the present specification, the term "cluster" refers to a group of independent computing devices interconnected through a high-speed network, which constitutes a cluster and is managed in the mode of a single system. Each computing device in the cluster is referred to as a "node". In this context, the computing device can be a terminal, a server or other computing device with data processing and network connection functions; for example, the computing device can be a host, a tablet, a desktop, a laptop, a notebook computer, a netbook, etc.

[0060] Throughout the present specification, the term "node" generally refers to a computing device executing one or more software programs in a networked environment, and the "node" can also be implemented as a virtual node (software program) executed on one or more computing devices regarded as a server in the network. In a specific implementation, the node can be a physical machine or a virtual machine installed on a physical machine.

[0061] Throughout the present specification, the term "brain split" refers to a situation where, when a node cannot check the survival status of the other node at the same time, the node will take over the resources of the other node when the other node is considered abnormal, resulting in the resources originally accessed by one node being simultaneously accessed by multiple nodes.

[0062] Please refer to Figure 1 , Figure 1 is a distributed system architecture diagram. As shown in Figure 1 , in hardware implementation, multiple groups of racks (cabinets storing computing devices) are deployed in the distributed system architecture, each group of racks includes racks 11, 12 and 13, and multiple computing devices (such as Node0-Node6) are installed in each rack. In this embodiment, the computing devices can be servers, and each computing device can also be referred to as a node. Control nodes, service nodes and management nodes are deployed under each group of racks in the distributed system. For example, as shown in Figure 1 , the control nodes correspondFigure 1 Among the plurality of Node0, the business node corresponds Figure 1 Among Node1~Node6, the management node corresponds Figure 1 Among the plurality of Node0, the business node corresponds

[0063] In terms of network division, the distributed system architecture includes a control network, a business network and a management network. The control network is composed of nodes where the control cluster is located (i.e. control nodes). For example, as shown in FIG. 1, the control network is composed of a plurality of Node0 through EoR and / or ToR wiring methods. The control network is used to monitor the cluster member state and maintain the cluster so that each process in the cluster works normally. The business network is composed of nodes where the business cluster is located (i.e. business nodes). For example, as shown in FIG. 2, the business network is composed of a plurality of Node1~Node6 through EoR and / or ToR wiring methods. The management network is composed of nodes where the management cluster is located (i.e. management nodes). For example, as shown in FIG. 3, the management network is composed of a plurality of management master nodes and management standby nodes through EoR and / or ToR wiring methods. Figure 1 Figure 1 Figure 1

[0064] Control cluster: a cluster composed of a plurality of control processes for monitoring business processes or other control processes. The control cluster has a control master process, a control standby process and a member control process. As shown in FIG. 4, the node where the control master process is deployed is called the control master node, and the node where the control standby process is deployed is called the control standby node. Figure 1

[0065] Business cluster: a cluster composed of business processes for processing customer data, and there is a failover and failback relationship between the business processes. For example, when a business process fails, the business it handles can be switched to another node, and when the business process recovers from failure, the business it handles can return to normal. As shown in FIG. 5, generally, the business processes in the business nodes under each group of racks can be controlled by the control master process in the control nodes on the same rack. Figure 1

[0066] ​​​​​Manager cluster: a cluster composed of management processes, which are distributed on management nodes, responsible for system alarm, monitoring, log, configuration, and other operation and maintenance functions. A distributed system can deploy 2 management nodes, which are management master node and management backup node, and can be independently deployed on separate nodes or combined with computing and storage fusion nodes.

[0067] The nodes where the control cluster, service cluster, and management cluster are located are combined for networking through EoR and / or ToR wiring methods. As shown in Figure 1 The network between the nodes where the control cluster, service cluster, and management cluster are located is isolated from each other, and communication between them needs to pass through various communication protocols.

[0068] EoR (end of row): a wiring method that centrally installs switches in 1-2 racks and connects host servers / minicomputers in the equipment rack through horizontal cables in a permanent link manner.

[0069] ToR (top of rack): a wiring method that is an extension of EOR, which deploys 1-2 access switches on each server rack, and the server accesses the switch in the cabinet through a cable, and the uplink port of the switch accesses the aggregation switch in the network cabinet through a cable.

[0070] In a distributed system, when the firewall or routing table of the node where the service process is located is abnormal, the service process and other service processes connected thereto cannot confirm the survival of each other through heartbeat messages or lease messages. For example, as shown in Figure 1 In any rack 11, the Node1 node and the Node5 node where different service processes are located cannot confirm the survival of each other, at which time the Node1 node and the Node5 node will introduce arbitration, and the arbitration node or arbitration device determines that one of the Node1 node and the Node5 node is alive, and the other node interrupts the service and restarts. However, based on the background of the isolation of the management network and the service network, when the service nodes cannot confirm the survival of each other, it may not affect the service execution of the service nodes, at which time interrupting the service will cause the system performance to decrease and negatively affect the service.

[0071] Based on this, the embodiment of the application provides a cluster management method, which manages the survival state of each slave node through a master node, and flexibly and accurately provides a processing strategy to the slave node according to the service situation of the slave node when the communication between the slave node and the master node is abnormal, which can effectively avoid the brain split of the cluster, and can also reduce the negative influence caused by interrupting the service of the slave node.

[0072] The method provided in the embodiment of the application is applicable to all distributed systems, and the aboveFigure 1 Fig. 1 shows a distributed system architecture diagram, which is applicable to, but not limited to, a distributed storage system, a distributed database system, and a distributed computing system.

[0073] Please refer to Figure 2 Fig. 1 shows a distributed system architecture diagram, which is applicable to, but not limited to, a distributed storage system, a distributed database system, and a distributed computing system. Figure 2 Fig. 1 shows a distributed system architecture diagram, which is applicable to, but not limited to, a distributed storage system, a distributed database system, and a distributed computing system.

[0074] The control master node 110 is connected with the control backup node 120 and the service node 130 respectively, and the control backup node 120 is connected with the service node 130. In combination with Figure 1 , the control master node 110 and the control backup node 120 can be control master nodes and control backup nodes in a rack respectively, and the service node 130 can be a service node in the rack. For example, the control master node 110 and the control backup node 120 correspond to each Node0 node in the rack 11, the rack 12, and the rack 13 of a group of racks respectively. The service node 130 corresponds to each Node1-Node6 node in the rack 11, the rack 12, and the rack 13 of a group of racks.

[0075] The control master node 110 is configured to send a control message to the service node 130. The control message is used for the control master node 110 to control the service node 130 to perform fault switchover, fault recovery, etc.

[0076] The service node 130 is configured to send a live message to the control master node 110, and the live message is used for the control master node 110 to perceive whether the service node 130 fails. The service node 130 is also configured to process services under the control of the control master node 110.

[0077] The control backup node 120 is configured to work instead of the control master node 110 when the control master node 110 fails. It can be understood that the control backup node 120 can also be configured to send a live message to the control master node 110 to perceive whether the control master node 110 fails.

[0078] The live message can be a heartbeat message, a lease message, or other messages used to realize long connection keep-alive between nodes.

[0079] It can be understood that, in order to improve service processing efficiency and make function division more clear, the distributed system can classify the control nodes of the service nodes according to different services. Specifically, the control nodes can be classified into a first-level control master node and a second-level control master node, the second-level control master node is controlled by the first-level control master node, and the control master nodes of more levels can be further classified, and each level of control master node is controlled by the control master node of the upper level.

[0080] Further, the method of the embodiments of the present application can be applied to a cluster architecture as shown in Figure 3 which includes at least one master node and at least one slave node, in Figure 3 a specific example, the cluster system architecture includes a master node 310 and slave nodes 320a to 320c (hereinafter collectively referred to as 320).

[0081] may be combined Figure 1 and Figure 2 , the cluster architecture corresponding to the cluster can be a control cluster including a control master node and a control slave node, specifically, the master node 310 can be a control master node at a certain level, and the slave node 320 can be a control slave node at the same level or a control master node at a lower level controlled by the control master node; the cluster can also be a combination of a business cluster and a control master node, wherein the master node 310 is the control master node, and the slave node 320 is a business node; the cluster can also be a combination of a control node and a business node, wherein the master node 310 can be a control master node at a certain level, and the slave node 320 can be a control master node at a lower level, a control slave node at the same level, and a business node controlled by the control master node.

[0082] Among them, the master node 310 is used to manage the survival state of the slave nodes in the cluster, and instruct the slave nodes to prevent brain split processing according to the business requirements of the slave nodes and the preset processing rules.

[0083] Among them, the slave node 320 refers to a node in the cluster managed by the master node; in terms of specific function division, the slave node 320 can be a control node responsible for processing business of a business node, or a business node responsible for storing and processing business data and executing business services.

[0084] Among them, Figure 1 The solid line between the nodes is used to represent the long connection state between the two nodes. Each slave node 320 needs to maintain a normal communication state with the master node 310 through a survival message, and the slave nodes 320 do not need to maintain a long connection state through a survival message. That is, each slave node 320 only needs to send a survival message indicating survival to the master node 310, and the survival state of the slave nodes 320 is managed by the master node 310 to better prevent or handle the brain split phenomenon of the cluster.

[0085] Among them, the slave nodes 320 can be communicatively connected, and the control slave nodes can be communicatively connected.

[0086] In the cluster architecture provided in the embodiment of the present application, the master node 310 uniformly manages the survival status of the cluster's slave nodes 320. When the slave node 320 and the master node 310 cannot confirm each other's survival, the master node 310 can judge the current status of each slave node based on the slave node information in the cluster and perform corresponding processing without introducing arbitration from the slave node 320 to preempt resources. While effectively preventing the cluster from experiencing brain split and data damage, it can maintain business continuity and reduce the negative impact of business interruptions caused by arbitration on the business.

[0087] The cluster system structure provided in the embodiment of the present application is introduced above. The cluster management method provided in the embodiment of the present application will be described below.

[0088] exist Figure 3 Based on the cluster architecture shown, see Figure 4 , Figure 4 A flow chart of a cluster management method provided in an embodiment of the present application is provided. The method is applied to a cluster, the cluster including at least one master node and at least one slave node, the at least one slave node including a first slave node. Figure 4 As shown, the method includes steps 401 to 407.

[0089] 401. The first slave node sends a survival message to the master node.

[0090] In the cluster to which the method of this embodiment is applied, the slave node and the master node need to periodically send survival messages to each other to confirm the survival status of the other party.

[0091] The survival message may be a heartbeat message or a lease message, or a response message corresponding to the two.

[0092] like Figure 5 As shown in the figure, after the cluster is operating normally, the master node sends cluster information to each slave node. This cluster information contains the cluster topology, the network addresses of the cluster nodes, and the network status. This allows the slave node to communicate with other slave nodes or upgrade to a master node if the preset conditions are met. The solid line represents the message sent, and the dotted line represents the response message returned.

[0093] Each slave node receives and records the cluster information and sends a response message to the master node. When a node in the cluster changes, such as adding or removing a node or changing the master node, the changed cluster master node will send the updated cluster information to each slave node.

[0094] The cluster topology refers to the connection relationship between cluster nodes, which is used to represent the network structure formed by cluster nodes. Through this topology and the network addresses of the corresponding cluster nodes, slave nodes can communicate with each other.

[0095] 402、if no response message about the alive message returned by the master node is received within a first preset time after sending the alive message, the first slave node determines that communication between the first slave node and the master node is abnormal.

[0096] The first preset time can be a time set by the system according to the feedback time of a general response message, for example, the preset time can be set to 2s; or the first preset time can be set by a user according to requirements in the system.

[0097] If no response message about the alive message returned by the master node is received within a first preset time after sending the alive message, it can be determined that the alive message cannot reach between the first slave node and the master node, and thus it is determined that communication between the first slave node and the master node is abnormal.

[0098] It can be understood that the condition for determining communication abnormality at least includes that the alive message cannot reach. In other embodiments, on the basis of taking that the alive message cannot reach as a determination condition, other determination conditions can be added, for example, that the alive message cannot reach between the first slave node and multiple slave nodes in the cluster, to further limit the communication abnormality state, improve the accuracy of determining communication abnormality, and reduce the negative impact of misjudgment on the business in the cluster.

[0099] When the first slave node determines that communication between the first slave node and the master node is abnormal, it is judged that the first slave node can cause a split-brain of the cluster, and step 403 is performed.

[0100] It can be understood that step 402 is not necessarily performed, and the first slave node can determine whether communication between the first slave node and the master node is abnormal through other manners.

[0101] In a possible implementation, the first slave node can obtain a first time of receiving the alive message from the master node most recently; if a difference between the first time and a current time is greater than or equal to a first threshold value, the first slave node determines that communication between the first slave node and the master node is abnormal.

[0102] The first threshold value can be set according to a sending period of the alive message and a historical communication delay between the first slave node and the master node.

[0103] It can be understood that if the difference between the first time and the current time is less than the first threshold value, the first slave node cannot determine whether communication between the first slave node and the master node is abnormal, and in this case, no processing is performed. Of course, in other embodiments, when the difference between the first time and the current time is less than the first threshold value, further confirmation can be performed, for example, steps 401 and 402 are performed; for another example, whether the difference between the first time and the current time is greater than or equal to the first threshold value is confirmed again after a period of time, and whether communication between the first slave node and the master node is abnormal is determined.

[0104] In another possible implementation, the master node and the slave node can also confirm the communication state between them through a management message; the first slave node can acquire a first time of the latest received alive message and a second time of the latest received management message, the alive message and the management message being sent by the master node; if a difference between the first time and a current time is greater than or equal to a first threshold value, and a difference between the second time and the current time is greater than or equal to a second threshold value, the first slave node determines that the communication between the first slave node and the master node is abnormal.

[0105] The management message can be any message sent by the master node to the first slave node except the alive message.

[0106] The second threshold value can be set according to the requirement of different services of the slave node for service continuity. For example, the second threshold value set in the slave node corresponding to a service with higher requirement for service continuity is lower, and the detection of whether the communication between the slave node and the master node is abnormal is triggered more easily.

[0107] It can be understood that the first time and the second time are two parallel judgment factors, and the communication between the first slave node and the master node is determined to be abnormal when both of them meet the preset conditions. Correspondingly, in the case that the difference between the first time and the current time is less than the first threshold value, and the difference between the second time and the current time is greater than the second threshold value; or, the difference between the first time and the current time is greater than the first threshold value, and the difference between the second time and the current time is less than the second threshold value; or, the difference between the first time and the current time is less than the first threshold value, and the difference between the second time and the current time is less than the second threshold value, the first slave node cannot determine whether the communication between the first slave node and the master node is abnormal, and in this case, no processing can be performed, or further confirmation can be performed. The specific confirmation manner can be understood with reference to the related part of the previous possible implementation, which will not be described here.

[0108] For example, the first threshold value is 3 seconds, and the second threshold value is 1 minute. When the first slave node detects that the current time is 4 seconds away from the time of the latest received alive message of the master node, if the first slave node is determined to be abnormal according to the first time of the first slave node receiving the alive message, step 403 is executed; and according to the two conditions of the first time and the second time of the first slave node, the current time is 20 seconds away from the time of the latest received management message of the master node, and it cannot be determined whether the communication between the first slave node and the master node is abnormal, and the first slave node does not execute step 403.

[0109] In the embodiments of the present application, the receiving time of the management message is added as a judgment factor parallel to the receiving time of the survival message, so that when the survival message is lost due to a firewall or a damaged routing table, a short busy of the cluster system, or a heartbeat process failure, etc., the communication state between the first slave node and the master node can be determined by using the management message, further increasing the stability and fault tolerance of the cluster system, while maintaining the maximum business continuity.

[0110] It can be understood that the slave node will record the corresponding receiving time when receiving the survival message or the management message from the master node, in order to perform the above method steps. The master node is the same. As shown in Figure 6 The slave node receives the survival message and the management message, and returns the corresponding response message to the master node; then records and updates the receiving time in the background. After receiving the response message returned by each slave node, the master node records the receiving time of the survival message or the management message between the cluster master and each slave node. For details, please refer to Table 1 and Table 2 below.

[0111]

[0112] Table 1 Master node record table

[0113]

[0114] Table 2 Slave node record table

[0115] It should be noted that the above two possible implementations of detecting whether the communication between the slave node and the master node is abnormal can be periodic node self-checking; or the master node can issue a self-checking instruction to each slave node, and each slave node can perform corresponding detection according to the self-checking instruction. When the master node does not receive the response message returned by the slave node according to the self-checking instruction within a certain time, the master node starts to detect the communication state of the slave node.

[0116] In addition, the node self-checking can also be triggered by a preset event, such as the node load being greater than or less than a certain threshold.

[0117] It can be understood that when the first slave node receives the response message after sending the survival message to the master node, it determines that the communication between itself and the master node is normal; further, when the first slave node determines that the communication between itself and the master node is normal, it judges that the current brain split risk has been eliminated, and ends the current processing flow for preventing cluster brain split.

[0118] 403、When the first slave node determines that the communication between itself and the master node is abnormal, the first slave node sends a first message to the master node.

[0119] The first message is used to request the master node to indicate how to handle the communication abnormality. That is, when the first slave node detects that it may cause the brain split phenomenon of the cluster, the first slave node inquires the master node for the handling strategy.

[0120] It can be understood that the alive message may be lost due to a firewall, a short busy of the cluster system, or a heartbeat process failure, and therefore, when the alive message communication between the slave node and the master node is abnormal, there is a possibility that the management message can be normally communicated.

[0121] If the handling strategy returned by the master node according to the first message is received within the second preset time after the first message is sent, the first slave node performs step 404.

[0122] If the handling strategy returned by the master node according to the first message is not received within the second preset time after the first message is sent, the first slave node performs step 405.

[0123] The second preset time can be determined according to the historical communication delay between the slave node and the master node.

[0124] It can be understood that if the first slave node receives the handling strategy after the second preset time after the first message is sent, step 404 is performed. Specifically, when the first slave node receives the handling strategy after step 405 is performed, the first slave node can immediately send a termination forwarding message to the second slave node, or can notify the second slave node that the communication with the master node has been restored after receiving the feedback message of the second slave node; the order of the notification action and the execution of step 404 is not distinguished.

[0125] 404. The first slave node performs a first operation according to the handling strategy.

[0126] The first operation refers to an operation corresponding to the handling strategy. For example, when the master node determines that the alive message communication between the current master node and the first slave node is abnormal and the management message communication is normal, a maintenance strategy and / or a heartbeat restart strategy can be returned; when the master node determines that the alive message communication between the current master node and the first slave node is abnormal and the management message communication is abnormal or the communication state is poor, a node restart strategy can be returned.

[0127] Corresponding to the maintenance strategy, the first slave node can perform the operation of responding to the management message and recording the management message receiving time, so as to maintain the current node state and the communication mode of communicating by the management message; corresponding to the heartbeat restart strategy, the first slave node can restart the heartbeat process and send a heartbeat message to the master node; corresponding to the node restart strategy, the first slave node can interrupt the service and rejoin the cluster after restarting itself.

[0128] It can be understood that the master node can return multiple restart processing strategies at the same time to flexibly cope with different communication abnormal situations.

[0129] It can be understood that when the first slave node is restarted, the access right of the corresponding resource will be redistributed by the master node in the cluster.

[0130] 405、The first slave node sends a second message to a second slave node.

[0131] The second message is used to request the second slave node to forward the first message to the master node, the second message includes the first message, and the IP address of the first slave node and the IP address of the master node.

[0132] The second slave node can be one or more of all slave nodes in the cluster except the first slave node.

[0133] It can be understood that the first slave node sends the second message to the second slave node according to the topological relationship recorded in the node.

[0134] If the first slave node receives a feedback message returned by the second slave node according to the second message, the first slave node performs a second operation according to the feedback message.

[0135] If the first slave node does not receive the feedback message returned by the second slave node according to the second message within a third preset time, step 407 is performed.

[0136] The feedback message can be a first feedback message containing a processing strategy, or a second feedback message indicating that the processing strategy returned by the master node is not received. It can be understood that after the second slave node forwards the first message to the master node, if the processing strategy returned by the master node is received within a second preset time, the first feedback message is returned to the first slave node; if the processing strategy returned by the master node is not received within the second preset time, the second feedback message is returned to the first slave node.

[0137] Specifically, if the first slave node receives the first feedback message, step 404 is performed; if the first slave node receives the second feedback message, step 406 is performed.

[0138] When the number of second slave nodes is multiple, the first slave node will send a second message to one second slave node, if the first slave node does not receive the first feedback message returned by the second slave node according to the second message within a third preset time, the first slave node sends the second message to another second slave node which has not sent the second message, until the first feedback message is received, or all second slave nodes have been sent the second message.

[0139] If the first slave node receives the first feedback message returned by one second slave node, step 404 is performed according to the first feedback message.

[0140] If all the feedback messages received by the first slave node are the second feedback messages, step 406 is performed.

[0141] If the first slave node does not receive any feedback message, step 407 is performed.

[0142] 406. The first slave node initiates a master election request to all the second slave nodes according to the second feedback messages.

[0143] If none of the second slave nodes capable of normal communication with the first slave node can receive the processing strategy returned by the master node, it can be further determined that the current master node has failed to perform the master node function, and thus a new master election request is initiated to each second slave node to continue responding to the service by the new master node selected.

[0144] Specifically, the first slave node and each second slave node can elect a master node by means of heartbeat voting, or can elect a master node according to the transaction ID and node ID of the slave node.

[0145] For example, each slave node sends heartbeat information to other slave nodes through a topological relationship, and the count is incremented by 1 for each response information returned by other slave nodes. After the fourth preset time, each slave node broadcasts the count result to all other slave nodes, and the slave node with the largest count value is elected as the master node. For another example, each slave node broadcasts its transaction ID and node ID to all other slave nodes, wherein the larger the transaction ID, the newer the data of the slave node. The slave node compares its transaction ID and node ID, and votes for the slave node with the largest transaction ID. If there are multiple slave nodes with the largest transaction ID, the slave node with the largest node ID is voted. Finally, the slave node with the highest votes is elected as the master node.

[0146] 407. The first slave node performs a self-restart operation.

[0147] When the first slave node cannot receive feedback messages from the master node and any second slave node, the first slave node determines that its external communication function is abnormal, and performs a preset restart operation to restore the external communication function. If the external communication function cannot be restored after multiple restarts, the master node outputs an alarm information to the management node in the distributed system where the cluster is located when the master node detects that the first slave node cannot establish communication for a long time, so that the management personnel can handle the communication abnormality of the first slave node.

[0148] In the embodiments of the present application, when the first slave node of the cluster detects that the communication with the master node is abnormal, the first slave node directly or indirectly inquires the processing strategy from the master node. If the processing strategy returned by the master node directly or through the second slave node is received, the first slave node performs the first operation according to the processing strategy. If the processing strategy is not received, the first slave node determines whether the communication abnormality is caused by itself or the master node according to the feedback of the second message by each second slave node. If the communication abnormality is caused by itself, the first slave node performs the preset restart operation. If the communication abnormality is caused by the master node, the first slave node performs the re-election of the master node in conjunction with each second slave node. In the embodiments of the present application, the first slave node tries to communicate with the master node through as many ways as possible to obtain the corresponding processing strategy. When the communication with the master node fails, the first slave node performs the preset operation according to the communication state of the second slave node, which can maximize the determination of the processing operation of the slave node according to the information of the slave node, and further flexibly and accurately process the communication abnormality in the cluster, prevent the cluster split-brain, and reduce the negative impact on the business.

[0149] The above is an embodiment of the cluster management method performed by the slave node in the embodiments of the present application. The cluster management method performed by the master node will be described below.

[0150] The following will be described with reference to Figure 7 , Figure 7 Another flowchart of a cluster management method provided in the embodiments of the present application is shown. The method is applied to a cluster, and the cluster includes at least one master node and at least one slave node, and the at least one slave node includes a third slave node. As shown in Figure 7 , the method includes steps 701 to 705.

[0151] 701. The master node determines the third slave node in communication abnormality with the master node.

[0152] In a possible implementation, the master node can periodically determine the third slave node, or can determine the third slave node when the cluster triggers a preset event, for example, when the business load of the cluster is greater than or less than a certain threshold. The third slave node can be any slave node in the cluster.

[0153] Specifically, the master node records the third time of receiving a survival message and the fourth time of receiving a response message corresponding to a management message, and the survival message and the response message are sent by each slave node. The master node can detect the communication state of each slave node with the master node according to the third time, the fourth time and the current time of each slave node, and determine the third slave node in communication abnormality according to the communication state.

[0154] It can be understood that the specific implementation of detecting whether the communication is abnormal is similar to the corresponding part in step 402 in the embodiment shown in Figure 4 .

[0155] In another possible implementation, the master node can receive a first message from a certain slave node, the first message carrying identification information of a third slave node; the master node can determine the third slave node according to the identification information.

[0156] After determining the third slave node, the master node can perform a corresponding third operation according to the number of the third slave node.

[0157] Specifically, when the number of the third slave node is less than the number of the at least one slave node, i.e., not all slave nodes have abnormal communication states with the master node, the master node performs step 702.

[0158] When the number of the third slave node is equal to the number of the at least one slave node, i.e., all slave nodes have abnormal communication states with the master node, the master node performs step 705.

[0159] 702. The master node determines a processing strategy according to service information of the third slave node.

[0160] In a possible implementation, if the master node receives a first message sent by the third slave node within a fifth preset time after determining that the number of the third slave node is less than the number of the at least one slave node, the master node determines a corresponding processing strategy according to the first message; if the master node does not receive the first message within the fifth preset time after determining that the number of the third slave node is less than the number of the at least one slave node, the master node actively determines the processing strategy according to service information of the third slave node.

[0161] The master node can obtain service information of the third slave node, and then determine a processing strategy according to the service information and a preset processing rule.

[0162] In a possible implementation, the first message carries service information of the slave node; the master node can obtain the service information from the first message.

[0163] In a possible implementation, the master node stores service information of all slave nodes, and the master node can obtain service information of the third slave node from the storage.

[0164] The service information can include encoding, category, and characteristics of the service.

[0165] For example, when the service characteristics of the slave node are continuity priority, the master node can determine a processing strategy corresponding to the service characteristics, i.e., maintaining the current node state and the current communication mode, according to a preset processing rule. When the service characteristics are security priority, the master node can determine a corresponding processing strategy as restarting the node.

[0166] It can be understood that when the master node determines that the third slave node has executed the multiple restart node strategies before the current time when the first message is sent, the master node can output alarm information to a management node in the distributed system where the cluster is located, so that the management personnel handle the third slave node with the communication exception.

[0167] Exemplarily, the processing strategy can include a restart network strategy and a restart system strategy in addition to the maintenance strategy, the restart heartbeat strategy and the restart node strategy mentioned above, to correspond to different business requirements and cluster node communication states.

[0168] When the third slave node is responsible for important functions in a sub-network of the distributed system, such as a control master node, a management master node, a business node where upstream business is located, the brain split of the third slave node can affect the normal operation of the network where the third slave node is located; at this time, the master node can determine that the corresponding processing strategy is the restart network strategy or the restart system strategy.

[0169] Specifically, the master node can determine whether the corresponding processing strategy is the restart network strategy or the restart system strategy according to the coupling degree between the target network where the third slave node is located and other networks. When the coupling degree between the networks is low and the isolation degree is high, restarting the target network will not affect other networks, and the master node can determine the restart network strategy as the processing strategy; when the coupling degree is high and the isolation degree is low, restarting the target network will affect the operation of other networks, and the master node can determine the restart system strategy as the processing strategy.

[0170] It can be understood that the processing strategies mentioned in the embodiments of the present application are only examples and are not limited, and the master node can preset more processing strategies to cope with different communication exception situations in actual execution.

[0171] 703、The master node sends the processing strategy to the third slave node.

[0172] The master node can directly send the processing strategy to the third slave node; in the case where a response message of the third slave node is not received within a second preset time, the processing strategy can be forwarded to the third slave node through a slave node with a normal communication state.

[0173] Specifically, the master node requests a slave node with a normal communication state to forward the processing strategy, and if the feedback message returned by the slave node according to the forwarding request is that the third slave node does not respond, the master node requests another slave node with a normal communication state to forward the processing strategy, which has not sent the forwarding request, until a feedback message from a slave node with a normal communication state is received that confirms that the third slave node has received the processing strategy, or the master node has sent the forwarding request to all slave nodes with a normal communication state.

[0174] It can be understood that the third slave node returns a response message to the slave node forwarding the processing strategy after receiving the processing strategy, and then the slave node forwarding the processing strategy returns a feedback message to the master node to confirm that the third slave node has received the processing strategy.

[0175] That is, the master node can determine whether the third slave node receives the processing strategy through the direct feedback of the third slave node and the indirect feedback of each slave node with normal communication state, and when the master node confirms that the third slave node cannot receive the processing strategy, step 704 is executed.

[0176] If the master node confirms that the third slave node receives the processing strategy, the current processing procedure for preventing cluster split-brain is ended.

[0177] 704, the master node isolates the third slave node.

[0178] If the feedback messages returned by all slave nodes with normal communication state indicate that the third slave node does not respond, the master node can determine that the third slave node is in a state of being unable to communicate or being down, and the execution of its business is likely to be affected, so the master node can interrupt the business of the third slave node, release the corresponding resources, and reallocate the resources and business in the cluster.

[0179] 705, the master node performs a self-restart operation.

[0180] When the master node determines that the communication states of all slave nodes and the master node itself are abnormal, the master node performs a preset self-restart operation and re-joins the cluster as a slave node after self-restarting.

[0181] In the embodiments of the application, the master node determines the third slave node with abnormal communication state with itself by actively detecting or receiving the first message, and performs corresponding operations according to the number of third slave nodes, which can quickly discover and process slave nodes with abnormal communication, maintain the stability of the cluster, and better maintain the continuity and security of the business.

[0182] The above respectively introduces the embodiments of the cluster slave node and the cluster master node as the execution subject of the cluster management method, and the following further describes the cluster management method provided by the embodiments of the application from the different fault conditions of the cluster node that can cause split-brain. The cluster includes a master node and at least one slave node, and the at least one slave node includes a first slave node.

[0183] I. The alive message between the first slave node and the master node is unreachable, and the management message communication is normal.

[0184] As mentioned above, when the survival message is unreachable, there is a possibility that the management message can be communicated normally. For example, the heartbeat message is intercepted by the firewall of the master node because it is a small data packet or an empty packet containing only a packet header, while the data volume of the management message is normal, which can pass through the firewall.

[0185] To prevent cluster brain split caused by the above-mentioned situation, please refer to Figure 8 , Figure 8 Another flowchart of a cluster management method provided by the embodiment of the present application is shown, which specifically includes steps 801 to 806.

[0186] 801. The first slave node sends a survival message to the master node.

[0187] 802. If the response message about the survival message returned by the master node is not received within the first preset time after sending the survival message, the first slave node determines that the communication between itself and the master node is abnormal.

[0188] It can be understood that the first slave node does not receive the response message, which can be caused by the failure of the first slave node in sending the survival message process, or by the failure of the master node in returning the response message process.

[0189] 803. The first slave node sends a first message to the master node.

[0190] The steps 801 to 803 in the embodiment are similar to the steps 401 to 403 in the embodiment shown in Figure 4 , and the content of the corresponding part can be understood by reference, which will not be described here.

[0191] 804. The master node determines a processing strategy according to the first message.

[0192] 805. The master node sends the processing strategy to the first slave node.

[0193] The steps 804 to 805 in the embodiment are similar to the steps 702 to 703 in the embodiment shown in Figure 7 , and the content of the corresponding part can be understood by reference, which will not be described here.

[0194] 806. The first slave node performs a first operation according to the processing strategy.

[0195] The step 806 in the embodiment is similar to the step 404 in the embodiment shown in Figure 4 , and the content of the corresponding part can be understood by reference, which will not be described here.

[0196] II. The first slave node cannot communicate.

[0197] Please refer to Figure 9 , Figure 9Another flowchart of the cluster management method provided by the embodiment of the present application is shown in FIG. 9, which specifically includes steps 901 to 904.

[0198] 901. When the first slave node determines that it is abnormal in communication with the master node, the first slave node sends a first message to the master node.

[0199] In this case, the first slave node is unable to communicate, and thus is unable to send or receive a survival message; the first slave node can determine that it is abnormal in communication with the master node according to the time of sending a survival message or the event of receiving a survival message.

[0200] Specifically, step 901 in the embodiment of the present application is similar to step 401 in the embodiment shown in FIG. 4, and can be understood with reference to the content of the corresponding part, which will not be described herein again. Figure 4 Steps 401 to 403 in the embodiment shown in FIG. 4 are similar, and can be understood with reference to the content of the corresponding part, which will not be described herein again.

[0201] 902. If the processing strategy returned by the master node is not received within a second preset time after the first message is sent, the first slave node sends a second message to a second slave node.

[0202] Step 902 in the embodiment of the present application is similar to step 405 in the embodiment shown in FIG. 4, and can be understood with reference to the content of the corresponding part, which will not be described herein again. Figure 4 903. The first slave node performs a self-restart operation.

[0203] In this case, the first slave node is unable to communicate, and thus is unable to send the second message to the second slave node, and is also unable to receive a feedback message about the second message.

[0204] When the corresponding feedback message is not received within a third preset time after each time the second message is sent, the first slave node can determine that it is faulty in communication function, and performs a preset self-restart function.

[0205] Figure 4 Step 903 in the embodiment of the present application is similar to step 407 in the embodiment shown in FIG. 4, and can be understood with reference to the content of the corresponding part, which will not be described herein again.

[0206] 904. The master node isolates the first slave node.

[0207] The master node can determine that the first slave node is abnormal in communication after survival messages of the first slave node are not received within a certain period; the first slave node is unable to communicate, and thus the master node isolates the third slave node after confirming that the first slave node is unable to receive the processing strategy by direct or indirect means.

[0208] It can be understood that the execution order of step 904 and steps 901 to 903 is not specifically limited in the embodiment of the present application, and step 904 can be executed before or after any one of steps 901 to 903.​

[0209] Specifically, step 904 in the embodiment can be understood by referring to the content of steps 401 to 403 in the embodiment shown in FIG. 4, which will not be described here again. Figure 7 Steps 701 to 704 in the embodiment shown in FIG. 7 can be understood by referring to the content of the corresponding parts, which will not be described here again.

[0210] III. The first slave node cannot communicate with the master node.

[0211] Reference can be made to Figure 10 , Figure 10 Another cluster management method provided by the embodiment of the present application is shown in the flowchart, and specifically includes steps 1001 to 1007.

[0212] 1001. When the first slave node determines that it cannot communicate with the master node, the first slave node sends a first message to the master node.

[0213] The first slave node cannot communicate with the master node, and the first slave node can determine that it cannot communicate with the master node according to the time of sending a survival message or the event of receiving a survival message.

[0214] Specifically, step 1001 in the embodiment can be understood by referring to the content of steps 401 to 403 in the embodiment shown in FIG. 4, which will not be described here again. Figure 4

[0215] 1002. If the first slave node does not receive a processing strategy returned by the master node within a second preset time after sending the first message, the first slave node sends a second message to the second slave node.

[0216] Step 1002 in the embodiment can be understood by referring to the content of step 405 in the embodiment shown in FIG. 4, which will not be described here again. Figure 4

[0217] 1003. The second slave node forwards the first message to the master node.

[0218] After receiving the request of the first slave node for forwarding the first message, the second slave node can forward the first message to the master node.

[0219] 1004. The master node determines a processing strategy.

[0220] 1005. The master node returns the processing strategy to the second slave node.

[0221] The master node can first return the processing strategy to the first slave node according to the identification information of the first slave node in the first message, and the first slave node cannot communicate with the master node, so the second slave node forwards the processing strategy to the first slave node.

[0222] ​​Steps 1004 to 1005 in this embodiment are similar to steps 404 to 405 in the embodiment shown in Figure 7 Steps 702 to 703 in the embodiment shown in FIG. 7 can be understood with reference to the content of the corresponding parts, which will not be described here again.

[0223] 1006. The second slave node sends a first feedback message to the first slave node.

[0224] After receiving the processing strategy of the master node and the request for forwarding the processing strategy, the second slave node can forward the processing strategy to the first slave node.

[0225] 1007. The first slave node performs a first operation according to the processing strategy in the first feedback message.

[0226] Step 1007 in this embodiment is similar to step 406 in the embodiment shown in Figure 4 Step 404 in the embodiment shown in FIG. 4 can be understood with reference to the content of the corresponding parts, which will not be described here again.

[0227] IV. The master node cannot communicate.

[0228] Reference can be made to Figure 11 , Figure 11 Another flowchart of a cluster management method provided by an embodiment of the present application is shown in FIG. 11, which specifically includes steps 1101 to 1106.

[0229] 1101. When the first slave node determines that it is abnormal in communication with the master node, it sends a first message to the master node.

[0230] Wherein, the master node cannot communicate, and the first slave node can determine that it is abnormal in communication with the master node according to the time of sending a survival message or the event of receiving a survival message.

[0231] Specifically, step 1101 in this embodiment is similar to steps 401 to 403 in the embodiment shown in Figure 4 Step 401 to 403 in the embodiment shown in FIG. 4 can be understood with reference to the content of the corresponding parts, which will not be described here again.

[0232] 1102. If the processing strategy returned by the master node is not received within a second preset time after sending the first message, the first slave node sends a second message to the second slave node.

[0233] Step 1102 in this embodiment is similar to step 405 in the embodiment shown in Figure 4 Step 405 in the embodiment shown in FIG. 4 can be understood with reference to the content of the corresponding parts, which will not be described here again.

[0234] 1103. The second slave node forwards the first message to the master node.

[0235] After receiving the first message forwarding request of the first slave node, the second slave node can forward the first message to the master node.

[0236] 1104、The second slave node returns a second feedback message to the first slave node.

[0237] The master node is unable to communicate, and the second slave node is unable to receive the processing strategy returned by the master node within a second preset time after forwarding the first message. Therefore, the second slave node returns a second feedback message to the first slave node.

[0238] 1105、The first slave node sends a re-election request to the second slave node.

[0239] The step 1105 in the embodiment is similar to the step 406 in the embodiment shown in Figure 4 The step 1106 in the embodiment is similar to the step 705 in the embodiment shown in

[0240] 1106、The master node performs a self-restart operation.

[0241] It can be understood that the application embodiments do not specifically limit the execution order of the step 1106 and the steps 1101 to 1105, and the step 1106 can be executed before or after any of the steps 1101 to 1105.

[0242] The step 1106 in the embodiment is similar to the step 705 in the embodiment shown in Figure 7 The step 1106 in the embodiment is similar to the step 705 in the embodiment shown in

[0243] Please refer to Figure 12 The application embodiments further provide a first computing device 1200, which can serve as a first slave node in the above embodiments and form a cluster with a second computing device serving as a master node in the above embodiments; the first computing device 1200 can be used to implement the method and functions related to the first slave node in the above embodiments.

[0244] As shown in Figure 12 The first computing device 1200 can include a processor 1201 and a transmitter 1202, and the processor 1201 and the transmitter 1202 are coupled.

[0245] The transmitter 1202 is configured to send a first message to the second computing device when the first computing device detects a communication anomaly with the second computing device, the first message being used to request the second computing device to indicate how to handle the communication anomaly;

[0246] The processor 1201 is configured to perform a first operation according to a processing strategy of the second computing device when the first message is received.

[0247] The transmitter 1202 is specifically configured to implement communication with other devices. The transmitter 1202 can be a transceiver interface, specifically a transceiver or an input / output interface. The transmitter 1202 may, for example, be an interface circuit.

[0248] Optionally, the first computing device 1200 further includes a memory 1203 configured to store program instructions.

[0249] The processor 1201 is further configured to execute the program instructions in the memory 1203, so that the first computing device 1200 performs the cluster management method provided in the embodiments. Figure 4 Specifically, the processor 1201 can drive the transmitter 1202 to implement the above functions by invoking the program instructions in the memory 1203.

[0250] Please refer to Figure 13 The embodiments of the present application also provide a second computing device 1300, which can serve as a master node in the above embodiments and form a cluster with the first computing device serving as a slave node in the above embodiments. The second computing device 1300 can be configured to implement the methods and functions related to the master node in the above embodiments.

[0251] As shown in Figure 7 The second computing device 1300 can include a processor 1301.

[0252] The processor 1301 is configured to determine a first computing device that has a communication anomaly with the second computing device itself.

[0253] The processor 1301 is further configured to perform a third operation according to the number of the first computing devices.

[0254] Optionally, the second computing device 1300 further includes a memory 1302 configured to store program instructions.

[0255] The processor 1301 is further configured to execute the program instructions in the memory 1302, so that the second computing device 1300 performs the cluster management method provided in the embodiments. Figure 14

[0256] Optionally, the second computing device 1300 further includes a receiver 1303 and a transmitter 1304.

[0257] ​The receiver 1303 and the transmitter 1304 are specifically configured to implement the communication with other devices. The receiver 1303 and the transmitter 1304 can be the same transceiver, which is the receiver 1303 when receiving information and is the transmitter 1304 when transmitting information, or can be different transceivers. Specifically, the receiver 1303 and the transmitter 1304 can be transceivers or input / output interfaces. For example, the receiver 1303 and the transmitter 1304 can be interface circuits.

[0258] The receiver 1303 is configured to receive a first message sent by a first computing device, and the first message is used to request the second computing device 1300 to indicate how to handle the communication exception between the second computing device 1300 and the first computing device.

[0259] The processor 1301 is further configured to determine a processing strategy according to the service information of the first computing device.

[0260] The transmitter 1304 is configured to send the processing strategy to the first computing device, so that the first computing device performs a first operation according to the processing strategy. Specifically, the processor 1301 can achieve the above functions by calling the program instructions in the memory 1302 to drive the receiver 1303 and the transmitter 1304.

[0261] Please refer to Figures 8 to 11 The embodiment of the present application also provides a cluster 1400, which includes at least one first computing device 1401 and at least one second computing device 1402.

[0262] The first computing device 1401 is similar to the first computing device 1200 described above, and is a slave node in the cluster; and the second computing device 1402 is similar to the second computing device 1300 described above, and is a master node in the cluster. The first computing device 1401 and the second computing device 1402 are configured to perform the cluster management method as described in any one of the above Figure 4 The provided cluster management method.

[0263] In another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores computer execution instructions. When at least one processor of a device executes the computer execution instructions, the device performs the cluster management method as described in any one of the above Figures 7 to 11 and Figure 4 The cluster management method described in any one of the above

[0264] In another embodiment of the present application, a computer program product is also provided, which includes computer-executable instructions stored in a computer-readable storage medium. At least one processor of a device can read the computer-executable instructions from the computer-readable storage medium, and the at least one processor executes the computer-executable instructions to cause the device to perform the cluster management method described in any of the above embodiments. Figures 7 to 11 and ​ any of the above embodiments.

[0265] Those skilled in the art can clearly understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0266] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0267] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0268] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0269] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0270] If the functions are implemented in the form of software function units and sold or used as independent products, the functions can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A cluster management method characterized by, The method is applied to a cluster, the cluster comprising at least one master node and at least one slave node, the at least one slave node comprising a first slave node; the method comprising: The first slave node acquires a first time of receiving a last alive message and a second time of receiving a last management message, the alive message and the management message being sent by the master node; If a difference between the first time and a current time is greater than a first threshold value, and a difference between the second time and the current time is greater than a second threshold value, the first slave node determines that communication between the first slave node and the master node is abnormal; When the first slave node detects that the communication between the first slave node and the master node is abnormal, the first slave node sends a first message to the master node, the first message being used to request the master node to indicate how to handle the communication abnormality; If the first slave node receives a processing strategy of the master node about the first message, the first slave node performs a first operation according to the processing strategy.

2. The method of claim 1, wherein, The at least one slave node further comprises a second slave node; after the first slave node sends the first message to the master node, the method further comprises: If the processing strategy is not received within a second preset time after the first message is sent, the first slave node sends a second message to the second slave node, the second message being used to request the second slave node to forward the first message to the master node; If a feedback message of the second slave node about the second message is received, the first slave node performs a second operation according to the feedback message.

3. The method of claim 2, wherein, After the first slave node sends the second message to the second slave node, the method further comprises: If the feedback message is not received within a third preset time after the second message is sent, the first slave node performs a self-restart operation.

4. The method of claim 2, wherein, The feedback message comprises a second feedback message, the second feedback message being used to indicate that the second slave node does not receive the processing strategy; The first slave node performing the second operation according to the feedback message comprises: The first slave node sends a re-election request to the second slave node according to the second feedback message.

5. A cluster management method characterized by, The method is applied to a cluster, the cluster comprising at least one master node and at least one slave node; the method comprising: The master node receives a first message sent by the slave node, the first message being used to request the master node to indicate how to handle communication abnormality between the slave node and the master node, and the master node determines whether the slave node and the master node are in communication abnormality by the following way: The slave node acquires a first time of receiving a last alive message and a second time of receiving a last management message, the alive message and the management message being sent by the master node; If a difference between the first time and a current time is greater than a first threshold value, and a difference between the second time and the current time is greater than a second threshold value, the slave node determines that communication between the slave node and the master node is abnormal; The master node determines a processing strategy according to service information of the slave node; The master node sends the processing strategy to the slave node, so that the slave node performs a first operation according to the processing strategy.

6. A cluster management method characterized by, The application is applied to a cluster, the cluster comprising at least one master node and at least one slave node, the at least one slave node comprising a third slave node; the method comprising: The master node determines the third slave node with abnormal communication; The master node performs a third operation according to the number of the third slave node, comprising: When the number of the third slave node is less than the number of the at least one slave node, and a first message sent by the third slave node is not received within a fifth preset time after the third slave node is determined, the master node determines a processing strategy according to service information of the third slave node, and sends the processing strategy to the third slave node; wherein the first message is used to request the master node to indicate how to process the abnormal communication; When the number of the third slave node is equal to the number of the at least one slave node, the master node performs a self-restart operation.

7. A first computing device, comprising: The first computing device and the second computing device constitute a cluster; the first computing device comprises a processor and a transmitter; The transmitter is used to send a first message to the second computing device when the first computing device detects abnormal communication with the second computing device, the first message being used to request the second computing device to indicate how to process the abnormal communication; The first computing device determines whether to communicate abnormally with the second computing device by the following way: Receiving a first time of receiving a survival message and a second time of recently receiving a management message, the survival message and the management message being sent by the second computing device; If the difference between the first time and the current time is greater than a first threshold value, and the difference between the second time and the current time is greater than a second threshold value, it is determined that the first computing device communicates abnormally with the second computing device; The processor is used to perform a first operation according to a processing strategy of the second computing device when the processing strategy of the second computing device about the first message is received.

8. A second computing device, comprising: The second computing device and the first computing device constitute a cluster; the second computing device comprises a processor; The processor is used to determine the first computing device with abnormal communication with the second computing device; The processor is further used to perform a third operation according to the number of the first computing device with abnormal communication; The processor is specifically used to determine a processing strategy according to service information of the first computing device with abnormal communication when the number of the first computing device with abnormal communication is less than the number of the first computing device in the cluster, and a first message sent by the first computing device with abnormal communication is not received within a fifth preset time after the first computing device with abnormal communication is determined; wherein the first message is used to request the second computing device to indicate how to process the abnormal communication; When the number of the first computing device with abnormal communication is equal to the number of the first computing device in the cluster, the second computing device performs a self-restart operation.

Images