Fault monitoring method and device, computer equipment and storage medium

Abstract

The invention relates to a fault monitoring method and device, computer equipment and a storage medium. The method comprises the following steps: a master node sends a first heartbeat message to each slave node in a server cluster and records first heartbeat sending time, so that first timeout starting time of the master node is determined according to the first heartbeat sending time and a first heartbeat reply returned by each slave node, and the first heartbeat message is sent to each slave node according to the first timeout starting time and a first timeout duration. And monitoring a connection path between the master node and each slave node to obtain a monitoring result. The abnormal state of the connection path can be found earlier, the situation that the connection path is determined to be in the abnormal state after the overtime heartbeat reply is received is avoided, and therefore the timeliness of monitoring the connection path is improved.

Description

Technical Field

[0001] The present application relates to the technical field of distributed systems, and in particular to a fault monitoring method, apparatus, computer equipment and storage medium. Background Art

[0002] With the explosive growth of data volume and the increasing demand for business continuity, existing centralized storage solutions face a series of challenges, including single point failure risks and limited scalability. The server cluster includes master nodes and slave nodes. The slave nodes periodically send heartbeat messages to the master node. Network fluctuations and network card and cable failures may cause the heartbeat messages sent from the slave nodes to the master node to be lost or out of order. It is necessary to monitor the status of the connection path between the master node and the slave node.

[0003] However, in traditional fault monitoring methods, it is difficult to detect critical faults such as split brain in a timely manner, resulting in the risk that the server cluster cannot provide services normally. Summary of the invention

[0004] Based on this, it is necessary to provide a fault monitoring method, device, computer equipment and storage medium that can timely monitor the fault status in response to the above technical problems.

[0005] In a first aspect, the present application provides a fault monitoring method, which is applied to a master node in a server cluster, and includes:

[0006] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0007] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0008] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0009] In one of the embodiments, according to the first timeout starting time and the first timeout duration, a first connection failure between the master node and each slave node is monitored to obtain a monitoring result, including:

[0010] Determine a first monitoring period according to the first timeout starting point time and the first timeout duration;

[0011] Send a second heartbeat message to each slave node and record the time when the second heartbeat was sent;

[0012] According to the second heartbeat replies received from the slave nodes during the first monitoring period, the connection paths between the master node and the slave nodes are monitored to obtain monitoring results.

[0013] In one embodiment, according to the second heartbeat reply received from each slave node during the first monitoring period, the connection path between the master node and each slave node is monitored to obtain the monitoring result, including:

[0014] For each slave node, if a second heartbeat reply returned by the slave node is received within the first monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat reply to obtain a monitoring result;

[0015] If the second heartbeat reply returned from the node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

[0016] In one embodiment, the connection path between the master node and each slave node is monitored according to the validity of the second heartbeat reply, and the monitoring result is obtained:

[0017] If the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state;

[0018] If the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state.

[0019] In one embodiment, determining the first timeout start time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node includes:

[0020] Verifying the first heartbeat reply to obtain a first verification result;

[0021] If the first verification result indicates that the verification of the first heartbeat reply is passed, then determining the first heartbeat sending time as the first timeout starting time;

[0022] If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

[0023] In one embodiment, the first heartbeat message includes first version information and a first sequence number, the first heartbeat reply includes second version information and a second sequence number, and the first heartbeat reply is verified to obtain a first verification result, including:

[0024] Determine whether the length of the first heartbeat response is legal according to the length of the first heartbeat response and the preset length;

[0025] If the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

[0026] In one embodiment, the method further comprises:

[0027] When it is determined that the connection path between the master node and the slave node is in an abnormal state, the master node split-brain operation is performed.

[0028] In a second aspect, the present application provides a fault monitoring method, which is applied to a slave node in a server cluster, and includes:

[0029] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0030] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0031] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0032] In one embodiment, according to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result, including:

[0033] Determine a second monitoring period according to the second timeout starting time and the second timeout duration;

[0034] Receive the second heartbeat message sent by the master node and record the time when the second heartbeat was received;

[0035] The connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result.

[0036] In one embodiment, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result, including:

[0037] If a second heartbeat message sent by the master node is received within the second monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result;

[0038] If the second heartbeat message sent by the master node is not received within the second monitoring period, it is determined that the connection path is in a fault state.

[0039] In one embodiment, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result, including:

[0040] If the second heartbeat message is a valid message, it is determined that the connection path is in a normal state;

[0041] If the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0042] In one embodiment, determining a second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message includes:

[0043] Verifying the first heartbeat message to obtain a second verification result;

[0044] If the second verification result indicates that the verification of the first heartbeat message is passed, the first heartbeat reception time is determined as the second timeout starting time;

[0045] If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

[0046] In one embodiment, the first heartbeat message includes the first version information and the first sequence number, and the first heartbeat message is verified to obtain a second verification result, including:

[0047] Determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range;

[0048] If the first version information is within the first preset range and the first sequence number is within the second preset range, the first version information and the first sequence number are verified to obtain a second verification result of the first heartbeat message.

[0049] In one embodiment, verifying the first version information and the first sequence number to obtain a second verification result of the first heartbeat message includes:

[0050] Determine whether the first version information is smaller than the historical version information pre-stored in the slave node;

[0051] If it is less than, the second verification result is determined to be a verification failure;

[0052] If it is greater, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed;

[0053] If they are equal, determine whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is not less than the serial number pre-stored in the slave node, update the serial number pre-stored in the slave node to the first serial number, and determine that the second verification result is verification passed.

[0054] In one embodiment, the method further comprises:

[0055] When it is determined that the connection path between the master node and the slave node is in an abnormal state, a slave node split-brain operation is performed.

[0056] In a third aspect, the present application further provides a fault monitoring device, comprising:

[0057] A sending module is used to send a heartbeat message to a slave node in a server cluster and record the heartbeat sending time; the first heartbeat message includes the first version information and the first sequence number corresponding to the master node;

[0058] A first determination module, used for determining a first timeout starting time of the master node according to a first heartbeat sending time and a first heartbeat reply returned by each slave node;

[0059] The first monitoring module is used to monitor the first connection failure between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result.

[0060] In a fourth aspect, the present application further provides a fault monitoring device, comprising:

[0061] A receiving module, used for receiving a first heartbeat message sent by a master node in a server cluster, and recording a first heartbeat receiving time;

[0062] A second determination module is used to determine the second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message, and send a first heartbeat reply to the master node;

[0063] The second monitoring module is used to monitor the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result.

[0064] In a fifth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0065] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0066] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0067] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0068] In a sixth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0069] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0070] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0071] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0072] In a seventh aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0073] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0074] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0075] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0076] In an eighth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0077] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0078] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0079] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0080] In a ninth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0081] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0082] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0083] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0084] In a tenth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0085] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0086] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0087] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0088] In the above fault monitoring method, device, computer equipment and storage medium, the master node sends a first heartbeat message to each slave node in the server cluster and records the first heartbeat sending time, thereby determining the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node, and then monitoring the connection path between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result. The first timeout starting time of the master node is determined in real time through the first heartbeat sending time and the first heartbeat reply, which improves the flexibility and accuracy of determining the first timeout starting time. Further, the connection path between the master node and each slave node is monitored according to the first timeout starting time and the first timeout duration, so that the abnormal state of the connection path can be discovered earlier, avoiding determining that the connection path is in an abnormal state after receiving a timed-out heartbeat reply, thereby improving the timeliness of monitoring the connection path. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0090] FIG1 is an application environment diagram of a fault monitoring method in one embodiment;

[0091] Figure 2 A schematic diagram of a flow chart of a fault monitoring method in an embodiment;

[0092] Figure 3 is a flow chart of a fault monitoring method in another embodiment;

[0093] Figure 4 is a flow chart of a fault monitoring method in another embodiment;

[0094] Figure 5 is a flow chart of a fault monitoring method in another embodiment;

[0095] Figure 6 is a flow chart of a fault monitoring method in another embodiment;

[0096] Figure 7 is a flow chart of a fault monitoring method in another embodiment;

[0097] Figure 8 is a flow chart of a fault monitoring method in another embodiment;

[0098] Fig. 9 is a flow chart of a fault monitoring method in another embodiment;

[0099] Fig.10 is a flow chart of a fault monitoring method in another embodiment;

[0100] Fig.11 is a flow chart of a fault monitoring method in another embodiment;

[0101] Fig.12 is a flow chart of a fault monitoring method in another embodiment;

[0102] Fig.13 is a flow chart of a fault monitoring method in another embodiment;

[0103] Fig.14 is a flow chart of a fault monitoring method in another embodiment;

[0104] Fig.15 is a flow chart of a fault monitoring method in another embodiment;

[0105] Fig.16 is a flow chart of a fault monitoring method in another embodiment;

[0106] Fig.17 is a flow chart of a fault monitoring method in another embodiment;

[0107] Fig.18 is a structural block diagram of a fault monitoring device in one embodiment;

[0108] Fig.19 is a structural block diagram of a fault monitoring device in another embodiment;

[0109] FIG. 20 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0111] Centralized storage arrays provide users with data redundancy and fast access through efficient data management technologies, such as RAID configuration. However, with the explosive growth of data volume and the increase in business continuity requirements, existing centralized storage solutions face a series of challenges, including single point failure risks and limited scalability. In traditional technologies, fault recovery based on consensus strategies relies on the master node to maintain the global state and periodically monitor the state of other nodes. The cluster requires at least one master node, one computing node, and one node that observes and destroys the computing node. During the periodic monitoring of the master node, the computing node periodically sends heartbeat messages to the master node, and the master node maintains the state. However, if there are network fluctuations and network card or cable failures during this process, the heartbeat messages sent by the computing node to the master node may be lost or out of order, which may cause the master node to unexpectedly identify the computing node for destruction, resulting in the unexpected shutdown of the computing node, and the server cluster may be at risk of not being able to provide services normally.

[0112] Based on this, this solution provides a fault monitoring method to reduce the risk that the server cluster cannot provide services normally.

[0113] The fault monitoring method provided in the embodiment of the present application can be applied in the application environment as shown in Figure 1. Among them, the master node 102 in the server cluster communicates with the slave node 104 in the server cluster. The master node 102 sends a first heartbeat message to each slave node 104, and receives a first heartbeat reply returned by the slave node 104. Among them, the master node 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The slave node 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0114] In one embodiment, Figure 2 As shown, a fault monitoring method is provided, which is described by taking the method applied to the master node in FIG1 as an example, including:

[0115] S201, sending a first heartbeat message to each slave node in the server cluster, and recording the time when the first heartbeat was sent.

[0116] The first heartbeat message may be a data packet used to detect and maintain the reliability of a connection, and the first heartbeat message may be transmitted based on a TCP / IP protocol.

[0117] In an embodiment of the present application, a server cluster includes multiple servers, each server is a node, a master node is obtained by election of each node in the server cluster, and other nodes in the server cluster except the master node are slave nodes. The master node determines the timing of sending the first heartbeat message according to a preset sending mechanism, thereby sending the first heartbeat message to each slave node in the server cluster, and records the sending time as the first heartbeat sending time. Optionally, the preset sending mechanism can allow the master node to send a heartbeat message to each slave node at a preset interval; or, the preset sending mechanism can allow the master node to send a heartbeat message to each slave node when receiving a trigger signal.

[0118] S202, determining a first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node.

[0119] In an embodiment of the present application, the slave node returns a first heartbeat reply to the master node in response to the first heartbeat message, so that the master node can determine the first timeout starting time of the master node based on the first heartbeat sending time and the first heartbeat reply.

[0120] As an optional implementation, the master node can parse the first heartbeat reply to obtain the reply duration corresponding to the first heartbeat reply, thereby determining the first timeout starting time of the master node according to the reply duration and the first heartbeat sending time. For example, the sum of the reply duration and the first heartbeat sending time is determined as the first timeout starting time; or, when the reply duration is within the first preset duration range, the first heartbeat sending time is determined as the first timeout starting time of the master node, and when the reply duration is outside the first preset duration range, the first heartbeat reply is determined to be in a reply timeout state, and the initial state of the first timeout starting time is maintained.

[0121] As another optional implementation, the master node verifies the first heartbeat reply. If the verification is successful, the first heartbeat sending time is determined as the first timeout starting time; if the verification is not successful, the initial state of the first timeout starting time is maintained.

[0122] S203: Monitor the connection path between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result.

[0123] In an embodiment of the present application, the first timeout starting time is used as the monitoring start time, and whether the connection between the master node and the slave node is normal within the first timeout period is monitored to obtain a monitoring result.

[0124] As an optional implementation, the master node can monitor the connection path between the master node and each slave node to obtain a monitoring result by monitoring whether a reply message sent by the slave node is received within the first timeout period; wherein the reply message sent by the slave node can be a second heartbeat reply or other message sent by the slave node.

[0125] As another optional implementation, the master node may determine the first timeout end time based on the first timeout start time and the first timeout duration, and when the current moment is the first timeout end time, determine whether a reply message sent by the slave node is received between the first timeout start time and the first timeout end time; if received, the monitoring result is that the connection path is in a normal state; if not received, the monitoring result is that the connection path is in an abnormal state.

[0126] In the above fault monitoring method, the master node sends a first heartbeat message to each slave node in the server cluster, and records the first heartbeat sending time, thereby determining the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node, and then monitoring the connection path between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result. The first timeout starting time of the master node is determined in real time through the first heartbeat sending time and the first heartbeat reply, which improves the flexibility and accuracy of determining the first timeout starting time. Further, according to the first timeout starting time and the first timeout duration, the connection path between the master node and each slave node is monitored, and the abnormal state of the connection path can be discovered earlier, avoiding determining that the connection path is in an abnormal state after receiving the timeout heartbeat reply, thereby improving the timeliness of monitoring the connection path.

[0127] In one embodiment, an implementation of the above S203 is provided, such as Figure 3 As shown, the above “monitoring the first connection failure between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result” includes:

[0128] S301: Determine a first monitoring period according to a first timeout starting time and a first timeout duration.

[0129] In an embodiment of the present application, the moment corresponding to the first timeout starting time is determined, and the first monitoring period is determined with the moment as the starting point and the first timeout duration as the time length, that is, the first monitoring period is the first timeout duration starting from the first timeout starting time.

[0130] S302, sending a second heartbeat message to each slave node, and recording the time when the second heartbeat was sent.

[0131] In an embodiment of the present application, the master node determines the timing of sending the second heartbeat message according to a preset sending mechanism, thereby sending the second heartbeat message to each slave node in the server cluster, and records the sending time as the second heartbeat sending time. Optionally, the time difference between the first heartbeat sending time and the second heartbeat sending time can be a preset interval duration.

[0132] S303: Monitor the connection path between the master node and each slave node according to the second heartbeat reply received from each slave node during the first monitoring period to obtain a monitoring result.

[0133] In an embodiment of the present application, the situation of receiving the second heartbeat reply returned by each slave node during the first monitoring period may include at least one of whether the second heartbeat reply is received, the number of second heartbeat replies, and the time of receiving the second heartbeat reply, so as to monitor the connection path between the master node and each slave node according to the situation of receiving the second heartbeat reply returned by each slave node during the first monitoring period. It should be noted that the state of the connection path between each slave node and the master node may be the same or different. For example, the server cluster may include slave node 1, slave node 2, and slave node 3. The monitoring results may include that the connection path between the master node and slave node 1 is in a normal state, the connection path between the master node and slave node 2 is in a normal state, and the connection path between the master node and slave node 3 is in an abnormal state.

[0134] Optionally, determine the number of second heartbeat replies received by the master node during the first monitoring period to determine whether the number of replies is greater than a preset threshold. If so, the monitoring result is that the connection path is in a normal state; if so, the monitoring result is that the connection path is in an abnormal state.

[0135] In the above application embodiment, the first monitoring period is determined based on the first timeout starting time and the first timeout duration, so that the determination of the first monitoring period is more flexible and accurate. Furthermore, the second heartbeat replies returned by each slave node received during the first monitoring period are analyzed, and the monitoring results of the connection path between the master node and each slave node can be obtained in time.

[0136] In one embodiment, an implementation of the above S303 is provided, such as Figure 4As shown, the above “monitoring the connection path between the master node and each slave node according to the second heartbeat reply received from each slave node during the first monitoring period to obtain the monitoring result” includes:

[0137] S401, for each slave node, determine whether a second heartbeat reply returned by the slave node is received within a first monitoring period, if so, execute S402; if not, execute S403.

[0138] S402, monitoring the connection path between the master node and the slave node according to the validity of the second heartbeat reply to obtain a monitoring result.

[0139] S403, determining that the connection path is in a fault state.

[0140] In an embodiment of the present application, for each slave node in the server cluster, the slave node can verify the second heartbeat message in response to the second heartbeat message sent by the master node, and return the second heartbeat reply to the master node when the second heartbeat message is verified. If the master node receives the second heartbeat reply returned by the slave node within the first monitoring period, it indicates that the slave node has passed the verification of the second heartbeat message, and the connection between the master node and the slave node is normal, and information exchange can be performed, then the second heartbeat reply is further tested for validity, and the monitoring result corresponding to the connection path is determined according to the test result. If the master node does not receive the second heartbeat reply returned by the slave node within the first monitoring period, it indicates that the slave node has failed to verify the second heartbeat message, and / or the connection between the master node and the slave node is abnormal, wherein the reason for the failure of the slave node to verify the second heartbeat message can be the message transmission abnormality caused by the abnormal connection path, then the connection path is in a fault state.

[0141] As an optional implementation, the validity test of the second heartbeat reply can be performed to determine whether the length of the second heartbeat reply is the same as the preset length. If they are the same, the monitoring result is that the connection path is in a normal state; if they are not the same, the monitoring result is that the connection path is in an abnormal state.

[0142] As another optional implementation, the validity test of the second heartbeat reply can be performed by the master node parsing the second heartbeat reply received and returned from the slave node during the first monitoring period, obtaining parsed information corresponding to the second heartbeat reply, and determining whether the parsed information is identical to the slave node information pre-stored in the master node; if they are identical, the monitoring result is that the connection path is in a normal state; if they are different, the monitoring result is that the connection path is in an abnormal state.

[0143] In the above application embodiment, for each slave node, the second heartbeat reply returned by the slave node to the master node within the first monitoring period is determined, and the monitoring process is accurately within the first monitoring period, thereby improving the timeliness of determining the fault status of the connection path.

[0144] In one embodiment, an implementation of the above S402 is provided, such as Figure 5 As shown, the above “monitoring the connection path between the master node and each slave node according to the validity of the second heartbeat reply to obtain the monitoring result” includes:

[0145] S501: If the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state.

[0146] The second heartbeat message and the master node both include the third version information and the third sequence number, and the second heartbeat reply includes the fourth version information and the fourth sequence number. It should be noted that the version information can be used to indicate the election coefficient of the server cluster, and the sequence number can be used to indicate the number of times the current master node sends heartbeat information.

[0147] In an embodiment of the present application, it is determined whether the length of the heartbeat reply is legal based on the length of the second heartbeat reply and a preset length; if the length is legal, it is determined whether the fourth version information is consistent with the third version information; if the fourth version information is consistent with the third version information, it is determined whether the fourth serial number is consistent with the third serial number; if the fourth serial number is consistent with the third serial number, it is determined that the first verification result is verification passed.

[0148] S502: If the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state.

[0149] In an embodiment of the present application, if the length of the second heartbeat reply is illegal, the second heartbeat reply is an invalid reply, and the connection path is determined to be in an abnormal state; if the length of the second heartbeat reply is legal, and the fourth version information and the third version information are inconsistent, the second heartbeat reply is an invalid reply, and the connection path is determined to be in an abnormal state; if the length of the second heartbeat reply is legal, and the fourth version information and the third version information are consistent, and the fourth sequence number and the third sequence number are inconsistent, the second heartbeat reply is an invalid reply, and the connection path is determined to be in an abnormal state.

[0150] In the above application embodiment, the legitimacy of the second heartbeat reply is tested, so that the status of the connection path is determined based on the validity of the second heartbeat reply, thereby improving the accuracy of monitoring the connection path. The legitimacy test of the second heartbeat reply is relatively simple and fast, ensuring the timeliness of monitoring the connection path.

[0151] In one embodiment, an implementation of the above S202 is provided, such as Figure 6As shown, the above “determining the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node” includes:

[0152] S601, verify the first heartbeat reply to obtain a first verification result.

[0153] In an embodiment of the present application, the first heartbeat reply is parsed to obtain reply information corresponding to the slave node, and then the first heartbeat reply is verified according to the reply information corresponding to the slave node and preset slave node information to obtain a first verification result.

[0154] Optionally, the master node can determine whether the reply information corresponding to the slave node is consistent with the preset slave node information. If they are consistent, it is determined that the first verification result indicates that the verification of the first heartbeat reply is passed; if they are inconsistent, it is determined that the first verification result indicates that the verification of the first heartbeat reply is not passed. Alternatively, the master node can determine the similarity between the reply information corresponding to the slave node and the preset slave node information. If the similarity is greater than a preset similarity threshold, it is determined that the first verification result indicates that the verification of the first heartbeat reply is passed; if the similarity is not greater than the preset similarity threshold, it is determined that the first verification result indicates that the verification of the first heartbeat reply is not passed.

[0155] S602: If the first verification result indicates that the verification of the first heartbeat reply is passed, determine the first heartbeat sending time as the first timeout starting time.

[0156] In an embodiment of the present application, if the verification of the first heartbeat reply is passed, the first interactive communication between the master node and the slave node is normal, that is, the connection between the master node and the slave node is normal before the first heartbeat sending time, so the first heartbeat sending time is determined as the first timeout starting time.

[0157] S603: If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

[0158] In an embodiment of the present application, if the verification of the first heartbeat reply fails, the communication of the first interaction between the master node and the slave node is abnormal, for example, the master node fails to send the first heartbeat message, the slave node fails to send the first heartbeat reply, the slave node fails to verify the first heartbeat message, etc., then the first timeout starting time is not modified, and the original initial value of the first timeout starting time is maintained. Optionally, after the verification of the first heartbeat reply fails, the above-mentioned steps of sending the first heartbeat message to each slave node in the server cluster and recording the time when the first heartbeat is sent can be returned.

[0159] In the above application embodiment, based on the first verification result of the first heartbeat reply, it is determined whether to determine the first heartbeat sending time as the first timeout starting time, thereby improving the accuracy of the first timeout starting time and making the monitoring of the connection path between the master node and the slave node more flexible and accurate.

[0160] In one embodiment, an implementation of the above S601 is provided, such as Figure 7 As shown, the first heartbeat message includes the first version information and the first sequence number, the first heartbeat reply includes the second version information and the second sequence number, and the above “verifying the first heartbeat reply to obtain the first verification result” includes:

[0161] S701, determining whether the length of the first heartbeat reply is legal according to the length of the first heartbeat reply and a preset length.

[0162] Among them, the first version information is used to indicate the election coefficient of the server cluster recorded by the master node, and the first serial number is used to indicate the number of times the master node sends heartbeat information after the election is successful; the second version information indicates the election coefficient of the server cluster recorded by the slave node, and the second serial number indicates the number of times the slave node receives heartbeat information.

[0163] In an embodiment of the present application, the first heartbeat reply is analyzed to obtain the length of the first heartbeat reply, and it is determined whether the length of the first heartbeat reply is the same as the preset length. If they are the same, the length of the first heartbeat reply is legal, and the information carried by the first heartbeat reply is further verified; if they are not the same, the length of the first heartbeat reply is illegal, and the first verification result is determined to be verification failure.

[0164] S702, if the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

[0165] In an embodiment of the present application, when the length of the first heartbeat reply is legal, the second version information and the second sequence number carried in the first heartbeat reply are verified to obtain a first verification result. It should be noted that the first version information and the first sequence number are pre-stored in the master node, and the first heartbeat message includes the first version information and the first sequence number.

[0166] In the embodiment of the present application, the second version information in the first heartbeat reply is first verified to determine whether the second version information is consistent with the first version information. If they are consistent, the second sequence number in the first heartbeat reply is verified; if they are inconsistent, the first verification result is verification failure. Further, when the second version information is consistent with the first version information, it is determined whether the second sequence number is consistent with the first sequence number. If they are consistent, the first verification result is determined to be verification passed; if they are inconsistent, the first verification result is determined to be verification failure.

[0167] In the above application embodiment, the length of the first heartbeat reply, the second version information and the second serial number in the first heartbeat reply are verified in sequence, and the accuracy of the first verification result is improved by performing multiple verifications.

[0168] In one embodiment, Figure 8 As shown, the above fault monitoring method also includes:

[0169] S204: When it is determined that the connection path between the master node and the slave node is in an abnormal state, a master node split-brain operation is performed.

[0170] In the embodiment of the present application, when the master node determines that the connection path between the master node and the slave node is in an abnormal state, the master node performs a master node split brain operation to disconnect the connection between the master node and the slave node. Further, the slave node does not receive a second heartbeat message within a second timeout period starting from a second timeout start time, the slave node determines that the connection path with the master node is abnormal, and performs a slave node split brain operation.

[0171] Optionally, the second timeout period may be greater than or equal to the first timeout period.

[0172] In the above application embodiment, since the first timeout starting time of the master node is determined according to the first heartbeat sending time, and the second timeout starting time of the slave node is determined according to the first heartbeat receiving time, the first timeout starting time is earlier than the second timeout starting time, and the master node split brain operation is performed when the connection path is in an abnormal state, which can ensure that the master node split brain is earlier than the slave node split brain, thereby reducing the risk of the server cluster failing to provide services normally.

[0173] In one embodiment, Fig. 9 As shown, a fault monitoring method is provided, which is described by taking the method applied to the slave node in FIG1 as an example, including:

[0174] S801, receiving a first heartbeat message sent by a master node in a server cluster, and recording a time when the first heartbeat is received.

[0175] The first heartbeat message may be a data packet used to detect and maintain the reliability of a connection, and the first heartbeat message may be transmitted based on a TCP / IP protocol.

[0176] In an embodiment of the present application, the master node determines the timing of sending the first heartbeat message according to a preset sending mechanism, thereby sending the first heartbeat message to the slave node in the server cluster, and records the first heartbeat reception time when the slave node receives the first heartbeat message.

[0177] Optionally, the preset sending mechanism may allow the master node to send a heartbeat message to each slave node at a preset interval; or, the preset sending mechanism may allow the master node to send a heartbeat message to each slave node when receiving a trigger signal.

[0178] S802: Determine the second timeout start time of the slave node according to the first heartbeat reception time and the first heartbeat message, and send a first heartbeat reply to the master node.

[0179] As an optional implementation, the slave node can parse the first heartbeat message to obtain the adjustment duration corresponding to the first heartbeat message, and thereby determine the second timeout starting time of the slave node based on the adjustment duration and the first heartbeat reception time; or, when the adjustment duration is within the second preset duration range, the first heartbeat reception time is determined as the second timeout starting time of the slave node; when the adjustment duration is outside the second preset duration range, the sum of the first heartbeat reception time and the adjustment duration is used as the second timeout starting time, and a first heartbeat reply is generated based on the information in the slave node, and the first heartbeat reply is sent to the master node.

[0180] As another optional implementation, the slave node verifies the first heartbeat message. If the verification is successful, the first heartbeat reception time is determined as the second timeout starting time; if the verification is not successful, the initial state of the second timeout starting time is maintained.

[0181] S803: Monitor the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result.

[0182] In an embodiment of the present application, the slave node uses the second timeout starting time as the monitoring start time, monitors whether the connection between the master node and the slave node is normal within the second timeout period, and obtains the monitoring result.

[0183] As an optional implementation, the slave node may monitor the connection path between the master node and the slave node by monitoring whether the next heartbeat message sent by the master node is received within the second timeout period to obtain a monitoring result.

[0184] As another optional implementation, the slave node can determine the second timeout end time based on the second timeout start time and the second timeout duration. When the current moment is the second timeout end time, it determines whether the next heartbeat message sent by the master node is received between the second timeout start time and the second timeout end time. If received, the monitoring result is that the connection path is in a normal state; if not received, the monitoring result is that the connection path is in an abnormal state.

[0185] In the above fault monitoring method, a first heartbeat message is sent to each slave node in the server cluster, and the first heartbeat sending time is recorded; according to the first heartbeat sending time and the first heartbeat reply returned by each slave node, the first timeout starting time of the master node is determined; according to the first timeout starting time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result. Through the first heartbeat receiving time, the second timeout starting time of the slave node is determined in real time, which improves the flexibility and accuracy of determining the second timeout starting time. Further, since the first heartbeat receiving time is later than the first heartbeat sending time, the second timeout starting time determined according to the first heartbeat receiving time is later than the first timeout starting time. According to the first timeout starting time and the first timeout duration, the connection path between the master node and each slave node is monitored to avoid the slave node detecting the abnormal state of the connection path earlier than the master node, thereby reducing the risk that the server cluster cannot provide services normally.

[0186] In one embodiment, an implementation of the above S801 is provided, such as Fig.10 As shown, the above “monitoring the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result” includes:

[0187] S901: Determine a second monitoring period according to a second timeout starting time and a second timeout duration.

[0188] In an embodiment of the present application, the moment corresponding to the second timeout starting time is determined, and the second monitoring period is determined with the moment as the starting point and the second timeout duration as the time length, that is, the second monitoring period is the second timeout duration starting from the second timeout starting time.

[0189] S902, receiving the second heartbeat message sent by the master node, and recording the second heartbeat receiving time.

[0190] In an embodiment of the present application, the master node determines the timing of sending the second heartbeat message according to a preset sending mechanism, thereby sending the second heartbeat message to the slave nodes in the server cluster. When the slave node receives the second heartbeat message, the second heartbeat reception time is recorded, and a second heartbeat reply is sent to the master node based on the second heartbeat message and the information of the slave node.

[0191] S903, monitoring the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result.

[0192] In an embodiment of the present application, the situation of receiving the second heartbeat message sent by the master node during the second monitoring period may include at least one of whether the second heartbeat message is received, the number of second heartbeat messages, and the time of receiving the second heartbeat message, thereby monitoring the connection path between the master node and the slave node based on the situation of receiving the second heartbeat message sent by the master node during the second monitoring period.

[0193] Optionally, determine the number of second heartbeat messages received from the node during the second monitoring period to determine whether the number of messages is greater than a preset threshold. If so, the monitoring result is that the connection path is in a normal state; if so, the monitoring result is that the connection path is in an abnormal state.

[0194] In the above application embodiment, the second monitoring period is determined based on the second timeout starting time and the second timeout duration, so that the determination of the second monitoring period is more flexible and accurate. Furthermore, the second heartbeat message sent by the master node received during the second monitoring period is analyzed, and the monitoring result of the connection path between the slave node and the master node can be obtained in time.

[0195] In one embodiment, an implementation of the above S903 is provided, such as Fig.11 As shown, the above “monitoring the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result” includes:

[0196] S1001, determine whether a second heartbeat message sent by the master node is received within a second monitoring period, if so, execute S1002; if not, execute S1003.

[0197] S1002, monitoring the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result.

[0198] S1003, determining that the connection path is in a fault state.

[0199] In an embodiment of the present application, if the slave node receives the second heartbeat message sent by the master node during the second monitoring period, it indicates that the connection between the master node and the slave node is normal and information exchange can be performed, and then the second heartbeat message is further tested for validity, and the monitoring result corresponding to the connection path is determined according to the test result. If the slave node does not receive the second heartbeat message sent by the master node during the second monitoring period, it indicates that the connection between the master node and the slave node is abnormal, and the connection path is in a faulty state.

[0200] As an optional implementation, the validity test of the second heartbeat message can be performed to determine whether the length of the second heartbeat message is the same as the preset length. If they are the same, the monitoring result is that the connection path is in a normal state; if they are not the same, the monitoring result is that the connection path is in an abnormal state.

[0201] As another optional implementation, the validity test of the second heartbeat message can be performed by the master node parsing the second heartbeat message received during the second monitoring period, obtaining parsed information corresponding to the second heartbeat message, and determining whether the parsed information is identical to the master node information pre-stored in the slave node; if they are identical, the monitoring result is that the connection path is in a normal state; if they are different, the monitoring result is that the connection path is in an abnormal state.

[0202] In the above application embodiment, the second heartbeat message sent by the master node to the slave node within the second monitoring period is determined, and the monitoring process is accurately adjusted to the second monitoring period, thereby improving the timeliness of determining the fault status of the connection path.

[0203] In one embodiment, an implementation of the above S1002 is provided, such as Fig.12 As shown, the above “monitoring the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result” includes:

[0204] S1101: If the second heartbeat message is a valid message, it is determined that the connection path is in a normal state.

[0205] The second heartbeat message and the master node both include the third version information and the third sequence number, and the slave node pre-stores the fourth version information and the fourth sequence number.

[0206] In an embodiment of the present application, when it is determined that the third version information in the second heartbeat message is within the first preset range, and when it is determined that the third sequence number is within the second preset range, the validity of the third version information is judged based on the fourth version information, and the validity of the third sequence number is judged based on the fourth sequence number. For example, when the fourth version information is the same as the third version information, and the fourth sequence number is the same as the third sequence number, the second heartbeat message is valid information, and it is determined that the connection path between the slave node and the master node is in normal state.

[0207] S1102: If the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0208] In this embodiment, if the second heartbeat message is an invalid message, the connection between the slave node and the master node is abnormal, and data transmission cannot be performed accurately, and it is determined that the connection path is in an abnormal state.

[0209] In the above application embodiment, the legitimacy of the second heartbeat message is tested, so that the status of the connection path is determined based on the validity of the second heartbeat message, thereby improving the accuracy of monitoring the connection path. The legitimacy test of the second heartbeat message is relatively simple and fast, ensuring the timeliness of monitoring the connection path.

[0210] In one embodiment, an implementation of the above S902 is provided, such as Fig.13 As shown, the above “determining the second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message” includes:

[0211] S1201, verify the first heartbeat message to obtain a second verification result.

[0212] In an embodiment of the present application, the first heartbeat message is parsed to obtain parsing information corresponding to the first heartbeat message, and then the first heartbeat message is verified according to the parsing information corresponding to the first heartbeat message and preset parsing information to obtain a second verification result.

[0213] Optionally, the slave node can determine whether the parsing information corresponding to the first heartbeat message is consistent with the preset parsing information. If they are consistent, it is determined that the second verification result indicates that the verification of the second heartbeat reply is passed; if they are inconsistent, it is determined that the second verification result indicates that the verification of the second heartbeat reply is not passed. Alternatively, the slave node can determine the similarity between the parsing information corresponding to the first heartbeat message and the preset parsing information. If the similarity is greater than a preset similarity threshold, it is determined that the second verification result indicates that the verification of the second heartbeat reply is passed; if the similarity is not greater than the preset similarity threshold, it is determined that the second verification result indicates that the verification of the second heartbeat reply is not passed.

[0214] S1202: If the second verification result indicates that the verification of the first heartbeat message is passed, determine the first heartbeat reception time as the second timeout start time.

[0215] In an embodiment of the present application, if the verification of the first heartbeat message passes, the first interactive communication between the master node and the slave node is normal, that is, the connection between the master node and the slave node before the first heartbeat reception time is normal, so the first heartbeat reception time is determined as the second timeout starting time.

[0216] S1203: If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

[0217] In an embodiment of the present application, if the verification of the first heartbeat message fails, the first interactive communication between the master node and the slave node is abnormal, the second timeout starting time is not modified, and the original initial value of the second timeout starting time is maintained.

[0218] In the above-mentioned application embodiment, based on the second verification result obtained by verifying the first heartbeat message, it is determined whether to determine the first heartbeat reception time as the second timeout starting time, thereby improving the accuracy of the second timeout starting time and making the monitoring of the connection path between the master node and the slave node more flexible and accurate.

[0219] In one embodiment, an implementation of the above S1201 is provided, where the first heartbeat message includes the first version information and the first sequence number, such as Fig.14 As shown, the above “verifying the first heartbeat message to obtain a second verification result” includes:

[0220] S1301, determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range.

[0221] The first preset range may be a preset version range; the second preset range may be a preset number range.

[0222] In the embodiment of the present application, the first upper limit value and the first lower limit value of the first preset range are determined, and the first version information is compared with the first upper limit value, and the first version information is compared with the first lower limit value. If the first version information is greater than the first lower limit value and less than the first upper limit value, the first version information is within the first preset range; if the first version information is not greater than the first lower limit value or the first version information is not less than the first upper limit value, the first version information is not within the first preset range. And the second upper limit value and the second lower limit value of the second preset range are determined, and the first sequence number is compared with the second upper limit value, and the first sequence number is compared with the second lower limit value. If the first sequence number is greater than the second lower limit value and less than the second upper limit value, the first sequence number is within the second preset range; if the first sequence number is not greater than the second lower limit value or the second version information is not less than the second upper limit value, the first sequence number is not within the first preset range.

[0223] S1302: If the first version information is within a first preset range and the first sequence number is within a second preset range, verify the first version information and the first sequence number to obtain a second verification result of the first heartbeat message.

[0224] In an embodiment of the present application, when the first version information is within a first preset range and the first sequence number is within a second preset range, the first version information and the first sequence number are further verified to determine the validity of the first heartbeat message.

[0225] Optionally, it can be determined whether the first version information is consistent with the preset version information in the slave node, and whether the first sequence number is consistent with the preset sequence number in the slave node. If the first version information is consistent with the preset version information, and the first sequence number is consistent with the preset sequence number, the first heartbeat message is determined to be valid; if the first version information is inconsistent with the preset version information, and / or the first sequence number is inconsistent with the preset sequence number, the first heartbeat message is determined to be invalid.

[0226] In the above application embodiment, the first version information and the first serial number are first verified. If the first verification passes, the first version information and the first serial number are then verified for a second time to obtain a second verification result of the first heartbeat message, thereby improving the accuracy of the verification result. In addition, the first verification is relatively simple, and the verification efficiency of the first heartbeat message can be improved when the first heartbeat message is invalid.

[0227] In one embodiment, an implementation of the above S1302 is provided, such as Fig.15 As shown, the above “verifying the first version information and the first serial number to obtain a second verification result of the first heartbeat message” includes:

[0228] S1401, determine whether the first version information is smaller than the historical version information pre-stored in the slave node.

[0229] As an optional implementation, the slave node can determine the ratio of the first version information to the historical version information. If the ratio is less than 1, the first version information is less than the historical version information; if the ratio is equal to 1, the first version information is equal to the historical version information; if the ratio is greater than 1, the first version information is greater than the historical version information.

[0230] As another optional implementation, the slave node can determine the difference between the first version information and the historical version information. If the difference is less than 0, the first version information is less than the historical version information; if the difference is equal to 0, the first version information is equal to the historical version information; if the difference is greater than 0, the first version information is greater than the historical version information.

[0231] S1402: If it is less than, determine that the second verification result is a verification failure.

[0232] In an embodiment of the present application, if the first version information is smaller than the historical version information, indicating that the version information of the master node sending the first heartbeat message is smaller than the version information of the historical master node, the master node is an invalid master node, and the second verification result is determined to be verification failure.

[0233] S1403: If it is greater than, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed.

[0234] In an embodiment of the present application, if the first version information is greater than the historical version information, indicating that the current master node has undergone a master node brain split and re-initiated an election to become the master node, the historical version information in the slave node is updated to the first version information, and the pre-stored sequence number is updated to the first sequence number. Optionally, the first sequence number can be 0, which means that the master node sends the heartbeat information for the first time after the re-election, and the slave node receives the heartbeat information for the first time after the re-election.

[0235] S1404, if equal, determine whether the first sequence number is less than the sequence number pre-stored in the slave node; if the first sequence number is not less than the sequence number pre-stored in the slave node, update the sequence number pre-stored in the slave node to the first sequence number, and determine that the second verification result is verification passed.

[0236] In an embodiment of the present application, if the first version information is equal to the historical version information, indicating that the historical version information is the version information corresponding to the master node, then it is further determined whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is greater than or equal to the serial number pre-stored in the slave node, indicating that the order in which the slave node receives the heartbeat information is the same as the order in which the master node sends the heartbeat information, then the serial number pre-stored in the node is updated to the first serial number, and the second verification result is determined to be verification failed; if the first serial number is less than the serial number pre-stored in the slave node, the heartbeat message sent by the master node may be in a first-come-first-served situation due to network fluctuations and other reasons, and the first heartbeat message is determined to be invalid, and the second verification result is verification failed.

[0237] In the above application embodiment, the first version information and the first serial number in the first heartbeat message are verified in sequence. By performing multiple verifications, the accuracy of the first verification result is improved, and the situation in which the connection path abnormality between the master node and the slave node cannot be monitored in time due to inaccurate first verification result is reduced.

[0238] In one embodiment, Fig.16 As shown, the above fault monitoring method also includes:

[0239] S804: When it is determined that the connection path between the master node and the slave node is in an abnormal state, a split-brain operation of the slave node is performed.

[0240] In an embodiment of the present application, when the slave node determines that the connection path between the master node and the slave node is in an abnormal state, the slave node performs a slave node split brain operation to disconnect the connection between the slave node and the master node. When the master node determines that the connection path between the master node and the slave node is in an abnormal state, the master node performs a master node split brain operation to disconnect the connection between the master node and the slave node.

[0241] Optionally, the second timeout period may be greater than or equal to the first timeout period.

[0242] In the above application embodiment, since the first timeout starting time of the master node is determined according to the first heartbeat sending time, and the second timeout starting time of the slave node is determined according to the first heartbeat receiving time, the first timeout starting time is earlier than the second timeout starting time, and the master node split brain operation is performed when the connection path is in an abnormal state, which can ensure that the master node split brain is earlier than the slave node split brain, thereby reducing the risk of the server cluster failing to provide services normally.

[0243] In one embodiment, a complete fault monitoring method is also provided, such as Fig.17 As shown, including:

[0244] S1, the master node sends a first heartbeat message to each slave node in the server cluster and records the time when the first heartbeat is sent.

[0245] S2, the slave node records the first heartbeat reception time.

[0246] S3, the slave node determines whether the first version information is within a first preset range, and determines whether the first serial number is within a second preset range.

[0247] S4: If the first version information is within a first preset range and the first serial number is within a second preset range, the slave node determines whether the first version information is smaller than the historical version information pre-stored in the slave node.

[0248] S5, if it is less than, the slave node determines that the second verification result is a verification failure; if it is greater than, the slave node updates the historical version information to the first version information, updates the serial number pre-stored in the slave node to the first serial number, and determines that the second verification result is a verification pass; if it is equal, it determines whether the first serial number is less than the serial number pre-stored in the slave node; when the first serial number is not less than the serial number pre-stored in the slave node, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be a verification pass.

[0249] S6. If the second verification result indicates that the verification of the first heartbeat message is passed, the slave node determines the first heartbeat reception time as the second timeout starting time, and sends the first heartbeat reply to the master node; if the second verification result indicates that the verification of the first heartbeat message is not passed, the original initial value of the second timeout starting time is maintained.

[0250] S7, the slave node sends the first heartbeat reply to the master node.

[0251] S8, the master node determines whether the length of the first heartbeat reply is legal based on the length of the first heartbeat reply and the preset length.

[0252] S9, when the length is legal, the master node determines whether the second version information is consistent with the first version information; when the second version information is consistent with the first version information, determines whether the second serial number is consistent with the first serial number; when the second serial number is consistent with the first serial number, determines that the first verification result is verification passed.

[0253] S10, if the first verification result of the master node indicates that the verification of the first heartbeat reply is passed, the first heartbeat sending time is determined as the first timeout starting time; if the first verification result indicates that the verification of the first heartbeat reply is not passed, the original initial value of the first timeout starting time is maintained.

[0254] S11, the master node determines a first monitoring period according to a first timeout starting time and a first timeout duration.

[0255] S12, the master node sends a second heartbeat message to each slave node and records the time when the second heartbeat is sent.

[0256] S13, for each slave node, if the master node receives a second heartbeat reply from the slave node within the first monitoring period, then when the second heartbeat reply is a valid reply, it determines that the connection path is in a normal state; when the second heartbeat reply is an invalid reply, it determines that the connection path is in an abnormal state.

[0257] S14: If the master node does not receive the second heartbeat reply returned by the slave node within the first monitoring period, it determines that the connection path is in a fault state.

[0258] S15, when the master node determines that the connection path between the master node and the slave node is in an abnormal state, the master node split-brain operation is performed.

[0259] S16, the slave node determines a second monitoring period according to the second timeout starting time and the second timeout duration.

[0260] S17, the slave node receives the second heartbeat message sent by the master node, and records the second heartbeat reception time.

[0261] S18, if the slave node receives a second heartbeat message sent by the master node during the second monitoring period, then when the second heartbeat message is a valid message, it is determined that the connection path is in a normal state; when the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0262] S19: If the slave node does not receive the second heartbeat message sent by the master node within the second monitoring period, it is determined that the connection path is in a fault state.

[0263] S20, when the slave node determines that the connection path between the master node and the slave node is in an abnormal state, the slave node split-brain operation is performed.

[0264] In the above fault monitoring method, the master node sends a first heartbeat message to each slave node in the server cluster, and records the first heartbeat sending time, thereby determining the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node, and then monitoring the connection path between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result. The first timeout starting time of the master node is determined in real time through the first heartbeat sending time and the first heartbeat reply, which improves the flexibility and accuracy of determining the first timeout starting time. Further, according to the first timeout starting time and the first timeout duration, the connection path between the master node and each slave node is monitored, and the abnormal state of the connection path can be discovered earlier, avoiding determining that the connection path is in an abnormal state after receiving the timeout heartbeat reply, thereby improving the timeliness of monitoring the connection path.

[0265] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0266] Based on the same inventive concept, the embodiment of the present application also provides a fault monitoring device for implementing the fault monitoring method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more fault monitoring device embodiments provided below can refer to the limitations of the fault monitoring method above, and will not be repeated here.

[0267] In one embodiment, Fig.18 As shown, a fault monitoring device is provided, comprising: a sending module 10, a first determining module 11 and a first monitoring module 12, wherein:

[0268] The sending module 10 is used to send a first heartbeat message to each slave node in the server cluster and record the first heartbeat sending time.

[0269] The first determining module 11 is used to determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node.

[0270] The first monitoring module 12 is used to monitor the first connection failure between the master node and each slave node according to the first timeout starting time and the first timeout duration to obtain a monitoring result.

[0271] In one embodiment, the first monitoring module 12 includes: a first determination unit, a first recording unit and a first monitoring unit, wherein:

[0272] A first determining unit, configured to determine a first monitoring period according to a first timeout starting time and a first timeout duration;

[0273] A first recording unit, used for sending a second heartbeat message to each slave node and recording a time when the second heartbeat is sent;

[0274] The first monitoring unit is used to monitor the connection path between the master node and each slave node according to the second heartbeat reply received from each slave node within the first monitoring period to obtain a monitoring result.

[0275] In one embodiment, the above-mentioned first monitoring unit is specifically used to monitor the connection path between the master node and the slave node according to the validity of the second heartbeat reply for each slave node to obtain a monitoring result; if the second heartbeat reply returned by the slave node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

[0276] In one embodiment, the first monitoring unit is specifically configured to determine that the connection path is in a normal state if the second heartbeat reply is a valid reply; and to determine that the connection path is in an abnormal state if the second heartbeat reply is an invalid reply.

[0277] In one embodiment, the first determination module 11 includes: a first verification unit, a second determination unit and a third determination unit, wherein:

[0278] The first verification unit is used to verify the first heartbeat reply to obtain a first verification result.

[0279] The second determining unit is used to determine the first heartbeat sending time as the first timeout starting time when the first verification result indicates that the verification of the first heartbeat reply is passed.

[0280] The third determining unit is configured to maintain the original initial value of the first timeout starting time when the first verification result indicates that the verification of the first heartbeat reply has failed.

[0281] In one embodiment, the first heartbeat message includes first version information and a first sequence number, the first heartbeat reply includes second version information and a second sequence number, and the above-mentioned first verification unit is specifically used to determine whether the length of the first heartbeat reply is legal based on the length of the first heartbeat reply and a preset length; if the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second sequence number is consistent with the first sequence number; if the second sequence number is consistent with the first sequence number, determine that the first verification result is verification passed.

[0282] In one embodiment, the fault monitoring device further includes: a first split-brain module, configured to execute a master node split-brain operation when determining that a connection path between the master node and the slave node is in an abnormal state.

[0283] In one embodiment, Fig.19 As shown, a fault monitoring device is provided, comprising: a receiving module 13, a second determining module 14 and a second monitoring module 15, wherein:

[0284] The receiving module 13 is used to receive the first heartbeat message sent by the master node in the server cluster and record the first heartbeat receiving time.

[0285] The second determining module 14 is used to determine the second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message, and send a first heartbeat reply to the master node.

[0286] The second monitoring module 15 is used to monitor the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result.

[0287] In one embodiment, the second monitoring module 15 includes: a fourth determining unit, a second recording unit and a second monitoring unit, wherein:

[0288] The fourth determining unit is used to determine the second monitoring period according to the second timeout starting time and the second timeout duration.

[0289] The second recording unit is used to receive the second heartbeat message sent by the master node and record the second heartbeat reception time.

[0290] The second monitoring unit is used to monitor the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result.

[0291] In one embodiment, the above-mentioned second monitoring unit is specifically used to monitor the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result when a second heartbeat message sent by the master node is received within the second monitoring period; and determine that the connection path is in a fault state when no second heartbeat message sent by the master node is received within the second monitoring period.

[0292] In one embodiment, the second monitoring unit is specifically configured to determine that the connection path is in a normal state if the second heartbeat message is a valid message; and to determine that the connection path is in an abnormal state if the second heartbeat message is an invalid message.

[0293] In one embodiment, the second determination module 14 includes: a second verification unit, a fifth determination unit and a sixth determination unit, wherein:

[0294] The second verification unit is used to verify the first heartbeat message to obtain a second verification result.

[0295] A fifth determining unit, configured to determine, when the second verification result indicates that the verification of the first heartbeat message is passed, to determine the first heartbeat reception time as the second timeout starting time;

[0296] The sixth determining unit is used to maintain the original initial value of the second timeout starting time when the second verification result indicates that the verification of the first heartbeat message has not passed.

[0297] In one embodiment, the above-mentioned second verification unit is specifically used to determine whether the first version information is within a first preset range, and to determine whether the first serial number is within a second preset range; if the first version information is within the first preset range, and the first serial number is within the second preset range, the first version information and the first serial number are verified to obtain a second verification result of the first heartbeat message.

[0298] In one embodiment, the above-mentioned second verification unit is specifically used to determine whether the first version information is less than the historical version information pre-stored in the slave node; if it is less than, the second verification result is determined to be verification failure; if it is greater, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed; if they are equal, it is determined whether the first serial number is less than the serial number pre-stored in the slave node; when the first serial number is not less than the serial number pre-stored in the slave node, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed.

[0299] In one embodiment, the fault monitoring device further includes: a second split-brain module, configured to execute a split-brain operation on the slave node when determining that the connection path between the master node and the slave node is in an abnormal state.

[0300] Each module in the above fault monitoring device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0301] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG20. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store fault monitoring data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a fault monitoring method is implemented.

[0302] Those skilled in the art will understand that the structure shown in FIG. 20 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0303] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0304] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0305] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0306] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0307] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0308] Determine a first monitoring period according to the first timeout starting point time and the first timeout duration;

[0309] Send a second heartbeat message to each slave node and record the time when the second heartbeat was sent;

[0310] According to the second heartbeat replies received from the slave nodes during the first monitoring period, the connection paths between the master node and the slave nodes are monitored to obtain monitoring results.

[0311] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0312] For each slave node, if a second heartbeat reply returned by the slave node is received within the first monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat reply to obtain a monitoring result;

[0313] If the second heartbeat reply returned from the node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

[0314] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0315] If the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state;

[0316] If the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state.

[0317] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0318] Verifying the first heartbeat reply to obtain a first verification result;

[0319] If the first verification result indicates that the verification of the first heartbeat reply is passed, then determining the first heartbeat sending time as the first timeout starting time;

[0320] If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

[0321] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0322] Determine whether the length of the first heartbeat response is legal according to the length of the first heartbeat response and the preset length;

[0323] If the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

[0324] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0325] When it is determined that the connection path between the master node and the slave node is in an abnormal state, the master node split-brain operation is performed.

[0326] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0327] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0328] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0329] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0330] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0331] Determine a second monitoring period according to the second timeout starting time and the second timeout duration;

[0332] Receive the second heartbeat message sent by the master node and record the time when the second heartbeat was received;

[0333] The connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result.

[0334] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0335] If a second heartbeat message sent by the master node is received within the second monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result;

[0336] If the second heartbeat message sent by the master node is not received within the second monitoring period, it is determined that the connection path is in a fault state.

[0337] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0338] If the second heartbeat message is a valid message, it is determined that the connection path is in a normal state;

[0339] If the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0340] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0341] Verifying the first heartbeat message to obtain a second verification result;

[0342] If the second verification result indicates that the verification of the first heartbeat message is passed, the first heartbeat reception time is determined as the second timeout starting time;

[0343] If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

[0344] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0345] Determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range;

[0346] If the first version information is within the first preset range and the first sequence number is within the second preset range, the first version information and the first sequence number are verified to obtain a second verification result of the first heartbeat message.

[0347] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0348] Determine whether the first version information is smaller than the historical version information pre-stored in the slave node;

[0349] If it is less than, the second verification result is determined to be a verification failure;

[0350] If it is greater, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed;

[0351] If they are equal, determine whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is not less than the serial number pre-stored in the slave node, update the serial number pre-stored in the slave node to the first serial number, and determine that the second verification result is verification passed.

[0352] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0353] When it is determined that the connection path between the master node and the slave node is in an abnormal state, a slave node split-brain operation is performed.

[0354] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0355] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0356] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0357] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0358] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0359] Determine a first monitoring period according to the first timeout starting point time and the first timeout duration;

[0360] Send a second heartbeat message to each slave node and record the time when the second heartbeat was sent;

[0361] According to the second heartbeat replies received from the slave nodes during the first monitoring period, the connection paths between the master node and the slave nodes are monitored to obtain monitoring results.

[0362] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0363] For each slave node, if a second heartbeat reply returned by the slave node is received within the first monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat reply to obtain a monitoring result;

[0364] If the second heartbeat reply returned from the node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

[0365] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0366] If the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state;

[0367] If the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state.

[0368] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0369] Verifying the first heartbeat reply to obtain a first verification result;

[0370] If the first verification result indicates that the verification of the first heartbeat reply is passed, then determining the first heartbeat sending time as the first timeout starting time;

[0371] If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

[0372] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0373] Determine whether the length of the first heartbeat response is legal according to the length of the first heartbeat response and the preset length;

[0374] If the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

[0375] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0376] When it is determined that the connection path between the master node and the slave node is in an abnormal state, the master node split-brain operation is performed.

[0377] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0378] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0379] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0380] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0381] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0382] Determine a second monitoring period according to the second timeout starting time and the second timeout duration;

[0383] Receive the second heartbeat message sent by the master node and record the time when the second heartbeat was received;

[0384] The connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result.

[0385] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0386] If a second heartbeat message sent by the master node is received within the second monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result;

[0387] If the second heartbeat message sent by the master node is not received within the second monitoring period, it is determined that the connection path is in a fault state.

[0388] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0389] If the second heartbeat message is a valid message, it is determined that the connection path is in a normal state;

[0390] If the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0391] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0392] Verifying the first heartbeat message to obtain a second verification result;

[0393] If the second verification result indicates that the verification of the first heartbeat message is passed, the first heartbeat reception time is determined as the second timeout starting time;

[0394] If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

[0395] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0396] Determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range;

[0397] If the first version information is within the first preset range and the first sequence number is within the second preset range, the first version information and the first sequence number are verified to obtain a second verification result of the first heartbeat message.

[0398] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0399] Determine whether the first version information is smaller than the historical version information pre-stored in the slave node;

[0400] If it is less than, the second verification result is determined to be a verification failure;

[0401] If it is greater, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed;

[0402] If they are equal, determine whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is not less than the serial number pre-stored in the slave node, update the serial number pre-stored in the slave node to the first serial number, and determine that the second verification result is verification passed.

[0403] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0404] When it is determined that the connection path between the master node and the slave node is in an abnormal state, a slave node split-brain operation is performed.

[0405] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0406] Send a first heartbeat message to each slave node in the server cluster and record the time when the first heartbeat was sent;

[0407] Determine the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each slave node;

[0408] According to the first timeout starting point time and the first timeout duration, the connection path between the master node and each slave node is monitored to obtain a monitoring result.

[0409] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0410] Determine a first monitoring period according to the first timeout starting point time and the first timeout duration;

[0411] Send a second heartbeat message to each slave node and record the time when the second heartbeat was sent;

[0412] According to the second heartbeat replies received from the slave nodes during the first monitoring period, the connection paths between the master node and the slave nodes are monitored to obtain monitoring results.

[0413] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0414] For each slave node, if a second heartbeat reply returned by the slave node is received within the first monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat reply to obtain a monitoring result;

[0415] If the second heartbeat reply returned from the node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

[0416] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0417] If the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state;

[0418] If the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state.

[0419] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0420] Verifying the first heartbeat reply to obtain a first verification result;

[0421] If the first verification result indicates that the verification of the first heartbeat reply is passed, then determining the first heartbeat sending time as the first timeout starting time;

[0422] If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

[0423] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0424] Determine whether the length of the first heartbeat response is legal according to the length of the first heartbeat response and the preset length;

[0425] If the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

[0426] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0427] When it is determined that the connection path between the master node and the slave node is in an abnormal state, the master node split-brain operation is performed.

[0428] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0429] Receive the first heartbeat message sent by the master node in the server cluster, and record the first heartbeat reception time;

[0430] According to the first heartbeat reception time and the first heartbeat message, determine the second timeout starting time of the slave node, and send the first heartbeat reply to the master node;

[0431] According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

[0432] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0433] Determine a second monitoring period according to the second timeout starting time and the second timeout duration;

[0434] Receive the second heartbeat message sent by the master node and record the time when the second heartbeat was received;

[0435] The connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result.

[0436] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0437] If a second heartbeat message sent by the master node is received within the second monitoring period, the connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result;

[0438] If the second heartbeat message sent by the master node is not received within the second monitoring period, it is determined that the connection path is in a fault state.

[0439] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0440] If the second heartbeat message is a valid message, it is determined that the connection path is in a normal state;

[0441] If the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state.

[0442] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0443] Verifying the first heartbeat message to obtain a second verification result;

[0444] If the second verification result indicates that the verification of the first heartbeat message is passed, the first heartbeat reception time is determined as the second timeout starting time;

[0445] If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

[0446] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0447] Determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range;

[0448] If the first version information is within the first preset range and the first sequence number is within the second preset range, the first version information and the first sequence number are verified to obtain a second verification result of the first heartbeat message.

[0449] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0450] Determine whether the first version information is smaller than the historical version information pre-stored in the slave node;

[0451] If it is less than, the second verification result is determined to be a verification failure;

[0452] If it is greater, the historical version information is updated to the first version information, the serial number pre-stored in the slave node is updated to the first serial number, and the second verification result is determined to be verification passed;

[0453] If they are equal, determine whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is not less than the serial number pre-stored in the slave node, update the serial number pre-stored in the slave node to the first serial number, and determine that the second verification result is verification passed.

[0454] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0455] When it is determined that the connection path between the master node and the slave node is in an abnormal state, a slave node split-brain operation is performed.

[0456] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0457] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0458] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fault monitoring method, characterized in that: The method is applied to a master node in a server cluster, and the method comprises: Sending a first heartbeat message to each slave node in the server cluster, and recording the time when the first heartbeat was sent; Determine a first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each of the slave nodes; According to the first timeout starting time and the first timeout duration, the connection path between the master node and each of the slave nodes is monitored to obtain a monitoring result.

2. The method according to claim 1, characterized in that The monitoring of the first connection failure between the master node and each of the slave nodes according to the first timeout starting time and the first timeout duration to obtain a monitoring result includes: Determine a first monitoring period according to the first timeout starting time and the first timeout duration; Send a second heartbeat message to each of the slave nodes, and record the time when the second heartbeat was sent; According to the second heartbeat replies received from the slave nodes during the first monitoring period, the connection path between the master node and the slave nodes is monitored to obtain a monitoring result.

3. The method according to claim 2, characterized in that The monitoring of the connection path between the master node and each of the slave nodes according to the second heartbeat reply received from each of the slave nodes within the first monitoring period to obtain a monitoring result includes: For each of the slave nodes, if a second heartbeat reply returned by the slave node is received within the first monitoring period, when the second heartbeat reply is a valid reply, it is determined that the connection path is in a normal state; when the second heartbeat reply is an invalid reply, it is determined that the connection path is in an abnormal state; If the second heartbeat reply returned by the slave node is not received within the first monitoring period, it is determined that the connection path is in a fault state.

4. The method according to any one of claims 1 to 3, characterized in that: Determining the first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each of the slave nodes includes: Verifying the first heartbeat reply to obtain a first verification result; If the first verification result indicates that the verification of the first heartbeat reply is passed, determining the first heartbeat sending time as the first timeout starting time; If the first verification result indicates that the verification of the first heartbeat reply fails, the original initial value of the first timeout starting time is maintained.

5. The method according to claim 4, characterized in that The first heartbeat message includes first version information and a first sequence number, the first heartbeat reply includes second version information and a second sequence number, and the verifying the first heartbeat reply to obtain a first verification result includes: Determining whether the length of the first heartbeat response is legal according to the length of the first heartbeat response and a preset length; If the length is legal, determine whether the second version information is consistent with the first version information; if the second version information is consistent with the first version information, determine whether the second serial number is consistent with the first serial number; if the second serial number is consistent with the first serial number, determine that the first verification result is verification passed.

6. A fault monitoring method, characterized in that: The method is applied to a slave node in a server cluster, and the method comprises: Receive a first heartbeat message sent by a master node in the server cluster, and record the first heartbeat reception time; Determine a second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message, and send a first heartbeat reply to the master node; According to the second timeout starting time and the second timeout duration, the connection path between the master node and the slave node is monitored to obtain a monitoring result.

7. The method according to claim 6, characterized in that The monitoring of the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result includes: Determine a second monitoring period according to the second timeout starting time and the second timeout duration; Receive the second heartbeat message sent by the master node, and record the second heartbeat reception time; The connection path between the master node and the slave node is monitored according to the validity of the second heartbeat message to obtain a monitoring result.

8. The method according to claim 7, characterized in that The monitoring of the connection path between the master node and the slave node according to the validity of the second heartbeat message to obtain a monitoring result includes: If a second heartbeat message sent by the master node is received within the second monitoring period, when the second heartbeat message is a valid message, it is determined that the connection path is in a normal state; when the second heartbeat message is an invalid message, it is determined that the connection path is in an abnormal state; If the second heartbeat message sent by the master node is not received within the second monitoring period, it is determined that the connection path is in a fault state.

9. The method according to any one of claims 6 to 8, characterized in that: Determining a second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message includes: Verifying the first heartbeat message to obtain a second verification result; If the second verification result indicates that the verification of the first heartbeat message is passed, determining the first heartbeat reception time as the second timeout starting time; If the second verification result indicates that the verification of the first heartbeat message fails, the original initial value of the second timeout starting time is maintained.

10. The method according to claim 9, characterized in that The first heartbeat message includes first version information and a first sequence number, and the verifying the first heartbeat message to obtain a second verification result includes: Determining whether the first version information is within a first preset range, and determining whether the first serial number is within a second preset range; If the first version information is within the first preset range, and the first sequence number is within the second preset range, the first version information and the first sequence number are verified to obtain a second verification result of the first heartbeat message.

11. The method according to claim 10, characterized in that The verifying the first version information and the first sequence number to obtain a second verification result of the first heartbeat message includes: Determine whether the first version information is smaller than the historical version information pre-stored in the slave node; If it is less than, the second verification result is determined to be a verification failure; If it is greater, updating the historical version information to the first version information, updating the serial number pre-stored in the slave node to the first serial number, and determining that the second verification result is verification passed; If they are equal, determine whether the first serial number is less than the serial number pre-stored in the slave node; if the first serial number is not less than the serial number pre-stored in the slave node, update the serial number pre-stored in the slave node to the first serial number, and determine that the second verification result is verification passed.

12. A fault monitoring device, characterized in that: The device comprises: A sending module, used to send a first heartbeat message to each slave node in the server cluster and record the first heartbeat sending time; A first determining module, used for determining a first timeout starting time of the master node according to the first heartbeat sending time and the first heartbeat reply returned by each of the slave nodes; The first monitoring module is used to monitor the first connection failure between the master node and each of the slave nodes according to the first timeout starting time and the first timeout duration to obtain a monitoring result.

13. A fault monitoring device, characterized in that: The device comprises: A receiving module, used for receiving a first heartbeat message sent by a master node in a server cluster, and recording a first heartbeat receiving time; A second determining module, configured to determine a second timeout starting time of the slave node according to the first heartbeat reception time and the first heartbeat message, and send a first heartbeat reply to the master node; The second monitoring module is used to monitor the connection path between the master node and the slave node according to the second timeout starting time and the second timeout duration to obtain a monitoring result.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images