Data center cluster monitoring method, device and system

Abstract

The invention relates to the technical field of data processing, in particular to a data center cluster monitoring method, device and system. The method comprises the following steps: S1, each master node receives synchronization signals sent by a plurality of corresponding redundant nodes, and sends response signals to the redundant nodes; s2, when the main node cannot receive the synchronizing signal of the ith redundant node or the ith redundant node cannot receive the response signal, an abnormal command is sent to a monitoring unit, and the monitoring unit sends a detection command and determines a fault type; s3, when the communication between the main node and the ith redundant node is normal, judging whether the data is synchronized or not based on the update list in the main node and the data version number of each data type of the main node and the ith redundant node in the synchronization signal; and S4, obtaining a target source node and a target time period, and sending target data to the target node for data synchronization in the target time period. According to the invention, the problem of insufficient cluster node monitoring precision is solved, and the monitoring precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically to a data center cluster monitoring method, device and system. Background Art

[0002] Cluster node fault monitoring technology is one of the key technologies to ensure the stable operation of large-scale cluster systems. With the widespread application of cloud computing, big data and distributed systems, cluster node fault monitoring technology is also constantly developing and improving. Similar prior art includes a Chinese patent with publication number CN115102887A, which discloses a cluster node monitoring method, device, electronic device and computer-readable storage medium, which are applied to each node in the cluster system. The method includes: using a heartbeat monitoring thread to initiate a heartbeat signal; using a transmission thread to send the heartbeat signal to other nodes in the cluster system, and receiving a reply signal fed back by other nodes; using the heartbeat monitoring thread to determine the offline nodes in the cluster system according to each reply signal, and upload the corresponding offline node information to the main thread; using the main thread to perform a node status update operation according to the offline node information. Applying the technical solution provided by this application can effectively avoid the impact on cluster services caused by too much operation content and excessive load pressure of the main service process in the cluster node, thereby ensuring the normal operation of the cluster service. In addition, similar prior art includes the US patent with publication number US10152382B2, which discloses a method and system for monitoring a virtual machine cluster, including: at a first preset time interval, sending a status parameter query instruction to the virtual machines in the virtual machine cluster through a physical machine; in response to receiving the query, sending a response message to the physical machine; the physical machine responds to the response message for more than a second preset time, determines that the virtual machine is faulty, determines whether the faulty machine meets the restart condition, and sends a restart instruction to the second computer running the faulty machine; the faulty machine meets the restart condition through the virtual machine; and restarts the second physical machine, i.e., the faulty virtual machine, according to the restart instruction. Both of the above patent documents solve the problem of cluster node fault monitoring, but neither further determines the fault location and data corruption problem, so the fault monitoring accuracy is low. Summary of the invention

[0003] In order to better solve the above problems, the present invention provides a data center cluster monitoring method, which comprises the following steps:

[0004] Step S1: Each master node in the cluster receives a synchronization signal sent by a plurality of redundant nodes corresponding to the master node, and sends a response signal to each of the redundant nodes, wherein the master node and any corresponding redundant node are not in the same sub-network;

[0005] Step S2: When the communication between the master node and the i-th redundant node is abnormal, an abnormal command is sent to a monitoring unit through the master node or the i-th redundant node, and the monitoring unit sends a detection command to the i-th redundant node or the master node based on the abnormal command, and determines the fault type according to a detection response signal of the detection command;

[0006] Step S3: When the master node and the i-th redundant node communicate normally, judging whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the master node and the data version number of each data type of the i-th redundant node in the synchronization signal;

[0007] Step S4: Take the master node or the i-th redundant node whose data is out of sync and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and target time period based on the sum of the number of visits to each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

[0008] As a preferred technical solution of the present invention, step S1 includes the following steps:

[0009] Step S11: each of the master nodes in the cluster receives the synchronization signal periodically sent by the redundant node corresponding to the master node, wherein the synchronization signal includes a data update record of the redundant node, and the data update record includes a data version and a data update time of each data type;

[0010] Step S12: When the master node successfully receives the synchronization signal of the i-th redundant node, the master node sends a response signal to the corresponding i-th redundant node.

[0011] As a preferred technical solution of the present invention, step S2 includes the following steps:

[0012] Step S21: When the master node fails to receive the synchronization signal of the i-th redundant node for T1 consecutive times, a first abnormal command is sent to the monitoring unit, wherein the first abnormal command includes the address information and abnormal type of the i-th redundant node, wherein the i-th redundant node is one of the redundant nodes, and the value range of T1 is a positive integer greater than or equal to 3;

[0013] Step S22: When at least T2 of the i-th redundant nodes fail to receive the response signal, a second abnormal command is sent to the monitoring unit through any of the i-th redundant nodes, wherein the second abnormal command includes the address information of the master node, and the value range of T2 is a positive integer greater than or equal to 2;

[0014] Step S23: the monitoring unit receives the first abnormal signal and the second abnormal signal, and sends a first detection signal and a second detection signal to the i-th redundant node and the master node respectively based on the first abnormal signal and the second abnormal signal. When the monitoring unit can receive the first detection response signal, the fault exists in the i-th redundant node. When the monitoring unit cannot receive the first detection response signal, the fault exists in the sub-network where the i-th redundant node is located. Similarly, the fault location of the master node is obtained.

[0015] The abnormal command includes the first abnormal command and the second abnormal command, the detection command includes the first detection command and the second detection command, and the detection response signal includes the first detection response signal and the second detection response signal.

[0016] As a preferred technical solution of the present invention, step S2 also includes: when the subnet where the master node is located fails or the subnet where the i-th redundant node is located fails, the monitoring unit sends a query instruction to each other redundant node other than the i-th redundant node at a preset time interval, and obtains data update records and load information of all other redundant nodes, creates an update list according to the data update records of all other redundant nodes, and obtains the j-th redundant node in which each data type in the update list is the latest version; when there are multiple j-th redundant nodes, in the case of a failure of the subnet where the master node is located, the k-th redundant node with the best performance among the multiple j-th redundant nodes is used as a temporary master node; in the case of a failure of the subnet where the i-th redundant node is located, the load of the i-th redundant node is distributed to multiple j-th redundant nodes; when the master node or the i-th redundant node fails, the monitoring unit queries the fault data corresponding to the fault position of the master node or the i-th redundant node, and distributes the access task of accessing the fault data to N other redundant nodes whose corresponding data at the fault position is the latest version and whose access task amount is the least.

[0017] As a preferred technical solution of the present invention, step S3 includes:

[0018] Step S31: when the master node and the i-th redundant node communicate normally, obtain the current version of each data type in the update record of the i-th redundant node through the synchronization signal, and compare the current version of each data type with the update version of the i-th redundant node stored in the update list in the master node to obtain a first comparison result;

[0019] Step S32: when the first comparison result is that the current version of the data type corresponding to the i-th redundant node is consistent with the updated version, the current version of the data type corresponding to the i-th redundant node is also compared with the updated versions of the data type corresponding to the master node and other redundant nodes, and a second comparison result is obtained. When the second comparison result is that the current version of the data type corresponding to the i-th redundant node is lower than the updated version of the master node or the other redundant nodes, the data type corresponding to the i-th redundant node is out of sync and needs to be updated; otherwise, it does not need to be updated.

[0020] Step S33: when the first comparison result is that the current version of the data type of the i-th redundant node is inconsistent with the updated version, when the current version of the data type is higher than the updated version, the data corresponding to the data type in the i-th redundant node is updated, and when the current version of the data type is lower than the updated version, the data corresponding to the data type in the i-th redundant node is out of sync and needs to be updated;

[0021] Step S34: Also by comparing the current version of the master node with the updated version in the update list and repeating the methods of steps S32 and S33, it is determined whether each of the data types in the master node is synchronized and updated.

[0022] As a preferred technical solution of the present invention, step S4 includes:

[0023] When the data of the master node or the i-th redundant node is not synchronized and needs to be updated, the master node or the i-th redundant node is used as the target node, and the target node address, the target data type that needs to be synchronized, and the addresses of multiple data source nodes with the latest version of the target data type are sent to the monitoring unit through the master node. The monitoring unit calculates the sum of the access volume of the target node and each corresponding data source node in each time period through the historical access volume of the target node and each of the data source nodes in different time periods, and obtains the target time period closest to the current time and the sum of the access volume is less than a set threshold, and uses the data source node corresponding to the target time period as the target source node, and sends the target data from the target source node to the target node within the target time. After receiving the target data, the target node backs up the data to the storage unit.

[0024] As a preferred technical solution of the present invention, step S4 further includes:

[0025] After the target node completes data update, the version number of the target node corresponding to the target data type is updated, sent to the master node, and replaces the updated version number of the target node corresponding to the target data type in the update list.

[0026] As a preferred technical solution of the present invention, the value range of i, j, k is a positive integer greater than or equal to 1, and the value range of N is a positive integer less than or equal to 5.

[0027] The present invention also provides a data center cluster monitoring system, the system is used to implement the above method, the system comprises:

[0028] A master node, used to receive synchronization signals sent by multiple redundant nodes corresponding to the master node, and send a response signal to each of the redundant nodes, wherein the master node and any corresponding redundant node are not in the same sub-network;

[0029] A redundant node, used to send a synchronization signal to the corresponding master node, and receive the response signal sent by the corresponding master node;

[0030] The monitoring unit is configured to: when the communication between the master node and the i-th redundant node is abnormal, receive an abnormal command sent by the master node or the i-th redundant node, send a detection command to the i-th redundant node or the master node based on the abnormal command, and determine the fault type according to the detection response signal of the detection command; when the communication between the master node and the i-th redundant node is normal, judge whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the master node and the data version number of each data type of the i-th redundant node in the synchronization signal; take the master node or the i-th redundant node whose data is not synchronized and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and the target time period based on the sum of the access volume of each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

[0031] The present invention also provides a data center cluster monitoring device, wherein a storage medium of the device stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the method described in any one of claims 1-8.

[0032] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0033] The present invention sends a synchronization signal to the corresponding master node through a redundant node, and when the master node receives the synchronization signal sent by the redundant node, sends a response signal to the redundant node, thereby detecting the communication status between each redundant node and the master node, and when the master node fails to receive the synchronization signal of the i-th redundant node for N consecutive times, sends the first abnormal command to the monitoring unit through the master node, and determines the fault location of the i-th redundant node based on the reception of the first detection response signal, and uses the same method to determine the fault location of the master node, when the master node and the i-th redundant node can communicate normally, and compares the current version of each data type in the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result to determine whether the data is updated, and also compares the current version of each data type corresponding to the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result to determine whether the data is updated, and also compares the current version of each data type corresponding to the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result to determine whether the data is updated, and The version is compared with the updated version of the above data type corresponding to other nodes in the above update list to determine whether the data of the data type corresponding to the above i-th redundant node is synchronized and whether it needs to be updated. Similarly, it is determined whether the data of each data type in the above master node is synchronized and whether it needs to be updated. When the target node data is not synchronized and needs to be updated, the target time period closest to the current time is obtained as the data synchronization time period, and the above data source node corresponding to the above target time period is used as the above target source node. Within the above target time period, the above target data is sent to the above target node through the above target source node, and the above target node is backed up after receiving the above target data to complete data synchronization. Through the mutual cooperation of the above technical solutions, not only can the fault and the corresponding fault location be accurately detected, but also when the damaged version of the data is restored to the previous version or the data is updated and upgraded, the data synchronization can be updated in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a data center cluster monitoring method of the present invention;

[0035] Figure 2 This is a structural diagram of a data center cluster monitoring system of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The present invention provides a data center cluster monitoring method, such as Figure 1 As shown, the method comprises the following steps:

[0038] Step S1: Each master node in the cluster receives a synchronization signal sent by a plurality of redundant nodes corresponding to the master node, and sends a response signal to the corresponding redundant node, wherein the master node and any corresponding redundant node are not in the same sub-network;

[0039] Specifically, the cluster includes multiple node groups, nodes in different node groups store different access data, and different nodes in the same node group store the same access data. In the same node group, the nodes are divided into a main node and multiple redundant nodes. Each of the i-th redundant nodes communicates with the main node by periodically sending the synchronization signal. When the main node receives the synchronization signal sent by the i-th redundant node, it sends a response signal to the redundant node. Since the synchronization signal includes the data update record of the corresponding node, through the technical solution, not only can the data update status of the redundant node be checked by receiving the synchronization signal, but also the communication status between each of the redundant nodes and the main node can be detected, thereby laying a foundation for obtaining the fault information of the main node and each of the redundant nodes.

[0040] Step S2: When the communication between the master node and the i-th redundant node is abnormal, an abnormal command is sent to a monitoring unit through the master node or the i-th redundant node, and the monitoring unit sends a detection command to the i-th redundant node or the master node based on the abnormal command, and determines the fault type according to a detection response signal of the detection command;

[0041] Specifically, when the master node fails to receive the synchronization signal of the i-th redundant node for N consecutive times, it is considered that the i-th redundant node or the sub-network where the i-th redundant node is located has a fault. Therefore, the master node sends the first abnormal command to the monitoring unit, sends the address information of the i-th redundant node to the monitoring unit through the first abnormal command, sends a first detection signal according to the address information of the i-th redundant node, and determines the fault position of the i-th redundant node through the first detection response signal sent by the i-th redundant node. When the monitoring unit can receive the first detection response signal, it indicates that there is no fault in the sub-network where the i-th redundant node is located, and the i-th redundant node may be faulty. When the monitoring unit cannot receive the first detection response signal, the sub-network where the i-th redundant node is located may be faulty. The fault position of the master node is determined based on the second abnormal signal sent by the monitoring unit and the second detection response signal by the same method. Through the above technical solution, the fault position of the master node and the corresponding i-th redundant node can be accurately monitored and determined.

[0042] Step S3: When the master node and the i-th redundant node communicate normally, judging whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the data version number of each data type in the master node, and the data version number of each data type of the i-th redundant node in the synchronization signal;

[0043] Specifically, when the master node and the i-th redundant node can communicate normally, the master node updates the current version of each data type in the record of the synchronization signal sent by the i-th redundant node, and compares the current version of each data type in the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result to determine whether the data is updated. However, it cannot be explained that the data corresponding to the data type of the i-th redundant node is synchronized with the data corresponding to the data type of other nodes. Therefore, the current version of the i-th redundant node corresponding to the data type is also compared with the updated version of the data type corresponding to other nodes in the update list to obtain multiple second comparison results, and determines whether the data of the data type of the i-th redundant node is synchronized and needs to be updated through the multiple second comparison results, and determines whether the data of each data type in the master node is synchronized and needs to be updated through the same method. Through the technical solution, when the master node and the i-th redundant node communicate, it is possible to accurately determine whether the data of the master node and the i-th redundant node are synchronized and need to be updated, and lay a foundation for data synchronization.

[0044] Step S4: Take the master node or the i-th redundant node whose data is out of sync and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and target time period based on the sum of the number of visits to each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

[0045] Specifically, when the target node data is not synchronized and needs to be updated, the address of the target node, the data type that needs to be synchronized, and the addresses of multiple data source nodes with the latest version of the target data type corresponding to the data obtained through the update list are sent to the monitoring unit through the master node. The monitoring unit collects and calculates the sum of the access volume of the target node and each of the data source nodes in each time period based on the historical access volume of the target node and each of the data source nodes in different time periods. Since it takes a certain amount of communication resources to send the target data corresponding to the target data type from the data source node to the target node, and the more timely the update, the better, therefore, the target time period closest to the current time and with the sum of the access volume less than the set threshold is used as the data synchronization time period, and the data source node corresponding to the target time period is used as the target source node. The target data is sent to the target node through the target source node within the target time period, and the target node is backed up after receiving the target data to complete data synchronization. Through the above technical solution, when the target node data is not synchronized and needs to be updated, it can be timely and does not affect normal access.

[0046] Furthermore, the step S1 comprises the following steps:

[0047] Step S11: each of the master nodes in the cluster receives the synchronization signal periodically sent by the redundant node corresponding to the master node, wherein the synchronization signal includes a data update record of the redundant node, and the data update record includes a data version and a data update time of each data type;

[0048] Step S12: When the master node successfully receives the synchronization signal from each redundant node, the master node sends a response signal to the corresponding redundant node.

[0049] Specifically, each of the above-mentioned i-th redundant nodes communicates with the above-mentioned master node by periodically sending the above-mentioned synchronization signal. When the above-mentioned master node receives the above-mentioned synchronization signal sent by the above-mentioned i-th redundant node, it sends a response signal to the above-mentioned redundant node. Since the above-mentioned synchronization signal includes the above-mentioned data update record of the corresponding node, through the above-mentioned technical solution, not only can the data update status of the corresponding redundant node be checked by receiving the above-mentioned synchronization signal and the above-mentioned response, but also the communication status between each of the above-mentioned redundant nodes and the above-mentioned master node can be detected, thereby laying a foundation for obtaining the fault information of the above-mentioned master node and each of the above-mentioned redundant nodes.

[0050] Furthermore, the step S2 comprises the following steps:

[0051] Step S21: When the master node fails to receive the synchronization signal of the i-th redundant node for T1 consecutive times, a first abnormal command is sent to the monitoring unit, wherein the first abnormal command includes the address information and abnormal type of the i-th redundant node, wherein the i-th redundant node is one of the redundant nodes, and the value range of T1 is a positive integer greater than or equal to 3;

[0052] Step S22: When at least T2 of the i-th redundant nodes fail to receive the response signal, a second abnormal command is sent to the monitoring unit through any of the i-th redundant nodes, wherein the second abnormal command includes the address information of the master node, and the value range of T2 is a positive integer greater than or equal to 2;

[0053] Step S23: the monitoring unit receives the first abnormal signal and the second abnormal signal, and sends a first detection signal and a second detection signal to the i-th redundant node and the master node respectively based on the first abnormal signal and the second abnormal signal. When the monitoring unit can receive the first detection response signal, the fault exists in the i-th redundant node. When the monitoring unit cannot receive the first detection response signal, the fault exists in the sub-network where the i-th redundant node is located. Similarly, the fault location of the master node is obtained.

[0054] The abnormal command includes the first abnormal command and the second abnormal command, the detection command includes the first detection command and the second detection command, and the detection response signal includes the first detection response signal and the second detection response signal.

[0055] Specifically, when the master node fails to receive the synchronization signal of the i-th redundant node for T1 consecutive times, it is considered that the i-th redundant node or the sub-network where the i-th redundant node is located has a fault. Therefore, the master node sends the first abnormal command to the monitoring unit, sends the address information of the i-th redundant node to the monitoring unit through the first abnormal command, sends a first detection signal according to the address information of the i-th redundant node, and determines the fault position of the i-th redundant node through the first detection response signal sent by the i-th redundant node. When the monitoring unit can receive the first detection response signal, it indicates that there is no fault in the sub-network where the i-th redundant node is located, and the i-th redundant node may have a fault. When the monitoring unit cannot receive the first detection response signal, the sub-network where the i-th redundant node is located may have a fault. The fault position of the master node is determined based on the second abnormal signal sent by the monitoring unit and the second detection response signal by the same method. Through the above technical solution, the fault position of the master node and the corresponding i-th redundant node can be accurately monitored and determined.

[0056] Furthermore, the step S2 further includes:

[0057] When the subnet where the master node is located fails or the subnet where the i-th redundant node is located fails, the monitoring unit sends a query instruction to each of the i-th redundant nodes at a preset time interval, and obtains data update records and load information of all the i-th redundant nodes, creates an update list according to the data update records of all the i-th redundant nodes, and obtains the j-th redundant node whose corresponding time of each data update event in the update list is the latest; when there are multiple j-th redundant nodes, when the subnet where the master node is located fails, the k-th redundant node with the smallest total access volume among the multiple j-th redundant nodes is used as a temporary master node; when the subnet where the i-th redundant node is located fails, the load of the i-th redundant node is distributed to the multiple j-th redundant nodes; when the master node or the i-th redundant node fails, the monitoring unit queries the fault data corresponding to the fault position of the master node or the i-th redundant node, and distributes the access task of accessing the fault data to the N other m-th redundant nodes whose corresponding data at the fault position is the latest and have the least access tasks.

[0058] Specifically, when the sub-network where the above-mentioned main node or the above-mentioned i-th redundant node is located fails, at this time, the above-mentioned main node and the above-mentioned i-th redundant node lose the node function, and the access task of the above-mentioned main node or the above-mentioned i-th redundant node needs to be allocated to other redundant nodes. Therefore, the above-mentioned monitoring unit sends the above-mentioned query instruction to each other redundant node other than the above-mentioned i-th redundant node at a preset time interval, obtains the update records of the above-mentioned other redundant nodes through the above-mentioned query instruction, and creates an update list, and obtains the latest update record of each data type based on the above-mentioned update list, and obtains the above-mentioned latest update record of each data type to filter out the above-mentioned j-th redundant node whose each data type is the latest version, and when there are multiple j-th redundant nodes, in the case of a failure of the sub-network where the above-mentioned main node is located, the above-mentioned k-th redundant node with the best performance among the multiple j-th redundant nodes is used as a temporary main node, and in the case of a failure of the sub-network where the above-mentioned i-th redundant node is located, the load of the above-mentioned i-th redundant node is distributed to the multiple j-th redundant nodes according to the load amount of each j-th redundant node. The remaining nodes, wherein the greater the load, the greater the load of the above-mentioned i-th redundant node allocated, and vice versa; when the above-mentioned main node or the above-mentioned i-th redundant node fails, when the above-mentioned main node or the above-mentioned i-th redundant node fails, the above-mentioned main node and the above-mentioned i-th redundant node are used as faulty nodes. Since the above-mentioned faulty node can communicate with the above-mentioned monitoring node, it is possible that the data is updated or part of the data is damaged, resulting in the above-mentioned synchronization signal and the corresponding response signal cannot be received normally. The update record of the above-mentioned faulty node is queried through the above-mentioned monitoring unit and the fault position is obtained, that is, the data version is restored to the data position of the previous old version due to data damage or data abnormality, and the access task at the above-mentioned fault position is allocated to the N above-mentioned m-th redundant nodes whose corresponding data at the above-mentioned fault position is the latest version and the least access task amount. Through the above-mentioned technical solution, when the above-mentioned main node and the above-mentioned i-th redundant node fail or when the corresponding sub-network fails, the above-mentioned temporary main node can be dynamically obtained, thereby ensuring that the access tasks of the above-mentioned main node and the above-mentioned i-th redundant node are executed normally.

[0059] Furthermore, the step S3 comprises:

[0060] Step S31: when the master node and the i-th redundant node communicate normally, obtain the current version of each data type in the update record of the i-th redundant node through the synchronization signal, and compare the current version of each data type with the update version of the i-th redundant node stored in the update list in the master node to obtain a first comparison result;

[0061] Step S32: when the first comparison result is that the current version of the data type corresponding to the i-th redundant node is consistent with the updated version, the current version of the data type corresponding to the i-th redundant node is also compared with the updated versions of the data types corresponding to the master node and other redundant nodes, and multiple second comparison results are obtained. When at least one of the multiple second comparison results is that the current version of the data type corresponding to the i-th redundant node is lower than the updated version of the master node or the other redundant nodes, the data type corresponding to the i-th redundant node is out of sync and needs to be updated; otherwise, it does not need to be updated.

[0062] Step S33: when the first comparison result is that the current version of the data type of the i-th redundant node is inconsistent with the updated version, when the current version of the data type is higher than the updated version, the data corresponding to the data type in the i-th redundant node is updated, and when the current version of the data type is lower than the updated version, the data corresponding to the data type in the i-th redundant node is out of sync and needs to be updated;

[0063] Step S34: Also by comparing the current version of the master node with the updated version in the update list and repeating the methods of steps S32 and S33, it is determined whether each of the data types in the master node is synchronized and updated.

[0064] Specifically, when the master node and the i-th redundant node can communicate normally, that is, the master node can receive the synchronization signal sent by the i-th redundant node and the i-th redundant node can also receive the response signal returned by the master node, including the case where the sub-node fails and is replaced by a temporary master node, the master node also updates the current version of each data type in the synchronization signal sent by the i-th redundant node, and compares the current version of each data type in the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result. When the comparison result is that the two are consistent, it means that the data of the above data type in the i-th redundant node has not been updated, but it cannot be explained whether the data of the above data type corresponding to the i-th redundant node is synchronized with the data of the above data type corresponding to other nodes, that is, whether the data versions are consistent. Therefore, the current version of the above data type corresponding to the i-th redundant node is also compared with the updated version of the above data type corresponding to other nodes in the update list. The other nodes include the above-mentioned main node and other redundant nodes except the above-mentioned i-th redundant node. When a plurality of the above-mentioned second comparison results are obtained, and when it is obtained through the plurality of the above-mentioned second comparison results that the current version of the data type corresponding to the above-mentioned i-th redundant node is lower than the above-mentioned updated version of the above-mentioned data type corresponding to at least one of the above-mentioned other nodes, that is, the data of the data type corresponding to the above-mentioned i-th redundant node is not the latest version, it means that the data of the data type corresponding to the above-mentioned i-th redundant node is not synchronized and needs to be updated. On the contrary, when the current version of the data type corresponding to the above-mentioned i-th redundant node is higher than or equal to the above-mentioned updated version of the above-mentioned data type corresponding to at least one of the above-mentioned other nodes, it means that the data of the data type corresponding to the above-mentioned i-th redundant node is synchronized with the latest version and does not need to be updated. The same method is used to judge whether the data of each data type in the above-mentioned main node is synchronized and whether it needs to be updated. Through the above-mentioned technical solution, when the above-mentioned main node and the above-mentioned i-th redundant node communicate, it can be accurately judged whether the data of the above-mentioned main node and the above-mentioned i-th redundant node are synchronized and whether they need to be updated, and a foundation is laid for data synchronization.

[0065] Furthermore, the step S4 comprises:

[0066] When the data of the master node or the i-th redundant node is not synchronized and needs to be updated, the master node or the i-th redundant node is used as the target node, and the address of the target node, the target data type that needs to be synchronized, and the addresses of multiple data source nodes with the latest version of the target data type are sent to the monitoring unit through the master node. The monitoring unit calculates the sum of the access volume of the target node and each corresponding data source node in each time period through the historical access volume of the target node and each of the data source nodes in different time periods, and obtains the target time period closest to the current time and the sum of the access volume is less than a set threshold, and uses the data source node corresponding to the target time period as the target source node, and sends the target data from the target source node to the target node within the target time. After receiving the target data, the target node backs up the data to the storage unit.

[0067] Specifically, when the target node data is not synchronized and needs to be updated, the address of the target node, the data type that needs to be synchronized, and the addresses of multiple data source nodes with the latest version of the target data type corresponding to the data obtained through the update list are sent to the monitoring unit through the master node. The monitoring unit collects and calculates the sum of the access volume of the target node and each of the data source nodes in each time period based on the historical access volume of the target node and each of the data source nodes in different time periods. Since it takes a certain amount of communication resources to send the target data corresponding to the target data type from the data source node to the target node, and the more timely the update, the better, therefore, the target time period closest to the current time and with the sum of the access volume less than the set threshold is used as the data synchronization time period, and the data source node corresponding to the target time period is used as the target source node. The target data is sent to the target node through the target source node within the target time period, and the target node is backed up after receiving the target data to complete data synchronization. Through the above technical solution, when the target node data is not synchronized and needs to be updated, it can be timely and does not affect normal access.

[0068] Furthermore, the step S4 further includes:

[0069] After the target node completes the data update, the data version number of the target node corresponding to the target data type is updated, and the updated data version number is sent to the master node and replaces the updated version number of the target node corresponding to the target data type in the update list.

[0070] Specifically, after the target node completes the data update, the data version number of the target node corresponding to the target data type is updated, and the updated data version number replaces the update version number in the update list. The technical solution is used to facilitate subsequent data synchronization detection.

[0071] Furthermore, the value range of i, j, and k is a positive integer greater than or equal to 1, and the value range of N is a positive integer less than or equal to 5.

[0072] The present invention also provides a data center cluster monitoring system, the system is used to implement the above method, such as Figure 2 As shown, the system comprises:

[0073] A master node, used to receive synchronization signals sent by multiple redundant nodes corresponding to the master node, and send a response signal to each of the redundant nodes, wherein the master node and any corresponding redundant node are not in the same sub-network;

[0074] A redundant node, used to send a synchronization signal to the corresponding master node, and receive the response signal sent by the corresponding master node;

[0075] The monitoring unit is configured to: when the communication between the master node and the i-th redundant node is abnormal, receive an abnormal command sent by the master node or the i-th redundant node, send a detection command to the i-th redundant node or the master node based on the abnormal command, and determine the fault type according to the detection response signal of the detection command; when the communication between the master node and the i-th redundant node is normal, judge whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the master node and the data version number of each data type of the i-th redundant node in the synchronization signal; take the master node or the i-th redundant node whose data is not synchronized and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and the target time period based on the sum of the access volume of each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

[0076] The present invention also provides a data center cluster monitoring device, wherein the storage medium of the device stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the above method.

[0077] In summary, the present invention sends a synchronization signal to the corresponding master node through a redundant node, and when the master node receives the synchronization signal sent by the redundant node, sends a response signal to the redundant node, thereby detecting the communication status between each redundant node and the master node, and when the master node fails to receive the synchronization signal of the i-th redundant node for N consecutive times, sends the first abnormal command to the monitoring unit through the master node, and determines the fault location of the i-th redundant node based on the reception of the first detection response signal, and uses the same method to determine the fault location of the master node, when the master node and the i-th redundant node can communicate normally, and compares the current version of each data type in the i-th redundant node with the updated version stored in the update list, and obtains the first comparison result to determine whether the data is updated, and also determines whether the i-th redundant node corresponds to the data type. The current version is compared with the updated versions of the above data types corresponding to other nodes in the above update list to determine whether the data of the data type corresponding to the above i-th redundant node is synchronized and whether it needs to be updated. Similarly, it is determined whether the data of each data type in the above master node is synchronized and whether it needs to be updated. When the target node data is not synchronized and needs to be updated, the target time period closest to the current time is obtained as the data synchronization time period, and the data source node corresponding to the above target time period is used as the above target source node. Within the above target time period, the above target data is sent to the above target node through the above target source node, and the above target node is backed up after receiving the above target data to complete data synchronization. Through the mutual cooperation of the above technical solutions, not only can the fault and the corresponding fault location be accurately detected, but also when the damaged version of the data is restored to the previous version or the data is updated and upgraded, the data synchronization can be updated in time.

[0078] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data center cluster monitoring method, characterized in that: The method comprises the following steps: Step S1: Each master node in the cluster receives a synchronization signal sent by a plurality of redundant nodes corresponding to the master node, and sends a response signal to each of the redundant nodes, wherein the master node and any corresponding redundant node are not in the same sub-network; Step S2: When the communication between the master node and the i-th redundant node is abnormal, an abnormal command is sent to a monitoring unit through the master node or the i-th redundant node, and the monitoring unit sends a detection command to the i-th redundant node or the master node based on the abnormal command, and determines the fault type according to a detection response signal of the detection command; Step S3: When the master node and the i-th redundant node communicate normally, judging whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the master node and the data version number of each data type of the i-th redundant node in the synchronization signal; Step S4: Take the master node or the i-th redundant node whose data is out of sync and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and target time period based on the sum of the number of visits to each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

2. The method according to claim 1, characterized in that The step S1 comprises the following steps: Step S11: each of the master nodes in the cluster receives the synchronization signal periodically sent by the redundant node corresponding to the master node, wherein the synchronization signal includes a data update record of the redundant node, and the data update record includes a data version and a data update time of each data type; Step S12: When the master node successfully receives the synchronization signal of the i-th redundant node, the master node sends a response signal to the corresponding i-th redundant node.

3. The method according to claim 1, characterized in that: The step S2 comprises the following steps: Step S21: When the master node fails to receive the synchronization signal of the i-th redundant node for T1 consecutive times, a first abnormal command is sent to the monitoring unit, wherein the first abnormal command includes the address information and abnormal type of the i-th redundant node, wherein the i-th redundant node is one of the redundant nodes, and the value range of T1 is a positive integer greater than or equal to 3; Step S22: When at least T2 of the i-th redundant nodes fail to receive the response signal, a second abnormal command is sent to the monitoring unit through any of the i-th redundant nodes, wherein the second abnormal command includes the address information of the master node, and the value range of T2 is a positive integer greater than or equal to 2; Step S23: the monitoring unit receives the first abnormal signal and the second abnormal signal, and sends a first detection signal and a second detection signal to the i-th redundant node and the master node respectively based on the first abnormal signal and the second abnormal signal. When the monitoring unit can receive the first detection response signal, the fault exists in the i-th redundant node. When the monitoring unit cannot receive the first detection response signal, the fault exists in the sub-network where the i-th redundant node is located. Similarly, the fault location of the master node is obtained. The abnormal command includes the first abnormal command and the second abnormal command, the detection command includes the first detection command and the second detection command, and the detection response signal includes the first detection response signal and the second detection response signal.

4. The method according to claim 1, characterized in that: The step S2 also includes: when the subnet where the master node is located fails or the subnet where the i-th redundant node is located fails, the monitoring unit sends a query instruction to each other redundant node other than the i-th redundant node at a preset time interval, and obtains data update records and load information of all other redundant nodes, creates an update list according to the data update records of all other redundant nodes, and obtains the j-th redundant node in which each data type in the update list is the latest version; when there are multiple j-th redundant nodes, in the case of a failure of the subnet where the master node is located, the k-th redundant node with the best performance among the multiple j-th redundant nodes is used as a temporary master node; in the case of a failure of the subnet where the i-th redundant node is located, the load of the i-th redundant node is distributed to the multiple j-th redundant nodes; when the master node or the i-th redundant node fails, the monitoring unit queries the fault data corresponding to the fault position of the master node or the i-th redundant node, and distributes the access task of accessing the fault data to the N other redundant nodes whose corresponding data at the fault position is the latest version and whose access task amount is the least.

5. The method according to claim 1, characterized in that The step S3 comprises: Step S31: when the master node and the i-th redundant node communicate normally, obtain the current version of each data type in the update record of the i-th redundant node through the synchronization signal, and compare the current version of each data type with the update version of the i-th redundant node stored in the update list in the master node to obtain a first comparison result; Step S32: when the first comparison result is that the current version of the data type corresponding to the i-th redundant node is consistent with the updated version, the current version of the data type corresponding to the i-th redundant node is also compared with the updated versions of the data type corresponding to the master node and other redundant nodes, and a second comparison result is obtained. When the second comparison result is that the current version of the data type corresponding to the i-th redundant node is lower than the updated version of the master node or the other redundant nodes, the data type corresponding to the i-th redundant node is out of sync and needs to be updated; otherwise, it does not need to be updated. Step S33: when the first comparison result is that the current version of the data type of the i-th redundant node is inconsistent with the updated version, when the current version of the data type is higher than the updated version, the data corresponding to the data type in the i-th redundant node is updated, and when the current version of the data type is lower than the updated version, the data corresponding to the data type in the i-th redundant node is out of sync and needs to be updated; Step S34: Also by comparing the current version of the master node with the updated version in the update list and repeating the methods of steps S32 and S33, it is determined whether each of the data types in the master node is synchronized and updated.

6. The method according to claim 1, characterized in that The step S4 comprises: When the data of the master node or the i-th redundant node is not synchronized and needs to be updated, the master node or the i-th redundant node is used as the target node, and the target node address, the target data type that needs to be synchronized, and the addresses of multiple data source nodes with the latest version of the target data type are sent to the monitoring unit through the master node. The monitoring unit calculates the sum of the access volume of the target node and each corresponding data source node in each time period through the historical access volume of the target node and each of the data source nodes in different time periods, and obtains the target time period closest to the current time and the sum of the access volume is less than a set threshold, and uses the data source node corresponding to the target time period as the target source node, and sends the target data from the target source node to the target node within the target time. After receiving the target data, the target node backs up the data to the storage unit.

7. The method according to claim 6, characterized in that The step S4 further comprises: After the target node completes data update, the version number of the target node corresponding to the target data type is updated, sent to the master node, and replaces the updated version number of the target node corresponding to the target data type in the update list.

8. The method according to claim 4, characterized in that The value range of i, j, and k is a positive integer greater than or equal to 1, and the value range of N is a positive integer less than or equal to 5.

9. A data center cluster monitoring system, the system being used to implement the method according to any one of claims 1 to 8, characterized in that: The system comprises: A master node, used to receive synchronization signals sent by multiple redundant nodes corresponding to the master node, and send a response signal to each of the redundant nodes, wherein the master node and any corresponding redundant node are not in the same sub-network; A redundant node, used to send a synchronization signal to the corresponding master node, and receive the response signal sent by the corresponding master node; The monitoring unit is configured to: when the communication between the master node and the i-th redundant node is abnormal, receive an abnormal command sent by the master node or the i-th redundant node, send a detection command to the i-th redundant node or the master node based on the abnormal command, and determine the fault type according to the detection response signal of the detection command; when the communication between the master node and the i-th redundant node is normal, judge whether the master node and the i-th redundant node are data synchronized and whether data needs to be updated based on the update list in the master node, the master node and the data version number of each data type of the i-th redundant node in the synchronization signal; take the master node or the i-th redundant node whose data is not synchronized and needs to be updated as the target node, obtain the data source node with the latest data version corresponding to the target data type that needs to be updated from the update list, and obtain the target source node and the target time period based on the sum of the access volume of each of the data source nodes and the target node in each time period, and send the target data corresponding to the target data type to the target node through the target source node within the target time period for data synchronization.

10. A data center cluster monitoring device, characterized in that: The storage medium of the device stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images