Fault detection method, device, equipment and medium of a distributed storage system

Abstract

The present application discloses a fault detection method, device, device, and computer-readable storage medium for a distributed storage system. The method includes: according to the storage pool type of the distributed storage system, using corresponding calculation rules to determine the fault threshold value; Obtain the number of times that each node in the distributed storage system is reported as being in an abnormal state; and determine the fault condition of the distributed storage system according to the number of reports and the fault threshold value. It can be seen that the fault threshold value in this method is determined according to the storage pool type of the storage system and using the corresponding calculation rules. Therefore, the fault condition of the distributed storage system is determined according to the number of reports and the fault threshold value. It can avoid misjudgment of faults caused by faulty nodes in the back-end network misreporting abnormalities of other nodes, improve the accuracy of fault detection in the distributed storage system, and relatively guarantee the normal use of the entire distributed storage system.

Description

technical field [0001] The present invention relates to the field of distributed storage systems, in particular to a fault detection method, device, equipment and computer-readable storage medium of a distributed storage system. Background technique [0002] In a distributed storage system, by setting up a daemon process (or service) on each node, it is used to provide access and monitoring to the hard disk in the storage pool; and through the communication between daemon processes (or services) between different nodes Heartbeat messages to detect whether the peer daemon process (or service) is normal. [0003] For each node, it includes the front-end network and the back-end network. The front-end network is used for customer business, and the back-end network is used for message communication and data interaction within the cluster. In order to detect the connectivity of the network, the daemon process between nodes will The network and the back-end network perform heartb...

AI Technical Summary

Problems solved by technology

However, there will be a problem with this method: for example, assuming that there are two actual faulty nodes in the current distributed storage system, other nodes will report the abnormality of these two actual faulty nodes, and these two actual faulty nodes will also report all other faulty nodes. The node is abnormal; in this way, each other node is reported abnormal by at least the two actual faulty nodes, and since the number of abnormal reports of each other node exceeds the preset threshold value, the cluster management process will send all The node is set as a failed node, resulting in the unavailability of the entire cluster

In reality, the cluster may still be available with only two actual failed nodes

It can be seen that the fault detection method for the distributed storage system in the prior art, when the back-end network of the node fails, there will be false alarms, which will affect the normal use of the entire distributed storage system

Images