A dual-node DSC cluster fault switching method, system, device and medium

Abstract

The present invention provides a dual-node DSC cluster fault switching method and system, device and medium, the method comprising: obtaining the data synchronization status of the local service node and the opposite service node in the DSC cluster; if the data synchronization status is abnormal, locking the disk of the local service node and the opposite service node, so that the status of the disk of the local service node and the opposite service node is set to an unreadable and unwritable state; performing fault identification processing on the local service node and the opposite service node to determine the faulty service node and the non-faulty service node; removing all disks of the faulty service node from the DSC cluster; unlocking the disks of the non-faulty service nodes. The present invention can automatically detect the non-faulty service node, and perform automatic fault switching, so as to realize the fault self-healing of the dual-node DSC cluster, and has high availability.

Description

Technical Field

[0001] The present invention relates to the field of data fault processing, and in particular to a dual-node DSC cluster fault switching method and system, equipment and medium. Background Art

[0002] A cluster system is a parallel or distributed system composed of interconnected computers. It can be managed as a separate, unified computing resource. Through cluster technology, relatively high benefits in performance, reliability, and flexibility can be obtained at a relatively low cost. Its task scheduling is the core technology in the cluster system.

[0003] At present, the DSC cluster used in the market is a shared storage database cluster system based on a single-node database management system. It has transformed the buffer, transaction system, blocking system and log system to meet the global concurrent access control requirements between shared storage cluster nodes. At the same time, the introduction of cache exchange technology can improve the efficiency of data transmission between nodes. The DSC cluster allows multiple database instances to access and modify the same database at the same time, which will inevitably bring global concurrency problems. At the same time, when one or more devices in a DSC cluster system fail, the load capacity of the entire DSC cluster will decrease, thereby affecting its stability and availability.

[0004] Existing DSC clusters all achieve failover through third-party arbitration. Generally, a cluster needs at least three nodes, two of which are used for the DSC cluster and the remaining nodes are used for storage. In addition, if data redundancy is required, additional nodes are required for backup. Therefore, the application cost of this DSC cluster failover method is too high. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] According to one aspect of the present application, a dual-node DSC cluster failover method is provided, comprising the following steps:

[0007] S100, obtaining the data synchronization status of the local service node and the opposite service node in the DSC cluster; the local service node and the opposite service node are two identical service nodes with storage function in the DSC cluster; the local service node and the opposite service node perform data synchronization through a data synchronization network.

[0008] S200, determining whether the data synchronization state is abnormal, if the data synchronization state is abnormal, executing step S300.

[0009] S300: Lock the disk of the local service node and the disk of the opposite service node, so that the status of the disk of the local service node and the disk of the opposite service node is set to a non-readable and non-writable state.

[0010] S400: Perform fault identification processing on the local service node and the peer service node to determine the faulty service node and the non-faulty service node.

[0011] S500. Move all disks of the failed service node out of the DSC cluster.

[0012] S600: Unlock the disk of the non-faulty service node.

[0013] In an exemplary embodiment of the present application, step S200 includes:

[0014] S210: If the local service node cannot identify the synchronization data of the opposite service node, it is determined that the data synchronization state between the local service node and the opposite service node is an abnormal state, and an abnormal flag is stored on the local service node and the opposite service node, and step S300 is executed.

[0015] In an exemplary embodiment of the present application, step S300 includes:

[0016] S310: If an abnormal flag is stored in the local service node and the opposite service node, the disk of the local service node and the disk of the opposite service node are locked.

[0017] In an exemplary embodiment of the present application, step S400 includes:

[0018] S410: Obtain the network service status of the local service node.

[0019] S420: Determine whether the network service status of the local service node is abnormal.

[0020] If the network service status of the local service node is normal, step S430 is executed.

[0021] If the network service status of the local service node is abnormal, step S490 is executed.

[0022] S430. If the priority of the local service node is higher than the priority of the peer service node, execute step S460.

[0023] If the priority of the local service node is lower than the priority of the peer service node, step S480 is executed.

[0024] S460: Obtain the data communication status between the local service node and the peer service node.

[0025] S470: Determine whether the data communication status between the local service node and the opposite service node is abnormal.

[0026] If the data communication status between the local service node and the opposite service node is abnormal, step S480 is executed.

[0027] If the data communication status between the local service node and the opposite service node is normal, step S490 is executed.

[0028] S480: Determine that the local service node is a normal service node, and determine that the opposite service node is a faulty service node.

[0029] S490: Determine that the local service node is a faulty service node, and determine that the opposite service node is a non-faulty service node.

[0030] In an exemplary embodiment of the present application, step S420 includes:

[0031] S421. Determine whether the system account service of the local service node is normal in the network service state.

[0032] If the system account service of the local service node in the network service state is normal, it is determined that the network service state of the local service node is normal, and step S430 is executed.

[0033] If the system account service of the local service node in the network service state is abnormal, it is determined that the network service state of the local service node is an abnormal state, and step S490 is executed.

[0034] In an exemplary embodiment of the present application, step S470 includes:

[0035] S471. If the local service node receives a data return packet corresponding to the network data packet sent by the opposite service node within n*t0, it is determined that the data communication status between the local service node and the opposite service node is normal, and step S490 is executed; wherein n is the threshold value of the number of network data packets sent by the local service node to the opposite service node; and t0 is the interval time between each two network data packets sent by the local service node to the opposite service node.

[0036] If the local service node does not receive the data return packet corresponding to the network data packet sent by the opposite service node within n*t0, it is determined that the data communication state between the local service node and the opposite service node is abnormal, and step S480 is executed.

[0037] In an exemplary embodiment of the present application, step S430 further includes:

[0038] S431. If t1≤t2, the priority of the local service node is higher than the priority of the peer service node; wherein t1 is the setting time of the local service node in the DSC cluster; t2 is the setting time of the peer service node in the DSC cluster.

[0039] S432: If t1>t2, the priority of the local service node is lower than the priority of the peer service node.

[0040] According to one aspect of the present application, a dual-node DSC cluster fault switching system is provided, including a local service node and a peer service node, wherein the local service node and the peer service node are communicatively connected via a fault switching module.

[0041] The fault switching module comprises:

[0042] A synchronization status acquisition unit, used to acquire the data synchronization status of the local service node and the peer service node in the DSC cluster;

[0043] A fault judgment unit, used to judge the abnormality of the data synchronization status between the local service node and the peer service node;

[0044] A disk locking unit, used to lock the disk of the local service node and the disk of the opposite service node, so that the status of the disk of the local service node and the disk of the opposite service node is set to a non-readable and non-writable state;

[0045] A fault node determination unit, used to perform fault identification processing on the local service node and the peer service node, and determine the faulty service node and the non-faulty service node;

[0046] A fault node processing unit, used to move all disks of a faulty service node out of the DSC cluster;

[0047] The non-faulty node processing unit is used to unlock the disk of the non-faulty service node.

[0048] According to one aspect of the present application, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the dual-node DSC cluster failover method.

[0049] According to one aspect of the present application, an electronic device is provided, including a processor and the non-transitory computer-readable storage medium.

[0050] The present invention has at least the following beneficial effects:

[0051] The dual-node DSC cluster fault switching method of the present invention first determines the data synchronization status of the local service node and the opposite service node in the DSC cluster. If the data synchronization status is abnormal, the disk of the local service node and the disk of the opposite service node are locked, and the disk is suspended to protect the original data, so as to prevent the local service node and the opposite service node from writing data respectively in the fault state, so that the dirty data is written into the corresponding disk, and the fault service node and the non-fault service node are determined, and all the disks of the fault service node are removed from the DSC cluster, and the disks of the non-fault service nodes are unlocked. The non-fault service node can be automatically detected, and the fault automatic switching is performed, so as to realize the fault self-healing of the dual-node DSC cluster, and have high availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 A flowchart of a dual-node DSC cluster failover method provided by an embodiment of the present invention;

[0054] Figure 2 A connection block diagram of a dual-node DSC cluster fault switching system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] The current DSC cluster is a shared storage database cluster system based on a single-node database management system. The DSC cluster enables multiple database instances to be accessed simultaneously. When one or more devices (nodes) fail in a DSC cluster system, the load capacity of the entire DSC cluster will decrease, thereby affecting its stability and availability.

[0057] Existing DSC clusters all achieve fault switching through third-party arbitration. Generally, a cluster needs at least three nodes, two of which are used as the DSC cluster and the remaining nodes are used for storage. In addition, if data redundancy is required, additional nodes need to be added for backup. Therefore, the application cost of this DSC cluster fault switching method is too high. Therefore, a dual-node DSC cluster fault switching method and a dual-node DSC cluster fault switching system are proposed, so that the DSC cluster fault switching can be achieved through only two nodes.

[0058] like Figure 1 As shown, the dual-node DSC cluster failover method comprises the following steps:

[0059] S100: Obtain data synchronization status of a local service node and a peer service node in a DSC cluster.

[0060] The local service node and the opposite service node are two identical service nodes with storage functions in the DSC cluster; the local service node and the opposite service node perform data synchronization through a data synchronization network.

[0061] The DSC cluster in the present application includes a local service node and a peer service node. The data of the local service node and the peer service node are kept synchronized. If one of the nodes is set as a local service node, the other node is set as a peer service node. The names of the local service node and the peer service node in the present application are set only for use and do not represent other meanings.

[0062] S200, determining whether the data synchronization status between the local service node and the peer service node is abnormal. If the data synchronization status is abnormal, executing step S300.

[0063] Further, step S200 includes:

[0064] S210. If the local service node cannot identify the synchronization data of the opposite service node, it is determined that the data synchronization state of the local service node and the opposite service node is an abnormal state (standalone state), and an abnormal identification is stored on the local service node and the opposite service node, and step S300 is executed; the abnormal identification is an identification indicating that the data synchronization state of the current local service node and the opposite service node is abnormal, and the abnormal identification is used as an identification basis for fault judgment.

[0065] The method for determining the data synchronization status of the local service node and the peer service node in step S200 can be determined according to the judgment rules for the synchronization status between nodes of the cluster system in the prior art, such as regularly checking the Pod running status on the node based on the execution logic of PLEG (PodLifecycleEvents Generator) to determine the synchronization status between nodes in the cluster.

[0066] S211. If the local service node can identify the synchronization data of the opposite service node, it is determined that the data synchronization state of the local service node and the opposite service node is normal (connecting state), indicating that the data of the current local service node and the opposite service node are synchronized normally and no abnormality occurs, then the dual-node DSC cluster fault switching method is exited.

[0067] S300: Lock the disk of the local service node and the disk of the opposite service node, so that the status of the disk of the local service node and the disk of the opposite service node is set to a non-readable and non-writable state.

[0068] Lock the disk and suspend it to protect the original data. This prevents writing data to the local service node and the peer service node respectively in a faulty state, which would cause dirty data to be written to the corresponding disk. Locking the disk can also prevent the DSC cluster from having a split-brain phenomenon.

[0069] Further, step S300 includes:

[0070] If an abnormal flag is stored on the local service node and the opposite service node, it means that the data synchronization status of the current local service node and the opposite service node is abnormal, and the disk of the local service node and the disk of the opposite service node are locked.

[0071] S400, perform fault identification processing on the local service node and the opposite service node, and determine the faulty service node and the non-faulty service node from the local service node and the opposite service node; it can be understood that: if the local service node fails, the local service node is determined as the faulty service node and the opposite service node is determined as the non-faulty service node; if the opposite service node fails, the opposite service node is determined as the faulty service node and the local service node is determined as the non-faulty service node.

[0072] Further, step S400 includes:

[0073] S410: Obtain the network service status of the local service node.

[0074] S420: Determine whether the network service status of the local service node is abnormal.

[0075] If the network service status of the local service node is normal, step S430 is executed.

[0076] If the network service status of the local service node is abnormal, step S490 is executed.

[0077] Specifically, in step S420, the method for determining the network service status of the local service node includes:

[0078] S421, determine whether the system account service (network.service service) of the local service node in the network service state is normal; the method for determining the system account service of the local service node in the network service state is the account service determination method of the node of the cluster system in the prior art.

[0079] S422: If the system account service of the local service node in the network service state is normal, that is, the service state is active, it is determined that the network service state of the local service node is normal, and step S430 is executed.

[0080] S423: If the system account service of the local service node in the network service state is abnormal, determine that the network service state of the local service node is an abnormal state, and execute step S490.

[0081] S430. If the priority of the local service node is greater than the priority of the peer service node, execute step S460.

[0082] If the priority of the local service node is lower than the priority of the peer service node, step S480 is executed.

[0083] Specifically, in step S430, the method for determining the priority of the local service node and the peer service node includes:

[0084] S431. If t1≤t2, the priority of the local service node is higher than the priority of the peer service node; wherein t1 is the setting time of the local service node in the DSC cluster; t2 is the setting time of the peer service node in the DSC cluster.

[0085] S432: If t1>t2, the priority of the local service node is lower than the priority of the peer service node.

[0086] The priority of the local service node and the opposite service node is determined by comparing the setting time of the local service node in the DSC cluster with the setting time of the opposite service node in the DSC cluster. The service node with an earlier setting time has a higher corresponding priority.

[0087] Step S430 is one of the embodiments of the method for determining the priority of the local service node and the peer service node. In addition, the standard for priority determination can also be set by setting the serial number of the service node. For example, the priority of service node No. 0 can be set to be higher, and the priority of service node No. 1 can be set to be lower than the priority of service node No. 0. At the same time, the service node with a high priority and normal network service can be defaulted to be a surviving node, that is, a non-faulty service node.

[0088] S460: Obtain the data communication status between the local service node and the peer service node.

[0089] S470: Determine whether the data communication status between the local service node and the opposite service node is abnormal.

[0090] If the data communication state between the local service node and the opposite service node is abnormal, step S480 is executed.

[0091] If the data communication status between the local service node and the opposite service node is normal, step S490 is executed.

[0092] Specifically, in step S470, the method for determining the data communication status between the local service node and the peer service node includes:

[0093] S471. If the local service node receives a data return packet corresponding to the network data packet sent by the opposite service node within n0*t0, it is determined that the data communication status between the local service node and the opposite service node is normal, and step S490 is executed; wherein, n0=T1 / T2, n0 is the threshold value of the number of network data packets sent by the local service node to the opposite service node; t0 is the interval time between each two network data packets sent by the local service node to the opposite service node.

[0094] S472. If the local service node does not receive the data return packet corresponding to the network data packet sent by the opposite service node within n0*t0, it is determined that the data communication state between the local service node and the opposite service node is abnormal, and step S480 is executed.

[0095] T1=T 11 / n1; T1 is the average delay time between network data signals in the network data packet sent by the local service node to the peer service node; T 11It is the sum of all delay times after removing the longest and shortest delay times in the network data packet sent by the local service node to the opposite service node; n1 is the number of network data signals after removing the network data signals corresponding to the longest and shortest delay times in the network data packet sent by the local service node to the opposite service node; T1 takes the average delay time for the purpose of reducing data interference caused by excessive network fluctuations, removing the two network data signals with the longest and shortest delay times and the corresponding delay times in the network data packet, so as to make the determined average delay time of the network data packet more accurate.

[0096] T2=T 22 / n2; T2 is the average delay time between network data signals in the data return packet sent by the peer service node to the local service node; T 22 It is the sum of all delay times after removing the longest and shortest delay times in the data return packet sent by the peer service node to the local service node; n2 is the number of network data signals after removing the network data signals corresponding to the longest and shortest delay times in the data return packet sent by the peer service node to the local service node; in the method for determining T2, the two network data signals with the longest and shortest delay times removed from the network data packet and the corresponding delay times are averaged to make the determined average delay time of the data return packet more accurate.

[0097] When determining the total delay time of T1 and T2, one embodiment may be to obtain the delay time between all network data signals in the network data packet and the data return packet, sort the delay time by size, delete the maximum and minimum delay time, and sum the delay time of the remaining network data signals; another embodiment may also obtain the average delay time by eliminating the maximum and minimum delay times and finding the normal distribution of the remaining network data signals.

[0098] The network data packet is a ping data packet. The local service node sends a ping data packet to the opposite service node, and determines whether the data return packet fed back by the opposite service node is received within the time n0*t0. If the feedback data from the opposite service node is received, it means that the opposite service node has received the network data packet sent by the local service node, indicating that the communication between the local service node and the opposite service node is normal. If the feedback data from the opposite service node is not received, it means that the opposite service node has not received the network data packet sent by the local service node, or the local service node cannot receive the data return packet sent by the opposite service node, indicating that the communication between the local service node and the opposite service node is abnormal, and data connection between the two cannot be established.

[0099] Step S471 is only one of the embodiments of the method for determining the data communication status between the local service node and the peer service node of the present application. In addition to the step S471, the communication status between the two can also be determined by determining the number of data return packets received from the peer service node within a specific time.

[0100] S480: Determine that the local service node is a normal service node, and determine that the opposite service node is a faulty service node.

[0101] S490: Determine that the local service node is a faulty service node, and determine that the opposite service node is a non-faulty service node.

[0102] S500. Move all disks of the faulty service node out of the DSC cluster, set the faulty service node as a dead node, and forcibly kick the dead node out of the DSC cluster, so as to prevent the faulty service node from damaging the original cluster and improve security.

[0103] S600: Unlock the disk of the non-faulty service node, and set the non-faulty service node as a surviving node, so that the non-faulty service node continues to provide read and write functions for the upper layer service.

[0104] The dead nodes are service nodes that need to be kicked out of the DSC cluster, and the surviving nodes are normal service nodes without faults that can continue to stay in the DSC cluster.

[0105] The dual-node DSC cluster fault switching method of the present invention first determines the data synchronization status of the local service node and the opposite service node in the DSC cluster. If the data synchronization status is abnormal, the disk of the local service node and the disk of the opposite service node are locked, and the disk is suspended to protect the original data, so as to prevent the local service node and the opposite service node from writing data respectively in the fault state, so that the dirty data is written into the corresponding disk, and the fault service node and the non-fault service node are determined, and all the disks of the fault service node are moved out of the DSC cluster to prevent damage to the original cluster and improve security, and the disk of the non-fault service node is unlocked to continue to provide read and write functions for the upper layer business. The dual-node DSC cluster fault switching method can automatically detect the non-fault service node, and perform automatic fault switching, so as to realize the fault self-healing of the dual-node DSC cluster, and has high availability.

[0106] A dual-node DSC cluster fault switching system includes a local service node and a peer service node, wherein the local service node and the peer service node are communicatively connected with a fault switching module;

[0107] like Figure 2 As shown, the fault switching module includes:

[0108] A synchronization status acquisition unit, used to acquire the data synchronization status of the local service node and the peer service node in the DSC cluster;

[0109] A fault judgment unit, used to judge the abnormality of the data synchronization status between the local service node and the peer service node;

[0110] A disk locking unit, used to lock the disk of the local service node and the disk of the opposite service node, so that the status of the disk of the local service node and the disk of the opposite service node is set to a non-readable and non-writable state;

[0111] A fault node determination unit, used to perform fault identification processing on the local service node and the peer service node, and determine the faulty service node and the non-faulty service node;

[0112] A fault node processing unit, used to move all disks of a faulty service node out of the DSC cluster;

[0113] The non-faulty node processing unit is used to unlock the disk of the non-faulty service node.

[0114] The dual-node DSC cluster fault switching system of the present invention realizes cross-node dual-copy redundancy of data through distributed storage, ensures data security, and only requires two nodes to build a cluster architecture. If any node goes down, the faulty service node can be automatically identified and removed from the DSC cluster without affecting the availability of the DSC cluster and the security of the data.

[0115] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0116] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0117] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A dual-node DSC cluster failover method, It is characterized in that The method comprises the following steps: S100, obtaining data synchronization status of a local service node and a peer service node in a DSC cluster; the local service node and the peer service node are two identical service nodes with storage functions in the DSC cluster; the local service node and the peer service node perform data synchronization through a data synchronization network; S200, determining whether the data synchronization state is abnormal, if the data synchronization state is abnormal, executing step S300; S300, locking the disk of the local service node and the disk of the peer service node, so that the status of the disk of the local service node and the disk of the peer service node are set to a non-readable and non-writable state; S400, performing fault identification processing on the local service node and the peer service node to determine a faulty service node and a non-faulty service node; S500, moving all disks of the failed service node out of the DSC cluster; S600, unlocking the disk of the non-faulty service node; Wherein, the step S400 includes: S410, obtaining the network service status of the local service node; S420, determining whether the network service status of the local service node is abnormal; If the network service status of the local service node is normal, execute step S430; If the network service status of the local service node is abnormal, execute step S490; S430. If the priority of the local service node is higher than the priority of the peer service node, execute step S460; If the priority of the local service node is lower than the priority of the peer service node, executing step S480; S460, obtaining the data communication status between the local service node and the opposite service node; S470, determining whether the data communication status between the local service node and the peer service node is abnormal; If the data communication state between the local service node and the opposite service node is abnormal, executing step S480; If the data communication status between the local service node and the opposite service node is normal, executing step S490; S480, determining that the local service node is a normal service node, and determining that the opposite service node is a faulty service node; S490: Determine that the local service node is a faulty service node, and determine that the opposite service node is a non-faulty service node.

2. The method according to claim 1, It is characterized in that The step S200 includes: S210: If the local service node cannot identify the synchronization data of the opposite service node, it is determined that the data synchronization state of the local service node and the opposite service node is an abnormal state, and an abnormal flag is stored in the local service node and the opposite service node, and step S300 is executed.

3. The method according to claim 2, It is characterized in that The step S300 includes: S310: If the local service node and the opposite service node store abnormal flags, lock the disks of the local service node and the opposite service node.

4. The method according to claim 1, It is characterized in that The step S420 includes: S421, determining whether the system account service of the local service node in the network service state is normal; If the system account service of the local service node in the network service state is normal, determining that the network service state of the local service node is normal, and executing step S430; If the system account service of the local service node in the network service state is abnormal, it is determined that the network service state of the local service node is an abnormal state, and step S490 is executed.

5. The method according to claim 1, It is characterized in that The step S470 includes: S471: If the local service node is in n×t 0 If a data return packet corresponding to the network data packet sent by the opposite service node is received within t, it is determined that the data communication status between the local service node and the opposite service node is normal, and step S490 is executed; wherein n is the number threshold of the network data packets sent by the local service node to the opposite service node; t 0 The interval time between each two network data packets sent by the local service node to the opposite service node; S472: If the local service node is n×t 0 If the data return packet corresponding to the network data packet sent by the opposite service node is not received within 10 seconds, it is determined that the data communication state between the local service node and the opposite service node is abnormal, and step S480 is executed.

6. The method according to claim 1, It is characterized in that The step S430 further includes: S431, if t 1 ≤t 2 , then the priority of the local service node is higher than the priority of the peer service node; where t 1 is the setting time of the local service node in the DSC cluster; t 2 Setting time for the opposite service node in the DSC cluster; S432, if t 1 >t 2 , the priority of the local service node is lower than the priority of the peer service node.

7. A dual-node DSC cluster failover system, It is characterized in that It includes a local service node and a peer service node, wherein the local service node and the peer service node are communicatively connected with a fault switching module; The fault switching module comprises: A synchronization status acquisition unit, used to acquire the data synchronization status of the local service node and the peer service node in the DSC cluster; A fault judgment unit, used to judge the abnormality of the data synchronization status between the local service node and the peer service node; A disk locking unit, used to lock the disk of the local service node and the disk of the opposite service node, so that the status of the disk of the local service node and the disk of the opposite service node is set to a non-readable and non-writable state; A fault node determination unit, used to perform fault identification processing on the local service node and the peer service node, and determine the faulty service node and the non-faulty service node; A fault node processing unit, used to move all disks of a faulty service node out of the DSC cluster; A non-faulty node processing unit, used to unlock the disk of the non-faulty service node; The fault node determination unit performs fault identification processing on the local service node and the peer service node to determine the faulty service node and the non-faulty service node, including: S410, obtaining the network service status of the local service node; S420, determining whether the network service status of the local service node is abnormal; If the network service status of the local service node is normal, execute step S430; If the network service status of the local service node is abnormal, execute step S490; S430. If the priority of the local service node is higher than the priority of the peer service node, execute step S460; If the priority of the local service node is lower than the priority of the peer service node, executing step S480; S460, obtaining the data communication status between the local service node and the opposite service node; S470, determining whether the data communication status between the local service node and the peer service node is abnormal; If the data communication state between the local service node and the opposite service node is abnormal, executing step S480; If the data communication status between the local service node and the opposite service node is normal, executing step S490; S480, determining that the local service node is a normal service node, and determining that the opposite service node is a faulty service node; S490: Determine that the local service node is a faulty service node, and determine that the opposite service node is a non-faulty service node.

8. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium. It is characterized in that The at least one instruction or the at least one program is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.

9. An electronic device, It is characterized in that The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 8.

Images