Fault detection method and device, computer device, storage medium and program product

Abstract

The application relates to a fault detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: obtaining node information sets of Redis nodes in a Redis cluster in a remote dictionary service in a first control environment; for each Redis node, determining a node type of the Redis node recognized by each Redis node according to the node information sets; and determining a fault detection result of the Redis cluster according to a first node number of the Redis nodes recognized as the same node type.

Description

Technical Field

[0001] This application relates to the field of fault detection technology, and in particular to a fault detection method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] In modern industrial and information systems, the control module, as a core component, is responsible for monitoring, regulating, and coordinating the operation of various subsystems. Its stability and reliability directly affect the performance and security of the entire system. However, due to factors such as prolonged operation, complex environments, and equipment aging, control modules may encounter various faults, which can lead to serious consequences such as system performance degradation and data loss. Therefore, the detection and handling of control module faults is particularly important.

[0003] Currently, fault detection is usually performed on the controller module. However, in addition to the controller, the remote dictionary server (Redis) cluster also participates in the operation. Therefore, how to detect Redis cluster faults has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] Therefore, it is necessary to provide a fault detection method, device, computer equipment, storage medium, and program product that can detect Redis cluster faults in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a fault detection method. The method includes:

[0006] Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node;

[0007] For each Redis node, based on the information set of each node, determine the node type that the Redis node is recognized by each Redis node; the node type includes at least one of being in the Redis cluster, not being in the Redis cluster, master node, and standby node;

[0008] The fault detection result of the Redis cluster is determined based on the number of first nodes identified as being of the same node type.

[0009] In one embodiment, determining the fault detection result of the Redis cluster based on the number of first nodes identified as the same node type includes:

[0010] If the number of the first node is greater than or equal to the first preset threshold, then the same node type is taken as the target type of the Redis node, and the fault detection result is determined to be that the Redis cluster is normal; the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0011] If the number of first nodes of the same node type is equal to half of the total number, then the fault detection result is determined to be a fault in the Redis cluster.

[0012] In one embodiment, the method further includes:

[0013] If the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type Redis node deployed on a single device as the master node;

[0014] If the number of the second node is greater than the second preset threshold, then the Redis node of the target type as the master node on the single device will be migrated.

[0015] In one embodiment, migrating the target type of Redis node as the master node on the single device includes:

[0016] Identify the nodes to be migrated from the Redis nodes that are the primary nodes for this target type;

[0017] Determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located;

[0018] If the number of the third node is less than the second preset threshold, then the node to be migrated will be migrated to the target device.

[0019] In one embodiment, the method further includes:

[0020] If the fault detection result indicates that the Redis cluster is faulty, obtain the storage location and storage method of the first resource data of the first control environment from the resource data table;

[0021] Based on the storage location and storage method, the first resource data is obtained and stored in the second control environment; the second control environment is a backup control environment for the first control environment.

[0022] In one embodiment, the method further includes:

[0023] If the fault detection result is determined to be a fault in the Redis cluster, the first business data of the first business process in the first control environment is obtained from the status data table.

[0024] The first business process is restored in the second control environment based on the first business data.

[0025] Secondly, this application also provides a fault detection device. The device includes:

[0026] The first acquisition module is used to acquire the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node.

[0027] The first determining module is used to determine the node type of each Redis node based on the information set of each node; the node type includes at least one of being in the Redis cluster, not being in the Redis cluster, master node, and standby node;

[0028] The second determining module is used to determine the fault detection result of the Redis cluster based on the number of first nodes that are identified as the same node type.

[0029] Thirdly, this application also provides a computer device, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the above methods.

[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0031] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0032] The aforementioned fault detection method, apparatus, computer equipment, storage medium, and program product acquire the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment. For each Redis node, based on the node information set, the node type recognized by each Redis node is determined. Then, based on the number of first nodes that are recognized as the same node type, the fault detection result of the Redis cluster is determined, thereby realizing the detection of faults in the Redis cluster. Attached Figure Description

[0033] Figure 1 This is an internal structural diagram of a computer device provided in an embodiment of this application;

[0034] Figure 2 This is a schematic flowchart of a fault detection method provided in an embodiment of this application;

[0035] Figure 3 This is a schematic diagram of the structure of a control module provided in an embodiment of this application;

[0036] Figure 4 This is a flowchart illustrating a method for determining fault detection results provided in an embodiment of this application;

[0037] Figure 5 This is a flowchart illustrating a master node migration method provided in an embodiment of this application;

[0038] Figure 6 This is a flowchart illustrating another master node migration method provided in an embodiment of this application;

[0039] Figure 7 This is a flowchart illustrating a Redis cluster fault detection and dynamic rebalancing method provided in an embodiment of this application.

[0040] Figure 8 This is a flowchart illustrating a resource data migration method provided in an embodiment of this application;

[0041] Figure 9 This is a flowchart illustrating a business process recovery method provided in an embodiment of this application;

[0042] Figure 10 This is a flowchart illustrating another business process recovery method provided in an embodiment of this application;

[0043] Figure 11 This is a structural block diagram of a fault detection device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] In modern industrial and information systems, the control module, as a core component, is responsible for monitoring, regulating, and coordinating the operation of various subsystems. Its stability and reliability directly affect the performance and security of the entire system. However, due to factors such as prolonged operation, complex environments, and equipment aging, control modules may encounter various faults, which can lead to serious consequences such as system performance degradation and data loss. Therefore, the detection and handling of control module faults is particularly important.

[0046] Currently, fault detection is usually performed on the controller module. However, in addition to the controller, the remote dictionary server (Redis) cluster also participates in the operation. Therefore, how to detect Redis cluster faults has become a technical problem that urgently needs to be solved in this field.

[0047] The fault detection method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown. Figure 1 This is an internal structure diagram of a computer device provided in an embodiment of this application. The computer device may be a server, and its internal structure diagram may be as follows: Figure 1 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a fault detection method.

[0048] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0049] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a fault detection method provided in an embodiment of this application. This method can be applied to... Figure 1 The method, using a computer device, includes the following steps:

[0050] S201, Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment.

[0051] The node information set includes node information for each Redis node.

[0052] For example, refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a control module provided in an embodiment of this application. Figure 3As shown, the control module can include two control environments, deployed as mirror images. The first control environment is the primary control environment, and the second control environment is a backup control environment for the first. When the first control environment fails, the system can switch to the second control environment to continue executing the corresponding business processes. Redis in the control module typically handles data caching, distributed locking, and other functions, and is a crucial component of the control module.

[0053] In one embodiment, Redis in the control environment can be deployed in a cluster. For example... Figure 3 As shown, a control environment includes at least one Redis cluster, and a Redis cluster includes multiple Redis nodes.

[0054] In this embodiment, each Redis node in the Redis cluster of the first control environment stores a node information set, which includes the node information of each Redis node in the Redis cluster. That is, assuming there are N Redis nodes in the Redis cluster of the first control environment, each Redis node stores a node information set, which includes the node information of the N Redis nodes.

[0055] In this embodiment, the node information of a Redis node may include, for example, the device identifier (device IP) of the device on which the Redis node is deployed, the port number (port) of the Redis node, its role (role), and its state (state). If the role of the Redis node is the master node, then the node information of the Redis node may also include the identifier (slaveIp) and port number (slavePort) of its corresponding slave node.

[0056] The roles of Redis nodes can include, for example, master nodes and slave nodes; the states can include, for example, being in a Redis cluster (connected) and not being in a Redis cluster (failed).

[0057] Optionally, the aforementioned node information set can be stored in a file named "nodes-nodeport_number.conf". The node information of each Redis node can be determined by parsing the data stored in the "nodes-nodeport_number.conf" file, and this node information can be stored as map data. For example, the map data format could be: map <device ip,list <port:role:state:slaveip:slaveport>>

[0058] S202, for each Redis node, determine the node type that the Redis node is recognized by each Redis node based on the information set of each node.

[0059] The node type includes at least one of the following: being in a Redis cluster, not being in a Redis cluster, master node, and backup node.

[0060] In this embodiment of the application, the node type cannot include both master node and backup node at the same time, and cannot include both nodes that are in a Redis cluster and nodes that are not in a Redis cluster at the same time.

[0061] In one embodiment, since the role or state of the same Redis node may differ from the perspective of different Redis nodes, the node information for the same Redis node stored in different node information sets may also differ. In other words, for each Redis node, the identified node type may differ depending on the different node information sets.

[0062] For example, assuming there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6, then for node 1, the node type identified by node 1 can be determined from the node information set of node 1, the node type identified by node 2 from the node information set of node 2, the node type identified by node 3 from the node information set of node 3, the node type identified by node 4 from the node information set of node 4, the node type identified by node 5 from the node information set of node 5, and the node type identified by node 6 from the node information set of node 6.

[0063] S203. Determine the fault detection result of the Redis cluster based on the number of first nodes identified as the same node type.

[0064] In this embodiment, a first preset threshold can be determined based on the total number of Redis nodes in the Redis cluster, and then the fault detection result of the Redis cluster can be determined based on the first preset threshold and the first number of nodes.

[0065] Optionally, half of the total number of Redis nodes in the Redis cluster can be set as the first preset threshold.

[0066] Alternatively, the first preset threshold can be determined by adding 1 to half the total number of Redis nodes in the Redis cluster. That is, the first preset threshold can be determined using the following formula:

[0067] z = m / 2

[0068] n = z + 1

[0069] Where m is the total number of Redis nodes in the Redis cluster, and n is the first preset threshold.

[0070] In one embodiment, with n as the first preset threshold, if the number of first nodes is greater than or equal to the first preset threshold, then the same node type is determined as the target type of Redis nodes, and the fault detection result is determined to be that the Redis cluster is normal. If the number of first nodes cannot be greater than or equal to the first preset threshold, that is, if the target type of Redis nodes cannot be determined, the fault detection result is determined to be that the Redis cluster is faulty.

[0071] For example, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. Node 1 is identified as a master node by node 1, node 2 as a master node, node 3 as a master node, node 4 as a backup node, node 5 as a backup node, and node 6 as a master node. Then the number of nodes that identify node 1 as a master node is 4, which is equal to the first preset threshold. Therefore, the master node is determined as the target type of node 1.

[0072] Alternatively, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. If node 1 is identified as a master node by node 1, node 2 as a master node, node 3 as a backup node, node 4 as a backup node, node 5 as a backup node, and node 6 as a master node, then the number of nodes identified as master nodes by node 1 is 3, and the number of nodes identified as backup nodes by node 1 is also 3. Since the number of first nodes cannot be greater than or equal to the first preset threshold, the fault detection result is determined to be a Redis cluster fault.

[0073] In another embodiment, with z as the first preset threshold, if the number of first nodes is greater than the first preset threshold, then the same node type is determined as the target type of Redis node, and the fault detection result is determined to be that the Redis cluster is normal; if the number of first nodes is equal to the first preset threshold, then the fault detection result is determined to be that the Redis cluster is faulty.

[0074] For example, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. If node 1 is identified as the master node, node 2 is identified as the master node, node 3 is identified as the backup node, node 4 is identified as the backup node, node 5 is identified as the master node, and node 6 is identified as the master node, then the first preset threshold is 3. The number of nodes that identify node 1 as the master node is 4, which is greater than the first preset threshold. Therefore, the master node is determined as the target type of node 1.

[0075] Alternatively, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. If node 1 is identified as a master node by node 1, node 2 as a master node, node 1 as a backup node by node 3, node 1 as a backup node by node 4, node 1 as a backup node by node 5, and node 1 as a master node by node 6, then the first preset threshold is 3. The number of nodes identified as master nodes by node 1 is 3, and the number of nodes identified as backup nodes by node 1 is also 3. Since the number of first nodes equals the first preset threshold, the fault detection result is determined to be a Redis cluster fault.

[0076] In this embodiment of the application, by obtaining the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment, for each Redis node, the node type of the Redis node is determined by each Redis node according to the node information set, and then the fault detection result of the Redis cluster is determined according to the number of first nodes that are identified as the same node type.

[0077] Reference Figure 4 , Figure 4 This is a flowchart illustrating a method for determining fault detection results provided in an embodiment of this application. This embodiment relates to a possible implementation of determining the fault detection result of a Redis cluster based on the number of first nodes identified as belonging to the same node type. Based on the above embodiment, S203 includes the following steps:

[0078] S401, if the number of first nodes is greater than or equal to the first preset threshold, then the same node type is used as the target type of Redis node, and the fault detection result is determined to be that the Redis cluster is normal.

[0079] The first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0080] In this embodiment, the first preset threshold can be determined according to the following formula:

[0081] z = m / 2

[0082] n = z + 1

[0083] Where m is the total number of Redis nodes in the Redis cluster, and n is the first preset threshold.

[0084] In one embodiment, if the number of first nodes is greater than or equal to a first preset threshold, then the same node type is determined as the target type of Redis node, and the fault detection result is determined to be that the Redis cluster is normal.

[0085] For example, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. Node 1 is identified as a master node by node 1, node 2 as a master node, node 3 as a master node, node 4 as a backup node, node 5 as a backup node, and node 6 as a master node. Then the number of nodes that identify node 1 as a master node is 4, which is equal to the first preset threshold. Therefore, the master node is determined as the target type of node 1.

[0086] It's important to note that a Redis cluster can only be considered healthy if all Redis nodes in the cluster can determine the target type. If even one Redis node cannot determine its target type, the cluster is considered faulty.

[0087] S402. If the number of first nodes of the same node type is equal to half of the total number, then the fault detection result is determined to be a Redis cluster fault.

[0088] In this embodiment of the application, if the number of first nodes of the same node type is equal to half of the total number, the fault detection result is determined to be a Redis cluster fault.

[0089] For example, suppose there are 6 Redis nodes in the Redis cluster: node 1, node 2, node 3, node 4, node 5, and node 6. The first preset threshold is 4. If node 1 is identified as a master node, node 2 is identified as a master node, node 3 is identified as a backup node, node 4 is identified as a backup node, node 5 is identified as a backup node, and node 6 is identified as a master node, then the number of nodes identified as master nodes is 3, and the number of nodes identified as backup nodes is also 3. The number of first nodes is equal to half of the total number of nodes. Therefore, the fault detection result is determined to be a Redis cluster fault.

[0090] In this embodiment, a first preset threshold is determined based on the total number of Redis nodes in the Redis cluster. If the number of nodes is greater than or equal to the first preset threshold, the same node type is used as the target type of the Redis node, and the fault detection result is determined to be that the Redis cluster is normal. If the number of nodes of the same node type is equal to half of the total number, the fault detection result is determined to be that the Redis cluster is faulty. This enables the detection of faults in the Redis cluster, and allows for timely switching of the control environment when a Redis cluster fails, thereby improving the stability and reliability of the controller service.

[0091] Reference Figure 5 , Figure 5 This is a flowchart illustrating a master node migration method provided in an embodiment of this application. Based on the above embodiment, the method further includes the following steps:

[0092] S501, if the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type master Redis node deployed on a single device.

[0093] Optionally, a Redis cluster typically includes M Redis nodes with target type as master nodes, and M Redis nodes with target type as backup nodes, corresponding to each master node. Here, M is an odd number. The 2M Redis nodes in a Redis cluster can be deployed on several devices.

[0094] Ideally, 2 million Redis nodes should be deployed on 2 million independent devices. However, with insufficient resources, multiple Redis nodes may be deployed on a single device. If the master nodes are overly concentrated—that is, too many master nodes are deployed on the same device—a failure of that device will cause the Redis cluster to fail. Therefore, it is necessary to migrate the concentrated master nodes.

[0095] In this embodiment, if the fault detection result indicates that the Redis cluster is normal, the target type of each Redis node in the Redis cluster can be determined. Then, the number of second nodes with the target type of the primary Redis node deployed on a single device can be determined. Here, a single device can be any device used to deploy the Redis nodes.

[0096] S502, if the number of second nodes is greater than the second preset threshold, then migrate the Redis nodes of the target type as master nodes on a single device.

[0097] For example, assuming the number of Redis nodes with the target type as master nodes in the Redis cluster is z, and the total number of devices deploying Redis nodes is x, then when z < x, the second preset threshold can be set to 2, that is, when the number of second nodes is greater than 2, the Redis nodes with the target type as master nodes on a single device are migrated.

[0098] Alternatively, when z ≧ x, the second preset threshold can be set to z / x + ((z % x) != 0 ? 1 : 0), that is, when the number of second nodes is greater than z / x + ((z % x) != 0 ? 1 : 0), the Redis node with the target type as the master node on a single device is migrated.

[0099] Here, " / " is the division by integer, meaning it only keeps the integer part of the quotient of z divided by x, discarding the decimal part without rounding. "(z % x) != 0 ? 1 : 0" is a ternary expression, meaning that if the remainder of z divided by x is not equal to 0 (i.e., z is not divisible by x), the result of the expression is 1; otherwise, the result of the expression is 0.

[0100] In one possible implementation, when determining which Redis node on a single device is the primary node and needs to be migrated, all primary Redis nodes on the single device can be traversed. First, one primary Redis node on the single device is randomly selected as the node to be migrated. Then, the number of third nodes on the target device corresponding to the backup node to be migrated is determined. This number is compared with a second preset threshold. If the number of third nodes is less than the second preset threshold, the primary node becomes a backup node, and the backup node becomes the primary node, thus achieving the migration of the primary node. If the number of third nodes is not less than the second preset threshold, another primary Redis node on the single device is selected as the primary node, and the process of comparing the number of third nodes with the second preset threshold is repeated until a primary node that can be migrated is found, and the migration of the primary node is completed.

[0101] In this embodiment, when the fault detection result indicates that the Redis cluster is normal, the number of second nodes of the target type master Redis nodes deployed on a single device is obtained. If the number of second nodes is greater than a second preset threshold, the target type master Redis nodes on the single device are migrated. This enables the migration of master nodes on a single device when the master node deployment is overly concentrated, avoiding the failure of the Redis cluster due to a single device failure, and improving the security and reliability of the control module operation.

[0102] Reference Figure 6 , Figure 6 This is a flowchart illustrating another master node migration method provided in this application embodiment. This embodiment relates to a possible implementation of how to migrate a Redis node on a single device whose target type is a master node. Based on the above embodiment, S502 includes the following steps:

[0103] S601, determine the node to be migrated from the Redis nodes whose target type is the master node.

[0104] In one embodiment, when it is determined that a Redis node of the target type is a master node on a single device needs to be migrated, one of the Redis nodes of the target type is a master node on the single device can be selected as the node to be migrated.

[0105] S602, determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located.

[0106] In this embodiment, the node information of the backup node corresponding to the node to be migrated can be determined based on the node information of the node to be migrated, and then the number of third nodes of the primary node on the target device where the backup node is located can be determined based on the node information of the backup node.

[0107] S603, if the number of third nodes is less than the second preset threshold, then the node to be migrated will be migrated to the target device.

[0108] In one embodiment, the number of third nodes can be compared with a second preset threshold. If the number of third nodes is less than the second preset threshold, the node to be migrated is changed to a backup node, and the backup node corresponding to the node to be migrated is changed to a master node, so as to realize the migration of the master node.

[0109] In another embodiment, if the number of third nodes is not less than the second preset threshold, then another Redis node among the Redis nodes of the target type as the main node on a single device is taken as the node to be migrated, and then the above comparison process of the number of third nodes and the second preset threshold is repeated until a main node that can be migrated is found and the migration of the main node is completed.

[0110] In this embodiment, the node to be migrated is determined from the Redis nodes of the target type as master nodes, and the number of third nodes of the master node on the target device where the backup node corresponding to the node to be migrated is located is determined. If the number of third nodes is less than a second preset threshold, the node to be migrated is migrated to the target device. This enables the migration of master nodes on a single device when the master node deployment on a single device is too concentrated, avoiding the failure of the Redis cluster due to the failure of a single device, and improving the security and reliability of the control module operation.

[0111] To provide a clearer description of the embodiments of this application, the following is combined with... Figure 7 An example is provided. Figure 7 This is a flowchart illustrating a Redis cluster fault detection and dynamic rebalancing method provided in an embodiment of this application. Figure 7 As shown, node information for each Redis node in the Redis cluster can be obtained and stored in the form of map data. Then, based on the node information of each Redis node in the cluster, a node information set is determined. Based on the information in each node information set, it is determined whether the Redis cluster is faulty. If the Redis cluster is faulty, a fault handling process is initiated, switching to a second control environment to continue executing the corresponding business processes. When the Redis cluster is normal (i.e., not faulty), it is determined whether the master nodes deployed on each device are reasonable. If it is determined that the master nodes deployed on a single device are unreasonable, the Redis node on that device with the target type as the master node is migrated. Then, the map data of the Redis nodes with the target type as the master node on that single device is traversed to find the node to be migrated. Based on the list in the map data of the node to be migrated, the corresponding backup node is determined. It is determined whether the target device where the backup node is located meets the migration conditions, i.e., the number of third nodes is compared with a second preset threshold. If the number of third nodes is less than the second preset threshold, the target device where the backup node is located meets the migration conditions, the node to be migrated is changed to a backup node, and the backup node corresponding to the node to be migrated is changed to a master node, thus realizing the migration of the master node. If the target device where the backup node is located does not meet the migration conditions, repeat the above steps until a master node that can be migrated is found and the migration of the master node is completed.

[0112] Reference Figure 8 , Figure 8 This is a flowchart illustrating a resource data migration method provided in an embodiment of this application. Based on the above embodiment, the method further includes the following steps:

[0113] S801, if the fault detection result is determined to be a Redis cluster fault, retrieve the storage location and storage method of the first resource data of the first control environment from the resource data table.

[0114] In this embodiment of the application, the resource type, resource identifier, storage location, and storage method of the first resource data temporarily stored in the Redis cluster in the first control environment can be saved in the resource data table.

[0115] For example, the resource data table can be as shown in Table 1 below. The first column of Table 1 is used to store the resource type of the first resource data, the second column is used to store the resource identifier of the first resource data, the third column is used to store the storage location of the first resource data, and the fourth column is used to store the storage method of the first resource data. The storage location of the first resource data can be an entry in the data table storing the first resource data, and the storage method can be a combination of the fields in the data table storing the first resource data.

[0116]

[0117] Table 1

[0118] In one embodiment, if the fault detection result indicates a Redis cluster failure, it is necessary to switch to the second control environment to continue executing the corresponding business process. At this time, the storage location and storage method of the first resource data can be obtained from the resource data table.

[0119] S802, based on the storage location and storage method, obtain the first resource data and store the first resource data in the second control environment.

[0120] The second control environment is a backup control environment for the first control environment.

[0121] In one embodiment, the storage locations of the first resource data can be traversed, and the resource data stored in the storage locations can be combined according to the storage method to restore the first resource data.

[0122] In addition, the first resource data can be stored in the second control environment according to the storage location and storage method.

[0123] In this embodiment, when the fault detection result is determined to be a Redis cluster fault, the storage location and storage method of the first resource data in the first control environment are obtained from the resource data table. Based on the storage location and storage method, the first resource data is obtained and stored in the second control environment. This enables the first resource data to be restored in the second control environment when the first control environment fails and the system switches to the second control environment, ensuring the continuity of business operations and enabling the control module to quickly resume operations after the control environment switch.

[0124] Reference Figure 9 , Figure 9 This is a flowchart illustrating a business process recovery method provided in an embodiment of this application. Based on the above embodiment, the method further includes the following steps:

[0125] S901, if the fault detection result is determined to be a Redis cluster fault, retrieve the first business data of the first business process in the first control environment from the status data table.

[0126] In this embodiment, the business process of the control module can be divided into five steps: parameter validation, configuration construction, configuration distribution, linkage processing, and data storage. Parameter validation checks the existence, format correctness, and matching of associated attributes of the input parameters. The input is the input parameters, and the output is the error information detected. Configuration construction is the process of constructing the configuration type, configuration parameters, and configuration format of the business according to the input parameters and business type requirements. The input is the business type and key parameters, and the output is the configuration format pointing to the device. Configuration distribution pushes the business configuration to the device pointed to by the configuration constructed in the previous step and records the push result. The input is the business configuration, and the output is the configuration push result. Linkage processing is when a business requires joint processing by multiple services, and the linkage processing service is invoked to notify it to perform related business processing. The input is the business type and key business parameters, and the output is the processing result of the related service. Data storage is the process of saving business-related data. The input is the data generated by the business, and the output is the storage result. All five steps must comply with the idempotency principle and maintain a sequential execution order for each business.

[0127] In one embodiment, when the first business process arrives at the first control environment, the first business data of the first business process can be stored in a status data table.

[0128] For example, the status data table can be as shown in Table 2 below. The first column of Table 2 is the business type, which is used to store the types of business; the second column is the input parameters, which are assembled and saved after being marked with colons; the third column is the business step, which records which of the above five steps the current business is in; the fourth column is the input, which records the input of the current step, with multiple parameters separated by semicolons; the fifth column is the status, which records the execution status of the current step, divided into three types: processing, success, and failure; the sixth column records the control environment of the processing, divided into the first control environment and the second control environment.

[0129]

[0130] Table 2

[0131] S902, Restore the first business process in the second control environment based on the first business data.

[0132] For example, in one embodiment, if the fault detection result indicates a Redis cluster failure, it is necessary to switch to the second control environment to continue executing the corresponding business process. At this time, the sixth column of the status data table can be traversed, and the first business process in the first control environment can be selected based on the control environment stored in the sixth column. Then, the execution status of the first business process can be obtained. If the execution status of the first business process is successful, the data is deleted from the status data table; if the execution status of the first business process is failed, the data is deleted from the status data table after the rollback process; if the execution status of the first business process is "processing", the first business process can be restored in the second control environment based on the first business data in the status data table.

[0133] For example, refer to Figure 10 , Figure 10 This is a flowchart illustrating another business process recovery method provided in an embodiment of this application. For example... Figure 10 As shown, if the fault detection result indicates a Redis cluster failure, it is necessary to switch to the second control environment to continue executing the corresponding business process. At this time, the first business process can be retrieved from the database. If the existence of the first business process is confirmed, the sixth column of the status data table is traversed, and the first business process in the first control environment is selected based on the control environment stored in the sixth column. Then, the execution status of the first business process is obtained. If the execution status of the first business process is successful, the data is deleted from the status data table; if the execution status of the first business process is failed, the rollback process is followed and the data is deleted from the status data table; if the execution status of the first business process is "processing", the first business process can be restored in the second control environment based on the first business data in the status data table.

[0134] In one embodiment, the data stored in the databases of the first control environment and the second control environment can be reconciled periodically: the status data tables in the first and second control environments are traversed, the quantity of business data in each status data table is counted, and the quantity of business data in the first and second control environments is compared. If the quantity is the same, the data is considered consistent; if the quantity is different, the configuration may be different. If the quantity is inconsistent in three consecutive reconciliations, a detailed reconciliation is triggered, and the business data in the status data tables of the first and second control environments is compared one by one. If the business data is inconsistent, an alarm message is generated to remind relevant management personnel that the business data in the first and second control environments is inconsistent, so that relevant management personnel can take timely measures.

[0135] In this embodiment of the application, when the fault detection result is determined to be a Redis cluster fault, the first business data of the first business process in the first control environment is obtained from the status data table, and the first business process is restored in the second control environment based on the first business data. This enables the first business process to be restored in the second control environment when the first control environment fails and the switch is made to the second control environment, thus ensuring the continuity of business and enabling the control module to quickly resume operations after the control environment is switched.

[0136] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0137] Based on the same inventive concept, this application also provides a fault detection device for implementing the fault detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more fault detection device embodiments provided below can be found in the limitations of the fault detection method described above, and will not be repeated here.

[0138] In one embodiment, such as Figure 11 As shown, Figure 11 This is a structural block diagram of a fault detection device provided in an embodiment of this application. The device 1100 includes:

[0139] The first acquisition module 1101 is used to acquire the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node.

[0140] The first determining module 1102 is used to determine the node type of each Redis node based on the information set of each node; the node type includes at least one of being in a Redis cluster, not being in a Redis cluster, master node, and backup node.

[0141] The second determining module 1103 is used to determine the fault detection result of the Redis cluster based on the number of first nodes that are identified as the same node type.

[0142] In one embodiment, the second determining module 1103 includes:

[0143] The first determining unit is used to determine the same node type as the target type of Redis node if the number of first nodes is greater than or equal to the first preset threshold, and to determine the fault detection result as the Redis cluster being normal; the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0144] The second determining unit is used to determine the fault detection result as a Redis cluster fault if the number of first nodes of the same node type is equal to half of the total number.

[0145] In one embodiment, the device 1100 further includes:

[0146] The second acquisition module is used to obtain the number of second nodes of the Redis nodes with the target type as the master node deployed on a single device, when the fault detection result is determined to be that the Redis cluster is normal.

[0147] The migration module is used to migrate Redis nodes on a single device whose target type is the master node if the number of second nodes is greater than the second preset threshold.

[0148] In one embodiment, the migration module includes:

[0149] The third determining unit is used to determine the node to be migrated from the Redis nodes whose target type is the master node.

[0150] The fourth determining unit is used to determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located.

[0151] The migration unit is used to migrate the node to be migrated to the target device if the number of third nodes is less than a second preset threshold.

[0152] In one embodiment, the device 1100 further includes:

[0153] The third acquisition module is used to obtain the storage location and storage method of the first resource data of the first control environment from the resource data table when the fault detection result is determined to be a Redis cluster fault.

[0154] The storage module is used to acquire first resource data and store the first resource data in the second control environment according to the storage location and storage method; the second control environment is a backup control environment for the first control environment.

[0155] In one embodiment, the device 1100 further includes:

[0156] The fourth acquisition module is used to obtain the first business data of the first business process in the first control environment from the status data table when the fault detection result is determined to be a Redis cluster fault.

[0157] The recovery module is used to restore the first business process in the second control environment based on the first business data.

[0158] Each module in the aforementioned fault detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0159] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0160] Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node;

[0161] For each Redis node, based on the information set of each node, determine the node type that the Redis node is recognized by each Redis node; the node type includes at least one of the following: in a Redis cluster, not in a Redis cluster, master node, and standby node;

[0162] The fault detection result of the Redis cluster is determined based on the number of first nodes identified as being of the same node type.

[0163] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0164] If the number of first nodes is greater than or equal to the first preset threshold, then the same node type will be used as the target type of Redis nodes, and the fault detection result will be determined as the Redis cluster being normal; the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0165] If the number of first nodes of the same node type is equal to half the total number, then the fault detection result is determined to be a Redis cluster fault.

[0166] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0167] If the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type master Redis node deployed on a single device;

[0168] If the number of second nodes is greater than the second preset threshold, then migrate the Redis nodes of the target type as master nodes on a single device.

[0169] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0170] Identify the nodes to be migrated from the Redis nodes whose target type is the master node;

[0171] Determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located;

[0172] If the number of third nodes is less than the second preset threshold, the node to be migrated will be migrated to the target device.

[0173] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0174] If the fault detection result indicates a Redis cluster fault, retrieve the storage location and storage method of the first resource data of the first control environment from the resource data table;

[0175] Based on the storage location and storage method, the first resource data is obtained and stored in the second control environment; the second control environment is a backup control environment for the first control environment.

[0176] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0177] If the fault detection result is determined to be a Redis cluster fault, retrieve the first business data of the first business process in the first control environment from the status data table;

[0178] The first business process is restored in the second control environment based on the first business data.

[0179] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0180] Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node;

[0181] For each Redis node, based on the information set of each node, determine the node type that the Redis node is recognized by each Redis node; the node type includes at least one of the following: in a Redis cluster, not in a Redis cluster, master node, and standby node;

[0182] The fault detection result of the Redis cluster is determined based on the number of first nodes identified as being of the same node type.

[0183] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0184] If the number of first nodes is greater than or equal to the first preset threshold, then the same node type will be used as the target type of Redis nodes, and the fault detection result will be determined as the Redis cluster being normal; the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0185] If the number of first nodes of the same node type is equal to half the total number, then the fault detection result is determined to be a Redis cluster fault.

[0186] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0187] If the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type master Redis node deployed on a single device;

[0188] If the number of second nodes is greater than the second preset threshold, then migrate the Redis nodes of the target type as master nodes on a single device.

[0189] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0190] Identify the nodes to be migrated from the Redis nodes whose target type is the master node;

[0191] Determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located;

[0192] If the number of third nodes is less than the second preset threshold, the node to be migrated will be migrated to the target device.

[0193] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0194] If the fault detection result indicates a Redis cluster fault, retrieve the storage location and storage method of the first resource data of the first control environment from the resource data table;

[0195] Based on the storage location and storage method, the first resource data is obtained and stored in the second control environment; the second control environment is a backup control environment for the first control environment.

[0196] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0197] If the fault detection result is determined to be a Redis cluster fault, retrieve the first business data of the first business process in the first control environment from the status data table;

[0198] The first business process is restored in the second control environment based on the first business data.

[0199] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0200] Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set includes the node information of each Redis node;

[0201] For each Redis node, based on the information set of each node, determine the node type that the Redis node is recognized by each Redis node; the node type includes at least one of the following: in a Redis cluster, not in a Redis cluster, master node, and standby node;

[0202] The fault detection result of the Redis cluster is determined based on the number of first nodes identified as being of the same node type.

[0203] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0204] If the number of first nodes is greater than or equal to the first preset threshold, then the same node type will be used as the target type of Redis nodes, and the fault detection result will be determined as the Redis cluster being normal; the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster.

[0205] If the number of first nodes of the same node type is equal to half the total number, then the fault detection result is determined to be a Redis cluster fault.

[0206] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0207] If the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type master Redis node deployed on a single device;

[0208] If the number of second nodes is greater than the second preset threshold, then migrate the Redis nodes of the target type as master nodes on a single device.

[0209] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0210] Identify the nodes to be migrated from the Redis nodes whose target type is the master node;

[0211] Determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located;

[0212] If the number of third nodes is less than the second preset threshold, the node to be migrated will be migrated to the target device.

[0213] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0214] If the fault detection result indicates a Redis cluster fault, retrieve the storage location and storage method of the first resource data of the first control environment from the resource data table;

[0215] Based on the storage location and storage method, the first resource data is obtained and stored in the second control environment; the second control environment is a backup control environment for the first control environment.

[0216] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0217] If the fault detection result is determined to be a Redis cluster fault, retrieve the first business data of the first business process in the first control environment from the status data table;

[0218] The first business process is restored in the second control environment based on the first business data.

[0219] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0220] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0221] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / port:role:state:slaveip:slaveport>

Claims

1. A fault detection method, characterized in that, The method includes: Obtain the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set of the Redis node includes the node information of each Redis node from the perspective of the Redis node. For each Redis node, based on the node information set, the node type of the Redis node is determined by each Redis node; the node type includes at least one of being in the Redis cluster, not being in the Redis cluster, master node, and backup node; wherein, the node type of the Redis node recognized by each Redis node is the node type of the Redis node from the perspective of each Redis node; For each Redis node, if the number of nodes identified as the same node type by all Redis nodes is greater than or equal to a first preset threshold, then the same node type is taken as the target type of the Redis node, and the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster; if the number of nodes of the same node type is equal to half of the total number, then the fault detection result is determined to be a Redis cluster fault; the first preset threshold is half of the total number plus 1. If the target type of each Redis node in the Redis cluster is obtained, the fault detection result is determined to be that the Redis cluster is normal.

2. The method according to claim 1, characterized in that, The method further includes: If the fault detection result indicates that the Redis cluster is normal, obtain the number of second nodes of the target type Redis node deployed on a single device as the master node; If the number of the second nodes is greater than the second preset threshold, then the Redis node of the target type as the master node on the single device will be migrated.

3. The method according to claim 2, characterized in that, The migration of the target type Redis node as the master node on the single device includes: The nodes to be migrated are determined from the Redis nodes of the target type that are the master nodes; Determine the number of third nodes of the primary node on the target device where the backup node corresponding to the node to be migrated is located; If the number of the third nodes is less than the second preset threshold, then the node to be migrated will be migrated to the target device.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: If the fault detection result is determined to be a fault in the Redis cluster, the storage location and storage method of the first resource data of the first control environment are obtained from the resource data table. Based on the storage location and the storage method, the first resource data is obtained and stored in the second control environment; the second control environment is a backup control environment for the first control environment.

5. The method according to claim 4, characterized in that, The method further includes: If the fault detection result is determined to be a fault in the Redis cluster, the first business data of the first business process in the first control environment is obtained from the status data table. The first business process is restored in the second control environment based on the first business data.

6. A fault detection device, characterized in that, The device includes: The first acquisition module is used to acquire the node information set of each Redis node in the remote dictionary service Redis cluster in the first control environment; the node information set of the Redis node includes the node information of each Redis node from the perspective of the Redis node. The first determining module is used to determine, for each Redis node, the node type that the Redis node is identified by each Redis node based on the node information set of each Redis node; the node type includes at least one of being in the Redis cluster, not being in the Redis cluster, master node, and backup node; wherein, the node type that the Redis node is identified by each Redis node is the node type of the Redis node from the perspective of each Redis node; The second determining module is used to determine the following for each Redis node: if the number of nodes identified as the same node type by each Redis node is greater than or equal to a first preset threshold, then the same node type is taken as the target type of the Redis node, and the first preset threshold is determined based on the total number of Redis nodes in the Redis cluster; if the number of nodes of the same node type is equal to half of the total number, then the fault detection result is determined to be a Redis cluster fault; the first preset threshold is half of the total number plus 1; and if the target type of each Redis node in the Redis cluster is obtained, then the fault detection result is determined to be a normal Redis cluster.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images