Troubleshooting methods and devices, storage media and electronic equipment

By acquiring fault description information of computer cluster nodes, calculating fault types and quantities, determining priorities, and prioritizing the handling of high-priority faults, the problem of low efficiency in cluster node fault location is solved, and efficient fault handling is achieved.

CN117857322BActive Publication Date: 2026-11-17INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410084439.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-19
Publication Date
2026-11-17
Estimated Expiration
2044-01-19

AI Technical Summary

Technical Problem

Existing technologies for locating faults in computer cluster nodes are inefficient and require a significant amount of time for manual analysis.

Method used

By obtaining fault description information from the server, the fault type and quantity are determined, and the priority is calculated based on the fault type and quantity. High-priority faults are processed first, thus achieving fault classification and sorting.

Benefits of technology

It improves the efficiency of fault handling, ensures timely handling of more serious or urgent faults, and reduces the waste of computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117857322B_ABST
    Figure CN117857322B_ABST
Patent Text Reader

Abstract

This application provides a fault handling method, apparatus, storage medium, and electronic device. The method includes: acquiring N sets of fault description information corresponding to N servers; determining N fault types and N sets of quantities corresponding to the N sets of faults based on the N sets of fault description information; and determining N priorities corresponding to the N sets of faults based on the N sets of fault types and N sets of quantities. This application solves the problem of low fault handling efficiency, thereby improving the efficiency of fault handling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a fault handling method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] A computer cluster, or simply cluster, is a computer system that connects multiple computers (also known as computing resources, such as software and / or hardware resources) to collaboratively complete computational tasks. These computing resources reside in the same management domain, have a unified management strategy, and provide services to users as a whole. Individual computers in a cluster system are typically called nodes or compute nodes.

[0003] When a node in a cluster fails, the common methods for locating the fault are alarm information and log queries. For example, logs can be manually queried to analyze the fault. However, due to the large number of nodes in a cluster, manually locating the faulty node and analyzing the cause of the fault takes a lot of time. Understandably, the alarm information and log query methods are extremely inefficient, resulting in a significant waste of time in determining the fault that occurred in the node. Summary of the Invention

[0004] This application provides a fault handling method, apparatus, storage medium, and electronic device to at least solve the problem of low fault handling efficiency in related technologies.

[0005] According to one embodiment of this application, a fault handling method is provided, comprising: obtaining N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults occurring in the N servers, and N is a positive integer greater than or equal to 1; determining N sets of fault types and N sets of quantities corresponding to the N sets of faults based on the N sets of fault description information, wherein the i-th set of fault types in the N sets of fault types includes different fault types in the i-th set of faults in the N sets of faults, and the i-th set of quantities in the N sets of faults includes the number of faults of each fault type in the i-th set of faults, and i is a positive integer less than or equal to N; determining N priorities corresponding to the N sets of faults based on the N sets of fault types and the N sets of quantities, wherein the i-th priority in the N priorities is used to represent the processing order of the i-th set of faults; and processing M sets of faults in the N sets of faults, wherein the priorities of the M sets of faults satisfy a preset target matching condition with a preset priority threshold, and M is a positive integer less than or equal to N.

[0006] In an exemplary embodiment, determining the N fault types and the number of N faults corresponding to the N faults based on the N fault description information includes: determining the i-th fault type and the number of i-th faults corresponding to the i-th fault by performing the following steps: based on the i-th fault description information in the N fault description information, wherein the i-th fault description information is used to describe the fault occurring in the i-th server among the N servers, and the i-th fault includes the fault occurring in the i-th server; determining the i-th fault type based on the i-th fault description information; and determining the number of i-th faults based on the i-th fault description information.

[0007] In an exemplary embodiment, determining the i-th group of fault types based on the i-th group of fault description information includes: determining an i-th group type identifier from the i-th group of fault description information; determining the i-th group of fault types corresponding to the i-th group type identifier from category identifiers and fault types with corresponding relationships; determining the i-th group quantity based on the i-th group of fault description information includes: determining the r-th quantity in the i-th group quantity by performing the following steps, wherein the r-th quantity is the number of faults of the r-th fault type in the i-th group of faults, and r is a positive integer greater than or equal to 1; determining the r-th quantity as the number of times the r-th type identifier appears in the i-th group of fault description information, wherein the r-th type identifier is the identifier of the fault of the r-th fault type.

[0008] In an exemplary embodiment, determining the N priorities corresponding to the N groups of faults based on the N groups of fault types and the N groups of quantities includes: determining the i-th priority corresponding to the i-th group of faults by performing the following steps: determining the i-th group of fault priorities corresponding to the i-th group of faults based on the i-th group of fault types and the i-th group of quantities; and determining the i-th priority as the sum of the fault priorities in the i-th group of fault priorities.

[0009] In an exemplary embodiment, determining the fault priority corresponding to the i-th group of faults based on the i-th group of fault types and the i-th group of quantities includes: determining the s-th fault priority in the i-th group of fault priorities by performing the following steps: determining the s-th fault priority in the i-th group of fault priorities based on the s-th fault type in the i-th group of fault types and the s-th quantity in the i-th group of quantities, where s is a positive integer; determining the s-th weight corresponding to the s-th fault type; and determining the s-th fault priority as the product value obtained by performing a product operation on the s-th quantity and the s-th weight.

[0010] In one exemplary embodiment, before processing M groups of faults out of the N groups of faults, the method further includes: determining priorities greater than or equal to the priority threshold from the N priorities; and, if the determined priorities greater than or equal to the priority threshold include M priorities, determining the M groups of faults corresponding to the M priorities from the N groups of faults, wherein the M priorities include the priority of each group of faults in the M groups of faults.

[0011] In an exemplary embodiment, processing M groups of faults out of the N groups of faults includes: processing the w-th group of faults out of the M groups of faults by performing the following steps, where w is a positive integer less than or equal to M, and the w-th group of faults includes a fault occurring on one of the N servers: determining a target task being executed on the one server; pausing processing of the w-th group of faults if the w-th group of faults is unrelated to the execution of the target task, and resuming processing of the w-th group of faults if the target task is completed; determining a target server from the servers other than the one server among the N servers if the w-th group of faults is related to the execution of the target task; re-executing the target task on the target server, and resuming processing of the w-th group of faults if the re-executing of the target task on the target server begins.

[0012] In an exemplary embodiment, determining the target server from the servers other than the first server among the N servers includes: determining the task type of the target task; determining a group of candidate servers from the servers other than the first server among the N servers, wherein the group of candidate servers are servers that are allowed to execute tasks of the task type; and determining the servers in the group of candidate servers whose load is less than or equal to a preset load threshold as the target server.

[0013] In an exemplary embodiment, re-executing the target task on the target server includes: if the target task comprises a set of tasks, obtaining the task execution duration of each task in the set of tasks from the server, wherein the task execution duration of each task is used to indicate the duration for which each task has been executed on the server; and re-executing each task in the set of tasks sequentially on the target server according to the task execution duration of each task in the set of tasks, wherein tasks with longer execution durations are re-executed earlier.

[0014] According to another embodiment of this application, a fault handling device is provided, comprising: an acquisition module, configured to acquire N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults occurring in the N servers, and N is a positive integer greater than or equal to 1; a first determination module, configured to determine N sets of fault types and N sets of quantities corresponding to the N sets of faults based on the N sets of fault description information, wherein the i-th set of fault types in the N sets of fault types includes different fault types in the i-th set of faults in the N sets of faults, and the i-th set of quantities in the N sets of faults includes the number of faults of each fault type in the i-th set of faults, and i is a positive integer less than or equal to N; a second determination module, configured to determine N priorities corresponding to the N sets of faults based on the N sets of fault types and the N sets of quantities, wherein the i-th priority in the N priorities is used to represent the processing order of the i-th set of faults; and a processing module, configured to process M sets of faults in the N sets of faults, wherein the priorities of the M sets of faults satisfy a preset target matching condition with a preset priority threshold, and M is a positive integer less than or equal to N.

[0015] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0016] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0017] This application determines N priorities corresponding to N groups of faults based on N groups of fault types and N groups of fault quantities. The priorities of M groups of faults meet the preset target matching conditions with the preset priority threshold. It can be understood that the M groups of faults may be more serious or urgent faults. In such cases, the M groups of faults are processed first. In this way, faults are classified with nodes (or servers) as the center, and then the faults appearing in each server are prioritized. The more serious or urgent faults are processed first, thus achieving effective fault handling and improving the efficiency of fault handling. Attached Figure Description

[0018] Figure 1 This is a hardware structure block diagram of a server device for a fault handling method according to an embodiment of this application;

[0019] Figure 2This is a schematic diagram illustrating an application scenario of an optional fault handling method according to an embodiment of this application;

[0020] Figure 3 This is a flowchart of a fault handling method according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating an optional method for determining the type and number of faults occurring in a server, according to an embodiment of this application.

[0022] Figure 5 This is a schematic diagram illustrating an optional method of executing a task on a target server according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of an optional fault handling process according to an embodiment of this application;

[0024] Figure 7 This is a structural block diagram of a fault handling device according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0027] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device for a fault handling method according to an embodiment of this application. For example... Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the fault handling method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0030] First, the features involved in the embodiments of this application are explained as follows:

[0031] node cluster: a cluster of nodes.

[0032] High-performance computing (HPC) researches parallel algorithms and develops related software to achieve supercomputing capabilities. It often employs cluster systems, where individual computers are typically called nodes. The cluster is designed to work in a way that leverages the computing power of its nodes. For example, when a cluster system uses load balancing, multiple computers can perform the same task, achieving higher efficiency and computing power. A high-performance computing cluster typically consists of a master node (home node) and several computation nodes.

[0033] To better understand the application scenarios of the fault handling methods in the embodiments of this application, the application scenarios of the fault handling methods in the embodiments of this application are explained and described below in conjunction with optional embodiments, which may be applicable to, but are not limited to, the embodiments of this application.

[0034] Figure 2This is a schematic diagram illustrating an application scenario of an optional fault handling method according to an embodiment of this application, such as... Figure 2 As shown, a computer cluster, or simply cluster, is a computer system that connects multiple computers (also known as computing resources, such as software and / or hardware resources) to collaboratively complete computing tasks. These computing resources reside in the same management domain, have a unified management strategy, and provide services to users as a whole. Individual computers in a cluster system are typically called nodes or computing nodes. Nodes can include, but are not limited to, devices capable of performing business processing, such as servers; servers 1 to 4 can also be referred to as nodes 1 to 4, respectively.

[0035] The cluster includes a master node (management node) and multiple slave nodes (service nodes). The master node manages the cluster, while the slave nodes implement user services. For example, node 1 is the master node, and nodes 2 to 4 are slave nodes. The master node and slave nodes communicate through a network. The network topology is not required, as long as network communication can be established. The master node can, but is not limited to, assign received tasks to slave nodes for execution. For example, if node 1 (or server 1) receives tasks a, b, c, and d to be executed, it can assign tasks a and b to node 2 (or server 2) for execution, task c to node 3 (or server 3) for execution, and task d to node 4 (or server 4) for execution.

[0036] During the execution of tasks on nodes 2, 3, and 4, faults occurring on the server can be handled in, but are not limited to, the following ways:

[0037] Step S202: Obtain fault description information 1 corresponding to server 1, obtain fault description information 2 corresponding to server 2, and obtain fault description information 3 corresponding to server 3. Fault description information 1 is used to describe the first set of faults that occur in server 1, fault description information 2 is used to describe the second set of faults that occur in server 2, and fault description information 3 is used to describe the third set of faults that occur in server 3.

[0038] Step S204: Based on fault description information 1, fault description information 2, and fault description information 3, determine the first group of fault types and the first group of quantities corresponding to the first group of faults; determine the second group of fault types and the second group of quantities corresponding to the second group of faults; and determine the third group of fault types and the third group of quantities corresponding to the third group of faults. The first group of fault types includes different fault types within the first group of faults, and the first group of quantities includes the number of faults of each fault type within the first group of faults. The second group of fault types includes different fault types within the second group of faults, and the second group of quantities includes the number of faults of each fault type within the second group of faults. The third group of fault types includes different fault types within the third group of faults, and the third group of quantities includes the number of faults of each fault type within the third group of faults.

[0039] Step S206: Determine the priority of the first group of faults, the second group of faults, and the third group of faults based on the first group of fault types, the second group of fault types, the third group of fault types, the first group of faults, the second group of faults, and the third group of faults.

[0040] Step S208: Process the first group of faults among the first group of faults, the second group of faults and the third group of faults, wherein the priority of the first group of faults meets the preset target matching condition with the preset priority threshold.

[0041] This embodiment provides a fault handling method. Figure 3 This is a flowchart of a fault handling method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps:

[0042] Step S302: Obtain N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults that occur in the N servers, and N is a positive integer greater than or equal to 1;

[0043] Step S304: Based on the N sets of fault description information, determine the N sets of fault types and N sets of quantities corresponding to the N sets of faults, wherein the i-th set of fault types in the N sets of fault types includes the fault types of different faults in the i-th set of faults in the N sets of faults, and the i-th set of quantities in the N sets of quantities includes the number of faults of each fault type in the i-th set of faults, where i is a positive integer less than or equal to N.

[0044] Step S306: Based on the N groups of fault types and the N groups of quantities, determine the N priorities corresponding to the N groups of faults, wherein the i-th priority among the N priorities is used to represent the processing order of the i-th group of faults;

[0045] Step S308: Process the M groups of faults in the N groups of faults, wherein the priority of the M groups of faults satisfies the preset target matching condition with the preset priority threshold, and M is a positive integer less than or equal to N.

[0046] Through the above steps, based on N groups of fault types and N groups of quantities, N priorities corresponding to the N groups of faults are determined. The priorities of M groups of faults meet the preset target matching conditions with the preset priority threshold. It can be understood that the M groups of faults may be more serious or urgent. In such cases, the M groups of faults are processed first. In this way, faults are classified with nodes (or servers) as the center, and then the faults appearing in each server are prioritized. The more serious or urgent faults are processed first, thus achieving effective fault handling and improving the efficiency of fault handling.

[0047] In the technical solution provided in step S302 above, a set of fault description information may include, but is not limited to, descriptions of faults occurring in one of the N servers. The server may experience the same fault multiple times, and / or some faults may occur only once. It is understood that a set of fault description information may be used to describe the same faults occurring multiple times in the server, and / or the faults occurring only once in the server, etc.

[0048] Optionally, in this embodiment, the faults occurring in the server may include, but are not limited to, hardware faults in the server, and / or software faults in the server, and / or firmware faults in the server, etc. As an optional example, hardware faults in the server may include, but are not limited to, faults in the server's processor, memory, or chipset, etc.; as an optional example, software faults in the server may include, but are not limited to, faults in the server's web services, HTTP services, or applications, etc.; as an optional example, firmware faults in the server may include, but are not limited to, faults in the server's BMC (Baseboard Management Controller), BIOS (Basic Input Output System), etc.

[0049] Optionally, in this embodiment, a set of fault description information may also include, but is not limited to, faults that occur in the communication network between any two of the N servers, such as network link faults (e.g., network link response is disconnected, network link response is slow), etc.

[0050] In the technical solution provided in step S304 above, the fault types corresponding to different faults in the i-th group of N faults may be the same or duplicated. It can be understood that the fault types included in the i-th group of fault types are only one of the duplicated fault types. That is to say, the fault types included in the i-th group of fault types have been deduplicated.

[0051] For example, if the i-th group of faults includes faults 1 to 5, where faults 1 and 2 belong to fault type 1, and faults 3 to 5 belong to fault type 2, then the i-th group of fault types includes fault type 1 and fault type 2. The number of faults of fault type 1 is 2, and the number of faults of fault type 2 is 3.

[0052] In an exemplary embodiment, the following steps can be performed, but are not limited to, to determine the type and quantity of the i-th group of faults corresponding to the i-th group of faults based on the i-th group of fault description information in the N groups of fault description information, wherein the i-th group of fault description information is used to describe the fault occurring in the i-th server among the N servers, and the i-th group of faults includes the fault occurring in the i-th server: determining the type of the i-th group of faults based on the i-th group of fault description information; and determining the quantity of the i-th group based on the i-th group of fault description information.

[0053] Optionally, in this embodiment, the fault type of the fault occurring in the i-th server can be determined based on the i-th group of fault description information in the following ways, but not limited to: determining the number of the i-th group based on the i-th group of fault description information: determining the i-th group of target fault description information from the i-th group of fault description information, wherein the i-th group of target fault description information is used to describe the different faults occurring in the i-th group of servers; determining the fault type of the fault occurring in the i-th server based on the i-th group of target fault description information; and determining the number of the i-th group based on the i-th group of target fault description information.

[0054] It is understandable that the fault description information in the i-th group, excluding the target fault description information in the i-th group, may be used to describe the same fault occurring on the i-th server.

[0055] Figure 4 This is a schematic diagram illustrating an optional method for determining the type and number of faults occurring in a server, according to an embodiment of this application. Figure 4As shown, the tasks being executed on server 2 may include, but are not limited to, task a, task b, and task e. Based on a set of fault description information of server 2, the set of fault types determined for the faults occurring in server 2 may include, but are not limited to, fault type 1, fault type 2, and fault type 3. Among them, the number of faults of fault type 1 occurring in server 2 is 1, the number of faults of fault type 2 occurring in server 2 is 3, and the number of faults of fault type 3 occurring in server 2 is 4.

[0056] In this way, duplicate descriptions of the same server faults in the fault description information are removed, reducing the computer resources required for the types and number of faults and improving the utilization rate of computer resources.

[0057] In an exemplary embodiment, the i-th group of fault types can be determined based on the i-th group of fault description information in the following manner, but not limited to: determining the i-th group of type identifiers from the i-th group of fault description information; determining the i-th group of fault types corresponding to the i-th group of type identifiers from category identifiers and fault types with corresponding relationships; the step of determining the i-th group of quantities based on the i-th group of fault description information includes: determining the r-th quantity in the i-th group of quantities by performing the following steps, wherein the r-th quantity is the number of faults of the r-th fault type in the i-th group of faults, and r is a positive integer greater than or equal to 1; determining the r-th quantity as the number of times the r-th type identifier appears in the i-th group of fault description information, wherein the r-th type identifier is the identifier of the fault of the r-th fault type.

[0058] Optionally, in this embodiment, the i-th group of fault description information may include, but is not limited to, the i-th group of type identifiers or description information of the i-th group of type identifiers. For example, the i-th group of fault description information may include a first type identifier, such as an identifier of IO (Input / Output) fault type, or the i-th group of fault description information may include description information of the first type identifier, such as description information of IO Error.

[0059] As an optional example, the identifiers of each type in the i-th group of type identifiers may include, but are not limited to, at least one of letters, numbers, and strings. It is understood that the identifiers of each type in the i-th group of type identifiers may not be related to the fault of the corresponding fault type. For example, the identifier of the IO type may include, but is not limited to, 1s3b. Alternatively, the identifiers of each type in the i-th group of type identifiers may be related to the fault of the corresponding fault type. For example, the identifier of the IO type may include, but is not limited to, IO.

[0060] In this way, different fault types can be distinguished by different identifiers, avoiding the use of the same type identifier for different fault types, and improving the accuracy of determining the i-th group of fault types and the number of i-th groups.

[0061] Optionally, in this embodiment, the number of times different type identifiers appear in the i-th group of fault description information may be once or multiple times. It is understood that the same fault may occur multiple times in the server, or some faults may only occur once.

[0062] In the technical solution provided in step S306 above, in order to improve the timely priority handling of more serious faults, the priority of each group of faults in the N groups of faults can be determined, but is not limited to. It can be understood that each group of faults in the N groups of faults can have a corresponding priority, but is not limited to.

[0063] Optionally, in this embodiment, the processing order of each fault in a higher priority group of faults is earlier. For example, the processing order of each fault in a higher priority group of faults is earlier than the processing order of each fault in a lower priority group of faults.

[0064] In one exemplary embodiment, the i-th priority corresponding to the i-th group of faults may be determined by performing, but is not limited to, the following steps: determining the i-th group of fault priorities corresponding to the i-th group of faults based on the i-th group of fault types and the i-th group of faults; and determining the i-th priority as the sum of the individual fault priorities in the i-th group of fault priorities.

[0065] Optionally, in this embodiment, the fault priority of the i-th group of faults can be determined according to the fault type and the number of the i-th group of faults, but is not limited to. It is understood that one fault in the i-th group of faults has a corresponding fault priority, and the sum of the fault priorities in the i-th group of fault priorities can be determined as the i-th priority, but is not limited to.

[0066] Optionally, in this embodiment, higher fault priority indicates greater severity or urgency, meaning these faults may cause server crashes. By determining the fault priorities of different faults, the priority of all faults occurring on a single server is considered, and the processing order of faults occurring on a single server is determined, thus improving the rationality of determining the processing order of faults occurring on various servers.

[0067] In one exemplary embodiment, the following steps can be performed, but are not limited to, to determine the s-th fault priority in the i-th group of fault priorities based on the s-th fault type in the i-th group of fault types and the s-th quantity in the i-th group of quantities, where s is a positive integer: determining the s-th weight corresponding to the s-th fault type; and determining the s-th fault priority as the product value obtained by performing a product operation on the s-th quantity and the s-th weight.

[0068] Optionally, in this embodiment, the higher the weight corresponding to a fault type, the more urgent or severe the fault. For example, the weight corresponding to a CPU unresponsive fault type in a server is greater than the weight corresponding to a fan malfunction fault type. This approach distinguishes the severity of faults occurring in the server, improving the rationality of fault handling.

[0069] Optionally, in this embodiment, the weights corresponding to different fault types are relatively high and low. It is understood that as a fault occurs in real time on a server, the fault types included in the i-th group of fault types may change, the number of faults of each fault type in the i-th group may change, and the fault priority of the i-th group will also change, as will the i-th priority.

[0070] To better understand the process of determining priority in the embodiments of this application, the process of determining priority in the embodiments of this application can be explained and described in conjunction with optional embodiments, and can be applied to the embodiments of this application, but is not limited to.

[0071] The master node can collect status information from slave nodes, including but not limited to network information and network status information. Network information generally includes node identifiers and cluster topology; network status information includes network links, etc. The faults that occur in the node can include but are not limited to node faults and link faults. In this embodiment, the focus is on viewing fault information of the node itself, and node faults and network link faults are comprehensively considered through node failures or response delays, feedback delays, etc.

[0072] The master node collects fault description information from each node in the entire cluster and matches the fault information with the nodes based on the network topology and link fault information. At the same time, the fault information is classified, such as feedback information faults, no response faults, etc. After classification, a fault information table for each slave node is formed, as shown in Table 1.

[0073] Table 1

[0074]

[0075] As shown in Table 1, node 1 experienced 3 read / write no-feedback failures, 6 information execution no-response failures, and 1 garbled feedback message failure. Node 2 experienced 6 read / write no-feedback failures, 7 information execution no-response failures, and 2 garbled feedback message failures.

[0076] The weight of a fault type with no feedback during read / write is 2, the weight of a fault type with no response during information execution is 3, and the weight of a fault type with garbled feedback information is 1. In this case, the priority of a set of faults occurring in node 1 includes 3*2=6, 6*3=18, and 1*1=1, which means the priority of a set of faults in node 1 is 6+18+1=25. Similarly, the priority of a set of faults occurring in node 1 includes 2*6=12, 7*3=18, and 2*1=2, which means the priority of a set of faults occurring in node 2 is 12+18+2=32. Therefore, the priority of a set of faults occurring in node 2 is higher than that of a set of faults occurring in node 1. Thus, it is possible, but not limited to, prioritizing the processing of each fault in the set of faults occurring in node 2.

[0077] In the technical solution provided in step S308 above, when the computer resources used for fault processing are limited, in order to improve the efficiency of fault processing and avoid processing all faults indiscriminately, which would lead to slow fault processing, it is possible, but not limited to, selectively processing M groups of faults out of N groups of faults.

[0078] Optionally, in this embodiment, when M equals 1, the highest priority fault among the N groups of faults can be processed, but is not limited to. When M is a positive integer greater than or equal to 2, each group of faults within the M groups can be processed in parallel, but is not limited to. In this way, limited computer resources are utilized to promptly handle more urgent or serious faults, improving the timeliness of handling such faults.

[0079] As an alternative example, M groups of faults out of N groups of faults can also be processed in the following manner, but not limited to: the N groups of faults include a first group of faults and a second group of faults that are coupled, the M groups of faults include the second group of faults, the M groups of faults do not include the first group of faults, and the coupling relationship indicates that the first group of faults is processed after the second group of faults is processed; and the second group of faults is processed after the first group of faults has been processed.

[0080] Optionally, in this embodiment, the first group of faults and the second group of faults that have a coupling relationship can be used, but are not limited to, to indicate that processing the second group of faults is only effective after processing the first group of faults. It is understood that the first group of faults is a fault that must be processed before processing the second group of faults.

[0081] For example, the first group of faults includes a power supply failure in the server, and the second group of faults includes a chip failure in the server. The server power supply can be, but is not limited to, supplying power to the chips in the server. In this case, there is a coupling relationship between the first and second groups of faults. It can be understood that the first group of faults must be addressed before the second group of faults can be addressed. Only after the first group of faults is addressed can the second group of faults be addressed. In other words, only under these circumstances can the chips in the server operate normally. If only the second group of faults is addressed, the chips in the server will still not operate normally because the power supply failure that powers the chips in the server is not resolved.

[0082] As an optional example, the first group of faults may be processed in the following manner, but not limited to: after processing the first group of faults, the second group of faults is processed by: adjusting the priority of the first group of faults so that the adjusted priority of the first group of faults is higher than the priority of the second group of faults, wherein, before adjusting the priority of the first group of faults, the priority of the first group of faults was lower than the priority of the second group of faults; processing each fault in the first group of faults in descending order of priority, and processing each fault in the second group of faults after processing the first group of faults.

[0083] In an exemplary embodiment, before processing M groups of faults out of the N groups of faults, the method further includes: determining priorities greater than or equal to the priority threshold from the N priorities; and, if the determined priorities greater than or equal to the priority threshold include M priorities, determining the M groups of faults corresponding to the M priorities from the N groups of faults, wherein the M priorities include the priority of each group of faults in the M groups of faults.

[0084] Optionally, in this embodiment, if no priority greater than or equal to the priority threshold is determined, it is understood that each group of faults in the N groups of faults may not be very urgent or very serious. In such cases, it is possible, but not limited to, not to process the N groups of faults, or to determine the highest priority from the N priorities and process the group of faults corresponding to the highest priority in the N groups of faults.

[0085] Optionally, in this embodiment, M groups of faults can be placed into a sequence of fault nodes to be processed, but not limited to those with higher priority. The higher the priority of a fault group, the earlier it appears in the sequence, and the earlier it is processed. This can be done, but is not limited to, processing the head node of the sequence (equivalent to the highest priority group of faults in the M groups). This approach comprehensively considers the impact of fault type and the number of faults of each type on cluster performance, maximizing the balance between cluster performance and fault handling, while simultaneously improving the effectiveness and efficiency of fault handling.

[0086] In an exemplary embodiment, the w-th group of faults in the M groups of faults can be processed by performing, but is not limited to, the following steps: w is a positive integer less than or equal to M, and the w-th group of faults includes faults occurring on one of the N servers: determining a target task being executed on the target task; pausing processing of the w-th group of faults if the w-th group of faults is unrelated to the execution of the target task, and resuming processing of the w-th group of faults once the target task is completed; determining a target server from the servers other than the target task among the N servers if the w-th group of faults is related to the execution of the target task; re-executing the target task on the target server, and resuming processing of the w-th group of faults once the re-executing of the target task on the target server begins.

[0087] Optionally, in this embodiment, if the w-th group of faults is unrelated to the execution of the target task, it can be understood that the w-th group of faults will not affect the execution of the target task. For example, even if the w-th group of faults occurs, the target task can still be executed normally and obtain accurate execution results. Alternatively, the w-th group of faults in the server will not affect the normal operation of the server hardware, software, and firmware required to execute the target task.

[0088] Optionally, in this embodiment, when the w-th group of faults is related to the execution of the target task, it is understood that the w-th group of faults will affect the execution of the target task. For example, when the w-th group of faults occurs, the target task cannot be executed normally and cannot obtain accurate execution results. It is understood that the target task may experience execution interruption, execution failure, or other execution anomalies, resulting in abnormal (e.g., inaccurate or failed) execution results. Alternatively, the w-th group of faults in the server may affect the normal operation of the server hardware, software, or firmware required to execute the target task, causing the target task to execute abnormally.

[0089] Figure 5This is a schematic diagram illustrating an optional method of executing a task on a target server according to an embodiment of this application, such as... Figure 5 As shown, the M group of faults may include, but is not limited to, a group of faults occurring on server 2 (equivalent to slave node 2). The tasks being executed on server 2 may include, but are not limited to, tasks a, b, and e. The group of faults occurring on server 2 may include, but is not limited to, faults 1 to 8. Faults 1 to 8 will affect the execution of at least one of tasks a, b, and e. In such a case, server a (equivalent to master node) may, but is not limited to, transfer tasks a, b, and e to server 4 (equivalent to target server) for execution. The tasks that were originally being executed on server 4 (equivalent to slave node 4) include task e. When tasks a, b, and e begin to be executed on server 4, server a (equivalent to master node) will handle faults 1 to 8 occurring on server 2.

[0090] This approach allows for monitoring cluster task processing during fault information handling, enabling a balance between performance and fault management. Specifically, when classifying fault information, the impact of a node's failure on cluster performance can be calculated based on the number and type of failures. Simultaneously, the impact of disabling or partially disabling a node during fault handling can be assessed. This allows for the selection of node fault handling processes based on the cluster task status. Furthermore, when certain nodes significantly impact current task processing, but their failures do not affect current task execution, their priority can be lowered in the task sequence. Timely task transfer before node fault handling minimizes the impact, thus improving cluster performance.

[0091] In an exemplary embodiment, a target server may be determined from the servers other than the one server among the N servers in the following manner, but not limited to: determining the task type of the target task; determining a set of candidate servers from the servers other than the one server among the N servers, wherein the set of candidate servers are servers that are allowed to execute tasks of the task type; determining the servers in the set of candidate servers whose load is less than or equal to a preset load threshold as the target server; and re-executing the target task on the target server in the following manner, but not limited to: if the target task includes a set of tasks, obtaining the task execution time of each task in the set of tasks from the one server, wherein the task execution time of each task is used to indicate the duration for which each task has been executed on the one server; and re-executing each task in the set of tasks in the target server in sequence according to the task execution time of each task in the set of tasks, wherein tasks with longer execution times are re-executed earlier.

[0092] Optionally, in this embodiment, the load of each server in a group of candidate servers may be determined based on, but is not limited to, the number of tasks currently being executed by each server, the number of tasks waiting to be executed by each server, and the time required for each server to complete its tasks.

[0093] Optionally, in this embodiment, tasks with longer execution times are re-executed earlier on the target server. This means that tasks with longer execution times are executed earlier on a single server. For example, tasks currently executing on the server may include, but are not limited to, task 1, task 2, and task 3. Task 1 has been executed for 2 hours, task 2 for 3 hours, and task 3 for 0.5 hours. This means that task 2 is the earliest executed task, task 1 is executed after task 2, and task 3 is the latest executed task. In such a case, task 2 may be re-executed first, then task 1, and finally task 3.

[0094] It is understood that, in this embodiment of the application, the fault severity of nodes is sorted according to the matching of fault description information of each slave node. Since different fault types and quantities have different impacts on cluster performance, they can be sorted in a predetermined manner, with the head being the target node requiring fault handling. The matching of slave node fault information is continuously refreshed, and the target node is processed, thus forming the processing of faults in each slave node. During the processing, a sequence of faulty nodes to be processed can be formed and processed in a task sequence manner.

[0095] Based on the sequence of faulty nodes, the node to be processed (i.e., the head node of the sequence) is identified. Node task information, such as task name, start time, and status, is obtained. This task is then transferred to a relevant idle node (equivalent to the target server) for processing. During abnormal task polling, tasks with start times (equivalent to task execution duration) are sorted in descending order, prioritizing tasks with longer start times for transfer. This task transfer further reduces the impact of fault processing on cluster performance and efficiency. To facilitate node identification, each node can be assigned an identifier upon task startup. This identifier allows for updating the fault severity ranking of nodes when initiating fault processing for a node to be processed, improving processing accuracy.

[0096] During task polling, if the task is solely handled by the faulty node to be processed, then a fault-free node (healthy node, equivalent to the target server) among the adjacent nodes of the faulty node to be processed is selected as the target node for task migration.

[0097] To better understand the fault handling process in the embodiments of this application, the fault handling process in the embodiments of this application will be explained and described below in conjunction with optional embodiments, which may be applied to the embodiments of this application but are not limited to them.

[0098] Figure 6 This is a schematic diagram of an optional fault handling process according to an embodiment of this application, such as... Figure 6 As shown, the steps may include, but are not limited to:

[0099] Step S1: The master node collects the status information of the slave nodes. For example, the status information includes network information and network status information. The network information generally includes node identifiers and cluster topology; the network status information includes network links, etc.

[0100] Step S2: The master node collects fault information from the entire cluster and matches the fault information with the nodes based on the network information and link fault information; at the same time, the fault information is classified and a fault information table for each slave node is formed after classification.

[0101] Step S3: Based on the matching of fault information of each slave node, sort and process the fault severity of the nodes. For example, due to the different types and quantities of faults, the impact on cluster performance varies. The nodes can be sorted in a predetermined manner to form a sequence of faulty nodes to be processed. The head node is the target node that needs to be processed. The matching of fault information of slave nodes is continuously refreshed, and the target node is processed to handle the fault, thereby forming the processing of faults of each slave node.

[0102] Step S4: Based on the sequence of faulty nodes, determine the nodes to be processed, obtain the node task information, poll the tasks of the nodes to be processed, determine the relevant nodes of each task in the cluster, transfer the task to the relevant node, and then process the fault.

[0103] The method in this embodiment focuses on the fault information of the nodes themselves. Based on the matching of fault information from each slave node, and considering the varying impacts on cluster performance due to differences in fault types and quantities, a sequence of faulty nodes to be processed is formed. Fault information is categorized, resulting in a fault information table for each slave node. A task identifier is added, and before fault processing, the task is transferred to the relevant node. This approach eliminates the need for sequentially querying node logs for fault diagnosis and avoids the data collection and display methods used in related technologies. Instead, it employs a node-centric strategy for fault information classification and sorting, maximizing the balance between cluster performance and fault processing, while simultaneously improving the effectiveness and efficiency of fault processing.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0105] This embodiment also provides a fault handling device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0106] Figure 7 This is a structural block diagram of a fault handling device according to an embodiment of this application, such as... Figure 7 As shown, the device includes:

[0107] The acquisition module 702 is used to acquire N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults that occur in the N servers, and N is a positive integer greater than or equal to 1;

[0108] The first determining module 704 is used to determine the N groups of fault types and N groups of quantities corresponding to the N groups of faults based on the N groups of fault description information, wherein the i-th group of fault types in the N groups of fault types includes the fault types of different faults in the i-th group of faults in the N groups of faults, and the i-th group of quantities in the N groups of quantities includes the number of faults of each fault type in the i-th group of faults, where i is a positive integer less than or equal to N.

[0109] The second determining module 706 is used to determine N priorities corresponding to the N groups of faults based on the N groups of fault types and the N groups of numbers, wherein the i-th priority among the N priorities is used to represent the processing order of the i-th group of faults;

[0110] The processing module is used to process M groups of faults out of the N groups of faults, wherein the priority of the M groups of faults satisfies the preset target matching condition with the preset priority threshold, and M is a positive integer less than or equal to N.

[0111] In this embodiment, based on N groups of fault types and N groups of quantities, N priorities corresponding to the N groups of faults are determined. The priorities of M groups of faults meet the preset target matching conditions with the preset priority threshold. It can be understood that the M groups of faults may be more serious or urgent faults. In such cases, the M groups of faults are processed first. In this way, faults are classified with nodes (or servers) as the center, and then the faults appearing in each server are prioritized. The more serious or urgent faults are processed first, thus achieving effective fault handling and improving the efficiency of fault handling.

[0112] In one exemplary embodiment, the first determining module is configured to:

[0113] By performing the following steps, based on the i-th group of fault description information in the N groups of fault description information, the type and quantity of the i-th group of faults corresponding to the i-th group of faults are determined, wherein the i-th group of fault description information is used to describe the fault occurring in the i-th server among the N servers, and the i-th group of faults includes the fault occurring in the i-th server:

[0114] Based on the fault description information of the i-th group, determine the fault type of the i-th group;

[0115] The number of the i-th group is determined based on the fault description information of the i-th group.

[0116] In an exemplary embodiment, the first determining module is further configured to: determine the i-th group type identifier from the i-th group of fault description information; and determine the i-th group of fault types corresponding to the i-th group type identifier from category identifiers and fault types that have a corresponding relationship.

[0117] The first determining module is further configured to: determine the r-th quantity in the i-th group of quantities by performing the following steps, based on the i-th group of fault description information, wherein the r-th quantity is the number of faults of the r-th fault type in the i-th group of faults, and r is a positive integer greater than or equal to 1; and determine the r-th quantity as the number of times the r-th type identifier appears in the i-th group of fault description information, wherein the r-th type identifier is the identifier of the fault of the r-th fault type.

[0118] In one exemplary embodiment, the second determining module is configured to:

[0119] The i-th priority corresponding to the i-th group of faults is determined by performing the following steps:

[0120] Based on the fault type of the i-th group and the quantity of the i-th group, determine the fault priority of the i-th group corresponding to the fault in the i-th group;

[0121] The i-th priority is determined as the sum of the fault priorities in the i-th group of fault priorities.

[0122] In one exemplary embodiment, the second determining module is further configured to:

[0123] By performing the following steps, the s-th fault priority in the i-th fault priority group is determined based on the s-th fault type in the i-th fault type group and the s-th quantity in the i-th quantity group, where s is a positive integer:

[0124] Determine the s-th weight corresponding to the s-th fault type;

[0125] The s-th fault priority is determined as the product value obtained by performing a product operation on the s-th quantity and the s-th weight.

[0126] In one exemplary embodiment, the device further includes:

[0127] The third determining module is used to determine a priority that is greater than or equal to the priority threshold from the N priorities before processing M groups of faults out of the N groups of faults;

[0128] The fourth determining module is used to determine the M groups of faults corresponding to the M priorities in the N groups of faults when it is determined that there are M priorities that are greater than or equal to the priority threshold, wherein the M priorities include the priority of each group of faults in the M groups of faults.

[0129] In one exemplary embodiment, the third determining module is configured to:

[0130] The following steps are performed to process the w-th group of faults in the M groups of faults, where w is a positive integer less than or equal to M, and the w-th group of faults includes a fault occurring in one of the N servers:

[0131] Identify the target task that is being executed on one of the servers;

[0132] If the w-th group of faults is unrelated to the execution of the target task, the processing of the w-th group of faults shall be suspended, and the processing of the w-th group of faults shall be resumed once the target task has been completed.

[0133] If the w-th group of faults is related to the execution of the target task, a target server is determined from the N servers excluding the first server; the target task is re-executed on the target server, and the processing of the w-th group of faults begins when the re-execution of the target task on the target server begins.

[0134] In one exemplary embodiment, the third determining module is further configured to: determine the task type of the target task;

[0135] A set of candidate servers is determined from the N servers excluding the first server, wherein the set of candidate servers are servers that are allowed to execute tasks of the task type; the server in the set of candidate servers whose load is less than or equal to a preset load threshold is determined as the target server;

[0136] The third determining module is further configured to: when the target task includes a set of tasks, obtain the task execution time of each task in the set of tasks from the server, wherein the task execution time of each task is used to indicate the duration for which each task has been executed on the server; and re-execute each task in the set of tasks in the target server in sequence according to the task execution time of each task in the set of tasks, wherein the task with the longer execution time is re-executed earlier.

[0137] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0138] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0139] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0140] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0141] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0142] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0143] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0144] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A failure handling method characterized by, include: Obtain N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults that occur in the N servers, and N is a positive integer greater than or equal to 1; Based on the N sets of fault description information, determine the N sets of fault types and N sets of quantities corresponding to the N sets of faults. The i-th set of fault types in the N sets of fault types includes the fault types of different faults in the i-th set of faults in the N sets of faults. The i-th set of quantities in the N sets of quantities includes the number of faults of each fault type in the i-th set of faults. i is a positive integer less than or equal to N. Based on the N groups of fault types and the N groups of quantities, determine the N priorities corresponding to the N groups of faults, wherein the i-th priority among the N priorities is used to indicate the processing order of the i-th group of faults; M groups of faults out of the N groups of faults are processed, wherein the priority of the M groups of faults satisfies the preset target matching condition with the preset priority threshold, and M is a positive integer less than or equal to N. The processing of M groups of faults out of the N groups of faults includes: The N groups of faults include a first group of faults and a second group of faults that are coupled; the M groups of faults include the second group of faults; the M groups of faults do not include the first group of faults; and the coupling relationship means that the first group of faults is processed after the second group of faults is processed; and the second group of faults is processed after the first group of faults is processed. The process of handling M groups of faults out of the N groups of faults includes: processing the w-th group of faults out of the M groups of faults by performing the following steps, where w is a positive integer less than or equal to M, and the w-th group of faults includes a fault occurring on one of the N servers: determining the target task being executed on the target task; pausing the processing of the w-th group of faults if the w-th group of faults is unrelated to the execution of the target task, and resuming the processing of the w-th group of faults once the target task is completed; determining the target server from the servers other than the target task among the N servers if the w-th group of faults is related to the execution of the target task; re-executing the target task on the target server, and resuming the processing of the w-th group of faults once the re-executing of the target task on the target server begins; The step of re-executing the target task on the target server includes: if the target task comprises a set of tasks, obtaining the task execution time of each task in the set of tasks from the server, wherein the task execution time of each task is used to indicate the duration for which each task has been executed on the server; and re-executing each task in the set of tasks sequentially on the target server according to the task execution time of each task in the set of tasks, wherein tasks with longer execution times are re-executed earlier.

2. The method according to claim 1, characterized in that, The step of determining the N fault types and the number of faults corresponding to the N faults based on the N sets of fault description information includes: By performing the following steps, based on the i-th group of fault description information in the N groups of fault description information, the type and quantity of the i-th group of faults corresponding to the i-th group of faults are determined, wherein the i-th group of fault description information is used to describe the fault occurring in the i-th server among the N servers, and the i-th group of faults includes the fault occurring in the i-th server: Based on the fault description information of the i-th group, determine the fault type of the i-th group; The number of the i-th group is determined based on the fault description information of the i-th group.

3. The method according to claim 2, characterized in that, The step of determining the fault type of the i-th group based on the fault description information of the i-th group includes: determining the type identifier of the i-th group from the fault description information of the i-th group; and determining the fault type of the i-th group corresponding to the type identifier of the i-th group from the category identifiers and fault types that have a corresponding relationship. The step of determining the quantity of the i-th group based on the i-th group of fault description information includes: determining the r-th quantity in the i-th group of quantities by performing the following steps, wherein the r-th quantity is the number of faults of the r-th fault type in the i-th group of faults, and r is a positive integer greater than or equal to 1; and determining the r-th quantity as the number of times the r-th type identifier appears in the i-th group of fault description information, wherein the r-th type identifier is the identifier of the fault of the r-th fault type.

4. The method according to claim 1, characterized in that, The step of determining the N priorities corresponding to the N groups of faults based on the N groups of fault types and the N groups of fault quantities includes: The i-th priority corresponding to the i-th group of faults is determined by performing the following steps: Based on the fault type of the i-th group and the quantity of the i-th group, determine the fault priority of the i-th group corresponding to the fault in the i-th group; The i-th priority is determined as the sum of the fault priorities in the i-th group of fault priorities.

5. The method according to claim 4, characterized in that, The step of determining the fault priority of the i-th group corresponding to the i-th group of faults based on the i-th group of fault types and the i-th group of faults includes: By performing the following steps, the s-th fault priority in the i-th fault priority group is determined based on the s-th fault type in the i-th fault type group and the s-th quantity in the i-th quantity group, where s is a positive integer: Determine the s-th weight corresponding to the s-th fault type; The s-th fault priority is determined as the product value obtained by performing a product operation on the s-th quantity and the s-th weight.

6. The method according to any one of claims 1 to 5, characterized in that, Before processing M groups of faults out of the N groups of faults, the method further includes: From the N priorities, determine the priority that is greater than or equal to the priority threshold; If it is determined that there are M priorities that are greater than or equal to the priority threshold, the M groups of faults corresponding to the M priorities are determined from the N groups of faults, wherein the M priorities include the priority of each group of faults in the M groups of faults.

7. The method according to claim 1, characterized in that, Determining the target server from the N servers excluding the first server includes: Determine the task type of the target task; A set of candidate servers is determined from the N servers excluding the one server, wherein the set of candidate servers are servers that are allowed to execute tasks of the task type; The server whose load is less than or equal to a preset load threshold from the group of candidate servers is identified as the target server.

8. A fault handling device, characterized in that, include: The acquisition module is used to acquire N sets of fault description information corresponding to N servers, wherein the N sets of fault description information are used to describe N sets of faults that occur in the N servers, and N is a positive integer greater than or equal to 1; The first determining module is used to determine the N groups of fault types and N groups of quantities corresponding to the N groups of faults based on the N groups of fault description information. The i-th group of fault types in the N groups of fault types includes the fault types of different faults in the i-th group of faults in the N groups of faults, and the i-th group of quantities in the N groups of quantities includes the number of faults of each fault type in the i-th group of faults, where i is a positive integer less than or equal to N. The second determining module is used to determine N priorities corresponding to the N groups of faults based on the N groups of fault types and the N groups of numbers, wherein the i-th priority among the N priorities is used to represent the processing order of the i-th group of faults; The processing module is used to process M groups of faults out of the N groups of faults, wherein the priority of the M groups of faults satisfies the preset target matching condition with the preset priority threshold, and M is a positive integer less than or equal to N. The processing module is further configured to: process the first group of faults when the N groups of faults include a first group of faults and a second group of faults that are coupled, the M groups of faults include the second group of faults, the M groups of faults do not include the first group of faults, and the coupling relationship indicates that the second group of faults is processed after the first group of faults is processed; and process the second group of faults after the first group of faults has been processed. The third determining module is configured to: process the w-th group of faults in the M groups of faults by performing the following steps, where w is a positive integer less than or equal to M, and the w-th group of faults includes faults occurring on one of the N servers: determining the target task being executed on the target task; pausing the processing of the w-th group of faults if the w-th group of faults is unrelated to the execution of the target task, and resuming the processing of the w-th group of faults if the target task is completed; determining the target server from the servers other than the target task among the N servers if the w-th group of faults is related to the execution of the target task; re-executing the target task on the target server, and starting the processing of the w-th group of faults if the re-executing of the target task on the target server begins; The third determining module is further configured to: when the target task includes a set of tasks, obtain the task execution time of each task in the set of tasks from the server, wherein the task execution time of each task is used to indicate the duration for which each task has been executed on the server; and re-execute each task in the set of tasks in the target server in sequence according to the task execution time of each task in the set of tasks, wherein the task with the longer execution time is re-executed earlier.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Server fault diagnosis rule screening method based on GRA

    CN113791924A

  • Fault processing method and device, electronic equipment and storage medium

    CN117290144A