Fault self-healing method, device, equipment and storage medium

Abstract

The present application relates to the field of data processing technology, and in particular provides a fault self-healing method, device, equipment and storage medium, the method comprising: obtaining alarm information when a system fault is detected; judging the self-healing level corresponding to the alarm information; when the self-healing level is the first level, determining the target operation and maintenance program set in a preset rule database based on the alarm information; when the self-healing level is the second level, processing the alarm information using a pre-trained reinforcement learning model to determine the target operation and maintenance program set; running each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing. The solution of the present application not only relies on pre-set expert rules to deal with known faults, but also can dynamically handle unknown or complex fault conditions, significantly improving the system's ability to deal with unknown faults.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a fault self-healing method, device, equipment and storage medium. Background Art

[0002] Supercomputers are complex in structure, large in scale, and have tightly coupled components, which makes ensuring their reliability a challenging task. Failures in supercomputer systems require a long period of accumulation before they can be discovered in the later production environment. At this time, under the premise that failures are difficult to avoid, reducing the fault handling time through fault self-healing methods and thus increasing the mean time between failures is an effective means to improve the reliability of supercomputers.

[0003] Common methods of fault handling include fault isolation and fault self-healing. The main goal of fault isolation is to prevent jobs or tasks from running on faulty nodes or in faulty areas. Fault self-healing uses repair technology to repair software faults and restore the supercomputer system to its pre-fault state. In a sense, fault isolation is a buffer before fault self-healing.

[0004] However, in the related fault handling methods, only the faults with known causes can be self-healed. How to self-heal the faults with unknown causes is an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a fault self-healing method, device, equipment and storage medium, which not only rely on pre-set expert rules to deal with known faults, but also can dynamically handle unknown or complex fault conditions, significantly improving the ability of supercomputer systems to deal with unknown faults.

[0006] In a first aspect, the present application provides a fault self-healing method, the method comprising: obtaining alarm information when a system fault is detected; determining the self-healing level corresponding to the alarm information; when the self-healing level is a first level, determining a target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to the level corresponding to a fault with a known fault cause and a pre-set self-healing rule; when the self-healing level is a second level, processing the alarm information using a pre-trained reinforcement learning model to determine a target operation and maintenance program set, wherein the second level refers to the level corresponding to a fault with an unknown fault cause or without a pre-set self-healing rule; and running each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0007] In the second aspect, the present application provides a fault self-healing device, which includes: an alarm information acquisition module, which is used to obtain alarm information when a system fault is detected; a self-healing level judgment module, which is used to judge the self-healing level corresponding to the alarm information; a rule base processing module, which is used to determine a target operation and maintenance program set in a preset rule database based on the alarm information when the self-healing level is the first level, wherein the first level refers to the level corresponding to the fault with a known fault cause and a pre-set self-healing rule; a reinforcement learning processing module, which is used to process the alarm information using a pre-trained reinforcement learning model to determine a target operation and maintenance program set when the self-healing level is the second level, wherein the second level refers to the level corresponding to the fault with an unknown fault cause or without a pre-set self-healing rule; a program running module, which is used to run each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0008] In a third aspect, the present application provides a fault self-healing device, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement the fault self-healing method as described in the first aspect above.

[0009] In a fourth aspect, the present application provides a storage medium, which may be a computer-readable storage medium, storing a computer program thereon, which, when executed by a processor, implements the fault self-healing method in the first aspect described above.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the fault self-healing method as described in any one of the above-mentioned first aspects is implemented.

[0011] Compared with the prior art, the technical solution provided by the embodiments of the present application has the following advantages:

[0012] The embodiment of the present application provides a fault self-healing method, device, equipment and storage medium, the method comprising: obtaining alarm information when a system fault is detected; judging the self-healing level corresponding to the alarm information; when the self-healing level is the first level, determining the target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to the level corresponding to the fault with a known fault cause and a pre-set self-healing rule; when the self-healing level is the second level, processing the alarm information using a pre-trained reinforcement learning model to determine the target operation and maintenance program set, wherein the second level refers to the level corresponding to the fault with an unknown fault cause or no pre-set self-healing rule; running each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing. Since after obtaining the alarm information, the fault level is judged based on the alarm information, when the fault cause is known and there is a pre-set self-healing rule, the operation and maintenance program to be executed is determined using the pre-set self-healing rule, and the above operation and maintenance program is executed to perform fault self-healing. When the cause of the fault is unknown or there is no corresponding self-healing rule, the reinforcement learning algorithm is used to process the alarm information, determine the operation and maintenance procedures that need to be executed, and execute the above operation and maintenance procedures to perform self-healing of the fault. It not only relies on pre-set expert rules to deal with known faults, but also can dynamically handle unknown or complex fault conditions, significantly improving the system's ability to deal with unknown faults and the system's flexibility in handling faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] Figure 1 A flowchart of a fault self-healing method provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of the structure of a fault self-healing device provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0019] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0020] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] A supercomputer is a computer with extremely high computing power, capable of performing complex and large-scale computing tasks that are difficult for ordinary computers to complete. Supercomputers far exceed regular personal computers or servers in performance. They usually have the following characteristics: Extremely high computing speed: The computing speed of supercomputers is usually measured in petaflops (PFLOPS) or exaflops (EFLOPS) of floating-point operations per second. Massive parallel processing: Supercomputers are usually composed of thousands of processor cores, which can perform a large number of computing tasks simultaneously. High-speed network interconnection: In order to achieve high-speed data transmission between processors, supercomputers are equipped with high-speed network interconnection technology. Large-capacity storage system: Due to the need to process and store large amounts of data, supercomputers are equipped with high-performance storage systems. Specialized hardware acceleration: Supercomputers may contain hardware accelerators such as graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs) to improve the performance of specific types of computing tasks. Advanced cooling system: Due to the large amount of heat generated, supercomputers require efficient cooling systems to keep the hardware running normally. System software and tools: Supercomputers run specialized operating systems and software to support their high-performance computing tasks and optimize the use of resources. Since supercomputers are usually composed of a large number of processors, storage resources and high-speed network interconnection, supercomputers are also called supercomputer systems.

[0024] As can be seen from the above, supercomputers are complex in structure, large in scale, and have tightly coupled components, making it a very challenging task to ensure the reliability of supercomputer operation. Supercomputers in production environments are threatened by failures almost every day. Among these failures, software failures are more common than hardware failures, and software failures are usually more difficult to recover than hardware failures.

[0025] It is well known that supercomputers are difficult to develop, have a long cycle and are expensive. It is difficult for supercomputer manufacturers to complete comprehensive testing before production, and it is also difficult for supercomputer users to obtain a large amount of testing time before production. Many problems in supercomputer systems require a long period of accumulation before they can be discovered in the later production environment. However, at this time, the possibility of error avoidance, error transfer and fault tolerance has actually been missed. Therefore, under the premise that failures are difficult to avoid, reducing the fault handling time through fault self-processing (fault self-healing) methods, thereby increasing the mean time between failures, is an effective means to improve the reliability of supercomputer systems.

[0026] Common fault repair implementation technologies for traditional Internet technology (IT) service clusters include application detection, service fuse, service degradation, current limiting, and traffic diversion. Among them, application detection technology is a method of active operation and maintenance. On the premise of actively discovering faults, it executes predetermined isolation and processing strategies according to the type of fault. System overload will cause service blockage and unavailability, and resources will be exhausted. Commercial data centers usually use service fuses to temporarily interrupt related calls of overloaded services to prevent the entire system from being unavailable. When system resources are fully restored, services are restored. When the server pressure suddenly surges, in order to ensure that the core business can still provide services, commercial data centers use service degradation methods to shut down certain non-core services to release server resources to ensure the normal operation of core tasks. Current limiting technology is used to control the number of concurrent requests. When the number of accesses exceeds a certain number, new access requests are rejected to ensure stable system operation. Traffic diversion technology is a common function in the load balancing mechanism. When the number of requests exceeds a certain concurrency, the load balancing device diverts the traffic to other servers to provide services, thereby improving service efficiency and reducing service pressure.

[0027] Supercomputers are different from traditional IT service clusters in that they are usually composed of components with different functions and provide services as a whole. The types of failures that occur in each functional component are different. Therefore, different fault repair methods are needed to improve the reliability of each functional component.

[0028] The computing array in the supercomputer system is mainly responsible for supporting the operation of high-performance computing applications. For failures in the computing array, the computing node can be actively tested before running the user job, and the computing nodes that fail the test can be removed from the resource pool in advance by marking different node states. However, this test cannot completely guarantee that the computing node will not fail during the operation.

[0029] In addition, fault prediction and anomaly detection technologies can be used to identify faulty computing nodes in advance, prompting jobs to record job checkpoints more promptly, and then subsequently remove faulty nodes from the resource pool and resubmit jobs to normal nodes for execution.

[0030] Login nodes are generally used for users to log in to supercomputer systems in order to use computing resources. Supercomputer systems generally have multiple login nodes to prevent users from being unable to log in to the system due to failure of a single login node.

[0031] Management nodes are mainly used to deploy supercomputer system management software, such as user management tools (LDAP), resource scheduling tools (SLURM), etc. If these software fail, the entire system will be unusable. For this reason, supercomputers currently generally deploy and run the main system management software in a master-slave mode. When the master node goes down due to some kind of failure, the backup node will directly replace the function of the master node, so that the entire machine service is not affected.

[0032] Storage arrays are used to store data generated by high-performance computing applications. Storage array failures are usually caused by hardware and software. Hardware failures are mainly caused by failures in disks, memory, or high-speed interconnection networks, while software failures are mainly caused by file systems or operating systems. Since the file system used in supercomputers usually builds a local file system first and then a parallel file system, the high availability of hard disks is usually guaranteed by the local file system to ensure redundancy. When other hardware or software failures cause the storage node to need to be restarted, there is no good automatic operation and maintenance method, and it is mainly solved manually. If the user cannot log in due to overload, use the current limiting method to alleviate the problem.

[0033] In the above-mentioned related fault handling methods, only self-healing can be performed for faults with known causes. Faults with unknown causes usually require intervention by operation and maintenance personnel. How to make the system automatically self-heal for faults with unknown causes is an urgent problem to be solved.

[0034] To solve the above technical problems, an embodiment of the present application provides a fault self-healing method, which includes: when a system fault is detected, obtaining alarm information; judging the self-healing level corresponding to the alarm information; when the self-healing level is the first level, determining a target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to the level corresponding to the fault with a known fault cause and a pre-set self-healing rule; when the self-healing level is the second level, processing the alarm information using a pre-trained reinforcement learning model to determine the target operation and maintenance program set, wherein the second level refers to the level corresponding to the fault with an unknown fault cause or without a pre-set self-healing rule; and running each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0035] After the alarm information is obtained, the fault level is determined based on the alarm information. When the cause of the fault is known and there are pre-set self-healing rules, the pre-set self-healing rules are used to determine the operation and maintenance procedures that need to be executed, and the above operation and maintenance procedures are executed to perform self-healing of the fault. When the cause of the fault is unknown or there are no corresponding self-healing rules, the alarm information is processed using a reinforcement learning algorithm to determine the operation and maintenance procedures that need to be executed, and the above operation and maintenance procedures are executed to perform self-healing of the fault. It not only relies on pre-set expert rules to deal with known faults, but also can dynamically handle unknown or complex fault conditions, significantly improving the ability of supercomputer systems to deal with unknown faults and the flexibility of supercomputer systems in handling faults.

[0036] The fault self-healing method provided by the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0037] Embodiment 1

[0038] Figure 1 This is a flowchart of a fault self-healing method in an embodiment of the present application. This embodiment can be applied to the case of handling a fault in a supercomputer. The method can be executed by a fault self-healing device. The fault self-healing device can be implemented in software and / or hardware. The fault self-healing device can be configured in a supercomputer or an IT service cluster. In the embodiment of the present application, the fault self-healing method is applied in a supercomputer as an example for explanation.

[0039] The fault self-healing method provided in the embodiment of the present application is applied in a supercomputer. After receiving a system startup instruction and the supercomputer is started, the fault self-healing method starts to be executed to automatically repair faults occurring in the supercomputer.

[0040] like Figure 1 As shown, the fault self-healing method provided in the embodiment of the present application mainly includes steps S101-S105.

[0041] S101. When a system failure is detected, alarm information is obtained.

[0042] The above-mentioned system may refer to a service cluster including one or more servers. In the embodiment of the present application, the system is described as a supercomputer as an example.

[0043] System failure can be understood as any one or more components in the service cluster failing and being unable to function normally, wherein the above components include: any server in the computing array, any server in the storage array, a server at the login node, or a server at the management node.

[0044] Among them, the computing array in the supercomputer system is mainly responsible for supporting the operation of high-performance computing applications. The login node is generally used for users to log in to the supercomputer system in order to use computing resources. The management node is mainly used to deploy the system management software of the supercomputer. The storage array is used to store the data generated by high-performance computing applications.

[0045] Among them, supercomputer failures may include but are not limited to the following known reasons: operating system failures, failures caused by application errors, failures caused by parallel computing errors, failures caused by data management problems, failures caused by network and communication problems, failures caused by resource management problems, failures caused by software security problems, failures caused by compatibility problems, failures caused by user operation errors, and failures caused by dependent library and framework problems.

[0046] Supercomputer failures also include: failures with unknown causes or failures of complex situations. Failures with unknown causes refer to failures that can only be detected as a system failure, but the cause of the failure cannot be determined. For example: a supercomputer crashes, but the cause of the crash cannot be determined. Another example: a supercomputer central processing unit (CPU) is in a full-load operation state for a long time, but the specific reason why the CPU is fully loaded for a long time cannot be determined. Complex failures can be understood as failures of supercomputers caused by multiple reasons.

[0047] After the supercomputer is started, the operating status of the supercomputer is monitored in real time to determine whether the supercomputer fails. After determining that a failure occurs, alarm information is obtained. The alarm information can be understood as information related to the failure. For example, the alarm information can include the failed component, the failure phenomenon, and the cause of the failure.

[0048] In a possible implementation, an alarm trigger and a trigger rule corresponding to the alarm trigger are preset, and setting information for the alarm trigger may be stored in a configuration memory.

[0049] After the supercomputer is started, the alarm trigger uses the alarm trigger rules to monitor and evaluate the health status of the supercomputer in real time, and obtains alarm information when a supercomputer failure is detected.

[0050] Table 1 is a definition and example of an alarm intensity level provided in an embodiment of the present application. As described in Table 1, the alarm level mainly includes four alarm intensities, namely, notification, warning, fault, and disaster.

[0051] Table 1

[0052]

[0053] Since the trigger conditions of the alarm trigger are pre-set, during the operation of the supercomputer, the operation status of the supercomputer is monitored in real time, and the level of the alarm information is determined based on the operation status and the pre-set trigger conditions. For example: during the operation of the supercomputer, if the disk usage rate is monitored to exceed 70%, the warning trigger condition is met and the alarm information related to the alarm is generated. For another example: during the operation of the supercomputer, if a service component is monitored to be disabled, the fault trigger condition is met and the alarm information related to the fault is generated.

[0054] S102: Determine the self-healing level corresponding to the alarm information.

[0055] In a possible implementation, when the alarm trigger and the triggering rule corresponding to the alarm trigger are pre-set, the fault level corresponding to the alarm trigger can be set at the same time. The setting information for the alarm trigger can be stored in the configuration memory.

[0056] The fault level can be divided according to whether the cause of the fault is known and the complexity of the fault. Table 2 is a definition and example table of a self-healing level provided in an embodiment of the present application. As shown in Table 2, the self-healing level is divided into three categories, including the third level, the second level and the first level.

[0057] Table 2

[0058]

[0059] As shown in Table 2, the first level can be understood as the matching self-healing level, that is, there are clear expert self-healing rules in the rule database for the faults that have occurred, and the system can be restored to a normal state by automatically executing the operation and maintenance procedures.

[0060] The second level can be understood as the possible self-healing level. That is, it is possible to repair through system self-processing, but there are no known clear processing rules. For example, all kinds of failures with unknown causes basically belong to the possible self-healing level. Such failures can be self-processed and repaired through the reinforcement learning model.

[0061] The third level can be understood as the non-self-healing level, that is, the fault that cannot be repaired by automated processing methods and requires manual intervention. Exemplarily, non-self-healing faults may include hardware faults, such as disk damage, power failure, network cable disconnection, etc., which cannot be repaired by automated processing methods and can only be repaired by manual intervention.

[0062] Before the supercomputer runs, the self-healing level corresponding to each fault is preset. For example, the self-healing level corresponding to power failure and disk damage is the third level, that is, the non-self-healing level. Another example: the supercomputer system is attacked by a virus and the virus cannot be intercepted and cleaned. The self-healing level corresponding to this fault is the second level, that is, the possible self-healing level. Another example: due to improper resource allocation, the CPU usage rate is too high. This fault has an expert self-healing rule. Therefore, the self-healing level corresponding to this fault is the first level, that is, the matching self-healing level.

[0063] The self-healing level corresponding to each fault is preset and stored in the configuration memory. After a supercomputer fault is detected, the self-healing level corresponding to the supercomputer fault is queried in the above correspondence according to the supercomputer fault to determine the self-healing level corresponding to the supercomputer fault, that is, the fault level corresponding to the alarm information.

[0064] Furthermore, the self-healing level corresponding to each fault can be set before the supercomputer is started, or can be set or updated after the supercomputer is started. The above update includes adding a new fault and its corresponding self-healing level, modifying the self-healing level corresponding to an existing fault, or deleting a fault.

[0065] S103. When the self-healing level is the first level, determine a target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to a level corresponding to a fault with a known fault cause and a preset self-healing rule.

[0066] The self-healing level of the alarm information is the first level, which can be understood as the corresponding self-healing treatment measures for the failure of the supercomputer can be determined by the rules in the existing rule database. The target operation and maintenance program set can include multiple operation and maintenance programs, and the order and dependency relationship between each operation and maintenance program are set.

[0067] The rule database sets a correspondence between the alarm information and the operation and maintenance program set identifier corresponding to the alarm information. After determining that the self-healing level corresponding to the alarm information is the first level, the rule data is searched based on the alarm information, the searched operation and maintenance program set identifier is used as the target operation and maintenance program set identifier, and the target operation and maintenance program set corresponding to the target operation and maintenance program set identifier is obtained.

[0068] In a possible implementation, the fault self-healing rule pairs are summarized, that is, the alarm information that can be self-healed and the set identifier of the operation and maintenance program that can eliminate the fault are bound together as a self-healing rule and input into the rule database. The operation and maintenance program set may include multiple operation and maintenance program identifiers and the sequence and dependency relationship between each operation and maintenance program.

[0069] The correspondence between the alarm information and the operation and maintenance program set identifier corresponding to the alarm information is stored in the expert rule table in the configuration memory. The expert rule table specifically includes the following information: rule creation time, operation script, whether to enable, and rule update time.

[0070] After each operation and maintenance program included in the operation and maintenance program set is run, the supercomputer system can be restored to the state before the failure, that is, the supercomputer fault self-healing is achieved. In other words, the self-healing rules stored in the rule database are a set of a series of operation and maintenance programs. The above target operation and maintenance program set can be understood as the self-healing rules for handling the current failure of the supercomputer.

[0071] In one possible implementation, the correspondence between the alarm identifier and the program set identifier is stored in a rule database; based on the alarm information, the target operation and maintenance program set is determined in a preset rule database, including: extracting the alarm identifier from the alarm information; based on the alarm identifier, querying in the correspondence between the alarm identifier and the program set identifier to determine the corresponding operation and maintenance program identifier; and determining the corresponding target operation and maintenance program set based on the operation and maintenance program identifier.

[0072] Among them, the alarm information is generally composed of an alarm template and an alarm variable. The alarm template refers to the main part of the alarm. For example, the alarm information is "The CPU usage of server A1 exceeds 90%." The alarm template is: "CPU usage exceeds 90%", and the alarm variable is: "Server A1". Another example: The disk usage of / dev / sda is greater than 70%, where "${device}'s disk usage is greater than 70%" is the alarm template, and " / dev / sda" is the alarm variable.

[0073] The alarm template can be used directly as the alarm identification, or a serial number can be assigned to each alarm template as the alarm identification. In this embodiment, the alarm template is used as the alarm identification as an example for description.

[0074] In a possible implementation, after the alarm information is obtained, the alarm information is split into two parts: the alarm template and the alarm variable. Based on the alarm template, the rule database is queried to determine the operation and maintenance program identifier corresponding to the alarm template, and the operation and maintenance program corresponding to each operation and maintenance program identifier is used as the target operation and maintenance program set. For example, the operation and maintenance program identifiers corresponding to the alarm template include (a1, a2, a3, a4), then the 4 operation and maintenance programs corresponding to (a1, a2, a3, a4) are obtained as the target operation and maintenance program set.

[0075] In another possible implementation, based on the alarm template, the alarm template is queried in the rule database to determine the operation and maintenance program set identifier corresponding to the alarm template, the operation and maintenance program set identifier corresponds to multiple operation and maintenance program identifiers, and the operation and maintenance programs corresponding to each operation and maintenance program identifier are used as the target operation and maintenance program set. For example, the operation and maintenance program set identifier corresponding to the alarm template is A1, and A1 is pre-set to include four operation and maintenance program identifiers (a1, a2, a3, a4), then the four operation and maintenance programs corresponding to (a1, a2, a3, a4) included in A1 are obtained as the target operation and maintenance program set.

[0076] S104. When the self-healing level is the second level, the alarm information is processed using a pre-trained reinforcement learning model to determine a target operation and maintenance program set. The second level refers to the level corresponding to faults with unknown fault causes or no pre-set self-healing rules.

[0077] The self-healing level corresponding to the alarm information is the second level. It can be understood that there is currently no clear expert self-healing rule for the failure of the supercomputer, but it is possible to determine a target operation and maintenance program set that can achieve fault self-healing through the operating status of the supercomputer and the reinforcement learning model.

[0078] In one possible implementation, a pre-trained reinforcement learning model is used to process alarm information to determine a target operation and maintenance program set, including: determining a faulty functional component from the alarm information; obtaining parameter values ​​of each indicator of the faulty functional component; for each indicator, determining a state value of the indicator based on the parameter value of the indicator; querying a Q table based on the state value of the indicator to obtain a Q value corresponding to each operation and maintenance program identifier, wherein the rows of the Q value table represent each state value, and the columns of the Q table represent each operation and maintenance program identifier, and the Q value corresponding to the tuple of the state value and the operation and maintenance program identifier in the Q table is obtained by a Q-learning algorithm; selecting a target operation and maintenance program identifier based on the Q value; and constructing a target operation and maintenance program set based on multiple target operation and maintenance program identifiers.

[0079] The failed functional component refers to a specific server or processor that has failed. In the self-healing scenario, the above indicators include but are not limited to the following information: CPU usage, memory usage, disk I / O, network traffic, all specific software system monitoring indicators and log information of the failed functional component. For example, specific software system monitoring indicators can be understood as the need to reinforce the learning model to anticipate the need to handle file system software failures, so it is necessary to add relevant indicators of the file system software when training the model.

[0080] The indicator status value is determined using the parameter values ​​of each indicator provided above. Among them, the indicator parameter value refers to the specific value of each indicator, for example: the CPU utilization rate is 78%, the memory utilization rate is 56%, the network traffic is 50Mbps, etc. The indicator status value can be understood as an identifier representing the indicator range. For example, the indicator status value may include three status values: low, medium, and high. The correspondence between the range of each indicator parameter value and the indicator status value is pre-set. After obtaining the indicator parameter values ​​of each functional component, for each indicator, based on the indicator parameter value, query in the correspondence between the range of the indicator parameter value and the indicator status value to determine its corresponding indicator status value.

[0081] For example, let's take the CPU usage as an example. It is preset that when the CPU usage is less than 70%, the corresponding CPU usage status value is low, when the CPU usage is less than 90% and greater than 70%, the corresponding CPU usage status value is medium, and when the CPU usage is greater than 90%, the corresponding CPU usage status value is high. When the CPU usage of the faulty server is 78%, it can be determined that the CPU usage status value is medium.

[0082] For the parameter values ​​of each indicator in the faulty server that have been obtained, the above steps are executed to determine the state value corresponding to each indicator, and the state s is constructed based on the state value corresponding to each indicator, which is the set of all indicator states, that is, s={indicator 1 state value, indicator 2 state value, indicator 3 state value, ..., indicator n state value}. Exhaust all possible combinations S={s1, s2, ..., s n} as the state set in the Q-table.

[0083] The main part of the reinforcement learning model is the Q table, and the Q-learning method is mainly used to determine the Q table of the reinforcement learning model. The core idea of ​​the Q-learning algorithm is to learn the optimal strategy through interaction with the environment.

[0084] Each row in the Q table represents a combination of state values, for example: the first row is state combination s1, the second row is state combination s2, ..., the nth row is state combination sn. Among them, state combination s1 = {low, low, low, ..., low}; state combination s2 = {low, high, low, ..., low}; state combination sn = {high, high, high, ..., high}. Enumerate all possible state combinations in the rows of the Q table. Each column in the Q table represents the identifier of an operation and maintenance program. Enumerate all executable operation and maintenance program set identifiers in the columns of the Q table. For example: A1, A2, A3, ..., A n .

[0085] The Q values ​​corresponding to all state combinations and operation and maintenance program set identifier tuples are initialized to 0, and the Q values ​​in the Q table are learned according to the reinforcement learning algorithm to obtain updated Q values.

[0086] After determining the state combination s' formed by the state values ​​of the indicators corresponding to the functional components of the supercomputer that have failed, a query is performed in the Q table based on the state combination to find the row where the state combination s' is located, and then the Q values ​​corresponding to the operation and maintenance program set identifiers in the row are obtained; the ε-greedy algorithm is used to select the target operation and maintenance program set identifier from each operation and maintenance program set identifier.

[0087] Specifically, when the operation and maintenance program set identifier needs to be executed, a random number n is generated. If the random number n is less than ε, an operation and maintenance program set identifier is randomly selected from each operation and maintenance program set identifier as the target operation and maintenance program set identifier. If the random number n is greater than or equal to ε, the operation and maintenance program set identifier with the largest Q value is selected as the target operation and maintenance program set identifier. If the Q values ​​corresponding to multiple operation and maintenance program set identifiers are the same, an operation and maintenance program set identifier is randomly selected from the operation and maintenance program set identifiers with the same Q value as the target operation and maintenance program set identifier. Among them, ε is learned according to the reward value in the reinforcement learning algorithm. As time changes, ε is changed to a smaller value according to the change of reward feedback, so that more and more emphasis is placed on experience selection.

[0088] Acquire multiple operation and maintenance program identifiers included in the target operation and maintenance program set identifier, and use the operation and maintenance programs corresponding to the multiple operation and maintenance program identifiers as the target operation and maintenance program set.

[0089] The reinforcement learning algorithm is introduced below.

[0090] In one possible implementation, the steps of obtaining the Q value in the Q table through the reinforcement learning algorithm include: initializing the Q value in the Q table; randomly selecting an operation and maintenance program identifier for each state value in the row of the Q table; running the operation and maintenance program corresponding to the operation and maintenance program identifier; obtaining the system state and reward value of the system after the operation and maintenance program is run; determining the updated Q value of the state value and the operation and maintenance program identifier tuple based on the system state, the reward value and a pre-set Q value calculation rule, and returning to execute the step of randomly selecting an operation and maintenance program identifier until the iteration stop condition is met.

[0091] The Q values ​​corresponding to all state combinations and operation and maintenance program set identifier tuples are initialized to 0, and the Q table of Q value initialization is shown in Table 3. The present application embodiment uses 4 state combinations and 4 operation and maintenance program set identifiers as an example for explanation.

[0092] Table 3

[0093]

[0094] In this application test, the following operations are performed for each of the above state combinations s1, s2, s3 and s4.

[0095] The ε-greedy algorithm is used to select the operation and maintenance program set identifier that may be executed by the state combination. The initial value of ε is set to 0.1. Each time the operation and maintenance program set identifier needs to be selected, a random number n is generated. If the random number is less than ε, an operation and maintenance program set identifier is randomly selected. If the random number n is greater than or equal to ε, the operation and maintenance program set identifier with the largest Q value is selected. If the Q values ​​of multiple operation and maintenance program set identifiers are the same, an operation and maintenance program set identifier is randomly selected from the operation and maintenance program set identifiers with the same Q value. Over time, changing ε to a smaller value based on the change in reward feedback makes it increasingly focused on experience selection.

[0096] After selecting the operation and maintenance program set identifier according to the above steps, obtain each operation and maintenance program corresponding to the operation and maintenance program set identifier, and execute each operation and maintenance program. After the operation and maintenance program is executed, obtain the operating status of the system, and update the Q value corresponding to the operation and maintenance program set identifier based on the operating status of the system.

[0097] Specifically, the Q value calculation rule is expressed by the following formula:

[0098]

[0099] in, is the learning rate, is the reward value, γ is the discount factor, For new status The maximum Q value under the condition that after running the operation and maintenance program, the fault is eliminated and no new faults are generated, the reward value A positive value means that if the fault is not eliminated after running the operation and maintenance program, or a new fault occurs after running the operation and maintenance program, or the system performance cannot meet the normal operating conditions, the reward value Is a negative value.

[0100] Regarding the setting of rewards, set a positive reward: when the supercomputer system runs the operation and maintenance program, the alarm disappears and no new alarm is generated, a positive reward can be given, r is 5. Negative reward: if the action causes the system performance to deteriorate or the problem to worsen, a negative reward should be given, r is -5. Zero reward: for ineffective or neutral actions, r is 0.

[0101] S105: Run each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0102] Before executing the fault self-healing method provided in the embodiment of the present application, an executable operation and maintenance program is pre-written, and a set of executable programs for system management is used as an operation and maintenance action set. The above operation and maintenance program can be completed by a scripting language such as shell, python, lua, etc.

[0103] Each operation and maintenance program can receive action parameters, which include, for example, the name of the action execution server, etc. The parameter format is json, such as {"variable name 1": "value 1", "variable name": "value 2"}. The operation and maintenance program is stored in the specified path of the server where the configuration storage is located in the form of text, such as / opt / thsre, etc. The metadata of the operation and maintenance program is stored in the configuration storage, mainly including the storage action id, action name, action storage path, addition time, etc.

[0104] In a possible implementation, the alarm information includes an alarm variable; running each operation and maintenance program in the target operation and maintenance program set includes: substituting the alarm variable into each operation and maintenance program; and running the operation and maintenance programs in sequence according to the sequence and dependency relationships of each operation and maintenance program in the target operation and maintenance program set.

[0105] An operation and maintenance program set consists of multiple or one operation and maintenance programs. The execution order and dependency relationships between the operation and maintenance programs involved in the operation and maintenance program set can be indicated through a configuration file in the yml format.

[0106] Substitute the alarm variable into the operation and maintenance program to indicate which fault each operation and maintenance program needs to repair. For example, if the alarm variable is "Server A", substitute Server A into the operation and maintenance program to instruct the operation and maintenance program to execute the operation and maintenance program on Server A and self-heal the fault on Server A.

[0107] After the alarm variables are substituted into each operation and maintenance program, each operation and maintenance program can be executed in the main server, which is the server that executes the fault self-healing method. Each operation and maintenance program can also be sent to the server where the fault occurs for execution.

[0108] Based on the above embodiment, it also includes: obtaining fault handling results; and optimizing the reinforcement learning model using the target operation and maintenance program set identifier and the fault handling results.

[0109] The fault handling result may include automatically generated processing results such as whether the fault is self-healed and whether new faults are generated, and may also include evaluation information related to the fault handling result input by the operation and maintenance personnel, such as a score value.

[0110] The system will be initially evaluated based on whether the alarm is restored and the system status loss. The operation and maintenance personnel will add appropriate human evaluation based on the actual response of the system in addition to the initial evaluation, and feed back the comprehensive evaluation results to the reinforcement learning model training module as a reference for the next model iteration.

[0111] At the same time, for each fault self-healing operation, a fault self-healing operation audit function is set up to record the script ID, operation time, operation results, etc. of each operation.

[0112] An embodiment of the present application provides a fault self-healing method, which includes: when a supercomputer fault is monitored, obtaining alarm information; determining the self-healing level corresponding to the alarm information; when the self-healing level is the first level, determining a target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to the level corresponding to a fault with a known fault cause and a pre-set self-healing rule; when the self-healing level is the second level, processing the alarm information using a pre-trained reinforcement learning model to determine a target operation and maintenance program set, wherein the second level refers to the level corresponding to a fault with an unknown fault cause or without a pre-set self-healing rule; and running each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0113] The fault self-healing method provided in the embodiment of the present application has the following advantages:

[0114] (1) After the alarm information is obtained, the fault level is determined based on the alarm information. If the cause of the fault is known and there are pre-set self-healing rules, the pre-set self-healing rules are used to determine the operation and maintenance procedures that need to be executed, and the above operation and maintenance procedures are executed to perform self-healing of the fault. If the cause of the fault is unknown or there are no corresponding self-healing rules, the alarm information is processed using a reinforcement learning algorithm to determine the operation and maintenance procedures that need to be executed, and the above operation and maintenance procedures are executed to perform self-healing of the fault. It not only relies on pre-set expert rules to deal with known faults, but also can dynamically handle unknown or complex fault conditions, significantly improving the ability of supercomputer systems to deal with unknown faults and the flexibility of supercomputer systems in handling faults.

[0115] (2) Integrating reinforcement learning for decision optimization not only increases the flexibility of fault handling, but also enables the system to continuously learn and adapt to new fault modes and solution strategies, thereby continuously improving the efficiency and effectiveness of fault handling over time.

[0116] (3) A complete feedback loop is designed to evaluate the effect of each fault response and feed the results back to the reinforcement learning model for the next round of iterative training. This self-evaluation mechanism not only ensures the immediate effect evaluation of fault handling actions, but also optimizes the long-term performance of the entire system through continuous learning and adjustment.

[0117] Embodiment 2

[0118] Figure 2 is a structural diagram of a fault self-healing device in an embodiment of the present application, such as Figure 2 As shown, the fault self-healing device 20 provided in the embodiment of the present application mainly includes:

[0119] Among them, the alarm information acquisition module 21 is used to obtain alarm information when a system failure is detected; the self-healing level judgment module 22 is used to judge the self-healing level corresponding to the alarm information; the rule base processing module 23 is used to determine the target operation and maintenance program set in the preset rule database based on the alarm information when the self-healing level is the first level, wherein the first level refers to the level corresponding to the failure with a known cause of the failure and a pre-set self-healing rule; the reinforcement learning processing module 24 is used to process the alarm information using a pre-trained reinforcement learning model to determine the target operation and maintenance program set when the self-healing level is the second level, and the second level refers to the level corresponding to the failure with an unknown cause of the failure or no pre-set self-healing rule; the program running module 25 is used to run each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

[0120] The embodiment of the present application provides a fault self-healing device, which is used to execute the following process: when a system fault is detected, alarm information is obtained; the self-healing level corresponding to the alarm information is determined; when the self-healing level is the first level, a target operation and maintenance program set is determined in a preset rule database based on the alarm information, wherein the first level refers to the level corresponding to the fault with a known fault cause and a preset self-healing rule; when the self-healing level is the second level, the alarm information is processed using a pre-trained reinforcement learning model to determine the target operation and maintenance program set, and the second level refers to the level corresponding to the fault with an unknown fault cause or no preset self-healing rule; each operation and maintenance program in the target operation and maintenance program set is run to perform fault self-healing. After the alarm information is obtained, the fault level is determined based on the alarm information. When the fault cause is known and there is a preset self-healing rule, the operation and maintenance program to be executed is determined using the preset self-healing rule, and the above operation and maintenance program is executed to perform fault self-healing. When the fault cause is unknown or there is no corresponding self-healing rule, the alarm information is processed using a reinforcement learning algorithm to determine the operation and maintenance program to be executed, and the above operation and maintenance program is executed to perform fault self-healing. It not only relies on pre-set expert rules to deal with known faults, but can also dynamically handle unknown or complex fault conditions, significantly improving the supercomputer system's ability to deal with unknown faults and the supercomputer system's flexibility in handling faults.

[0121] In a possible implementation, the correspondence between the alarm identifier and the program set identifier is stored in the rule database; the rule base processing module 23 is specifically used to extract the alarm identifier from the alarm information; based on the alarm identifier, the correspondence between the alarm identifier and the program set identifier is queried to determine the corresponding operation and maintenance program identifier; based on the corresponding operation and maintenance program identifier, the corresponding target operation and maintenance program set is determined.

[0122] In one possible implementation, the reinforcement learning processing module 24 is specifically used to determine the faulty functional component from the alarm information; obtain the parameter values ​​of each indicator of the faulty functional component; for each indicator, determine the state value of the indicator based on the parameter value of the indicator; query in the Q table based on the state value of the indicator to obtain the Q value corresponding to each operation and maintenance program identifier, the row of the Q value table represents the state value combination, the column of the Q table represents each operation and maintenance program identifier, and the Q value corresponding to the tuple of the state value and the operation and maintenance program identifier in the Q table is obtained through the Q-learning algorithm; select the target operation and maintenance program identifier based on the Q value; and construct a target operation and maintenance program set based on the target operation and maintenance program identifier.

[0123] In one possible implementation, it also includes a reinforcement learning model training module, which is used to obtain the Q value in the Q table through the reinforcement learning algorithm. Specifically, it initializes the Q value in the Q table; for each state value in the row of the Q table, randomly selects an operation and maintenance program identifier; runs the operation and maintenance program corresponding to the operation and maintenance program identifier; obtains the system state and reward value of the system after the operation and maintenance program is run; determines the updated Q value of the state value and the operation and maintenance program identifier tuple based on the system state, the reward value and the pre-set Q value calculation rule, and returns to execute the step of randomly selecting an operation and maintenance program identifier until the iteration stop condition is met.

[0124] In a possible implementation, the Q value calculation rule is expressed by the following formula:

[0125]

[0126] in, is the learning rate, is the reward value, γ is the discount factor, For new status The maximum Q value under the condition that after running the operation and maintenance program, the fault is eliminated and no new faults are generated, the reward value A positive value means that if the fault is not eliminated after running the operation and maintenance program, or a new fault occurs after running the operation and maintenance program, or the system performance cannot meet the normal operating conditions, the reward value Is a negative value.

[0127] In a possible implementation, the alarm information includes an alarm variable; the program running module 25 is specifically used to substitute the alarm variable into each operation and maintenance program; and the operation and maintenance programs are run in sequence according to the sequence and dependency relationship of each operation and maintenance program in the target operation and maintenance program set.

[0128] In a possible implementation, it also includes: a model optimization module, which is used to obtain fault handling results; and optimize the reinforcement learning model using each operation and maintenance program identifier and fault handling results included in the target operation and maintenance program set.

[0129] The fault self-healing device provided in the embodiment of the present application can execute the fault self-healing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0130] Figure 3 is a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device may include a fault self-healing device, and the fault self-healing device may include a supercomputer, such as Figure 3 As shown, the electronic device 300 includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of the processor 310 in the electronic device can be one or more. Figure 3 A processor 310 is taken as an example; the processor 310, the memory 320, the input device 330 and the output device 340 in the electronic device can be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0131] The memory 320, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the fault self-healing method in the embodiment of the present invention. The processor 310 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 320, that is, implements the fault self-healing method provided in the embodiment of the present invention.

[0132] The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include a memory remotely arranged relative to the processor 310, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The input device 330 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, and may include a keyboard, a mouse, etc. The output device 340 may include a display device such as a display screen.

[0134] This embodiment also provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to implement the fault self-healing method provided by the embodiment of the present invention.

[0135] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, and can also execute related operations in the fault self-healing method provided by any embodiment of the present invention.

[0136] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0137] It is worth noting that in the above-mentioned embodiment of the fault self-healing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0138] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0139] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault self-healing method, characterized in that: The method comprises: When a system failure is detected, obtain alarm information; Determine the self-healing level corresponding to the alarm information; In the case where the self-healing level is the first level, determining a target operation and maintenance program set in a preset rule database based on the alarm information, wherein the first level refers to a level corresponding to a fault with a known fault cause and a preset self-healing rule; In the case where the self-healing level is the second level, the alarm information is processed using a pre-trained reinforcement learning model to determine a target operation and maintenance program set, wherein the second level refers to a level corresponding to a fault with an unknown fault cause or a fault without a pre-set self-healing rule; wherein the process of processing the alarm information using a pre-trained reinforcement learning model to determine a target operation and maintenance program set includes: determining a functional component where the fault occurs from the alarm information; obtaining parameter values ​​of each indicator of the functional component where the fault occurs; for each indicator, determining a state value of the indicator based on the parameter value of the indicator; querying a Q table based on the state value of the indicator to obtain a Q value corresponding to each operation and maintenance program identifier, wherein the rows of the Q table represent state value combinations, the columns of the Q table represent each operation and maintenance program identifier, and the Q values ​​corresponding to the tuples of state values ​​and operation and maintenance program identifiers in the Q table are obtained by a Q-learning algorithm; selecting a target operation and maintenance program identifier based on the Q value; and constructing a target operation and maintenance program set based on the target operation and maintenance program identifier. Run each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

2. The method according to claim 1, characterized in that The rule database stores the corresponding relationship between the alarm identifier and the program set identifier; Determining a target operation and maintenance program set in a preset rule database based on the alarm information includes: Extracting an alarm identifier from the alarm information; Based on the alarm identifier, searching in the correspondence between the alarm identifier and the program set identifier to determine the corresponding operation and maintenance program identifier; A corresponding target operation and maintenance program set is determined based on the corresponding operation and maintenance program identifier.

3. The method according to claim 1, characterized in that The step of obtaining the Q value in the Q table by the Q-learning algorithm includes: Initializing the Q value in the Q table; For each state value in the row of the Q table, randomly select an operation and maintenance program identifier; Run the operation and maintenance program corresponding to the operation and maintenance program identifier; Obtaining the system status and reward value of the system after the operation and maintenance program is run; Based on the system state, the reward value and the preset Q value calculation rule, the updated Q value of the state value and the operation and maintenance program identifier tuple is determined, and the step of randomly selecting an operation and maintenance program identifier is returned to execute until the iteration stop condition is met.

4. The method according to claim 3, characterized in that The Q value calculation rule is expressed by the following formula: in, is the learning rate, is the reward value, γ is the discount factor, For new status The maximum Q value under After running the operation and maintenance program, if the fault is eliminated and no new fault occurs, the reward value is a positive value. If the fault is not eliminated after running the operation and maintenance program, or a new fault occurs after running the operation and maintenance program, or the system performance cannot meet the normal operating conditions, the reward value Is a negative value.

5. The method according to any one of claims 1 to 2, characterized in that: The alarm information includes an alarm variable; The running of each operation and maintenance program in the target operation and maintenance program set includes: Substituting the alarm variables into each operation and maintenance program; The operation and maintenance programs are run in sequence according to the sequence and dependency relationship of each operation and maintenance program in the target operation and maintenance program set.

6. The method according to claim 1, characterized in that Also includes: Get the fault handling result; The reinforcement learning model is optimized using each operation and maintenance program identifier included in the target operation and maintenance program set and the fault handling result.

7. A fault self-healing device, characterized in that: The device comprises: The alarm information acquisition module is used to obtain the alarm information when a system failure is detected; A self-healing level judgment module, used to judge the self-healing level corresponding to the alarm information; A rule base processing module, configured to determine a target operation and maintenance program set in a preset rule database based on the alarm information when the self-healing level is the first level, wherein the first level refers to a level corresponding to a fault with a known fault cause and a preset self-healing rule; A reinforcement learning processing module, used for, when the self-healing level is the second level, using a pre-trained reinforcement learning model to process the alarm information and determine a target operation and maintenance program set, wherein the second level refers to a level corresponding to a fault with an unknown fault cause or a fault without a pre-set self-healing rule; wherein, using a pre-trained reinforcement learning model to process the alarm information and determine a target operation and maintenance program set comprises: determining a functional component having a fault from the alarm information; obtaining parameter values ​​of each indicator of the functional component having a fault; for each indicator, determining a state value of the indicator based on the parameter value of the indicator; querying a Q table based on the state value of the indicator to obtain a Q value corresponding to each operation and maintenance program identifier, wherein the rows of the Q table represent a combination of state values, the columns of the Q table represent each operation and maintenance program identifier, and the Q value corresponding to a tuple of a state value and an operation and maintenance program identifier in the Q table is obtained by a Q-learning algorithm; selecting a target operation and maintenance program identifier based on the Q value; and constructing a target operation and maintenance program set based on the target operation and maintenance program identifier; The program running module is used to run each operation and maintenance program in the target operation and maintenance program set to perform fault self-healing.

8. A fault self-healing device, characterized in that: The device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the fault self-healing method according to any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the fault self-healing method according to any one of claims 1 to 6 is implemented.

Images