Fault processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a fault processing method and device, a storage medium and electronic equipment, and relates to the technical field of data processing, and the method comprises the steps: obtaining an alarm log corresponding to alarm information in response to the detected first alarm information of a first system; second alarm information corresponding to a second system is obtained based on the alarm log, and the second system is an associated system of the first system; determining a fault processing strategy of the first system according to the second alarm information and the first alarm information; and finally, repairing the first system according to the fault processing strategy. According to the method, the second alarm information of the second system associated with the first system can be inferred based on the alarm log of the first system, and then the fault processing strategy of the first system is determined according to the first alarm information of the first system and the second alarm information of the second system, so that the first system is repaired; fault mining between associated systems is realized, and the accuracy of fault positioning is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a fault handling method, device, storage medium and electronic device. Background Art

[0002] Troubleshooting refers to taking a series of measures to solve the problem and restore the normal operation of the system after a system or equipment failure is detected.

[0003] At present, in the related technology, when a fault occurs in the current system, the fault handling measures corresponding to the fault are usually executed. However, in the process of repairing the current system, the associated systems of the current system are prone to failure, and the accuracy of fault location is low, resulting in a low success rate of fault handling, which in turn affects the efficiency of fault handling. Summary of the invention

[0004] In view of this, the present application provides a fault handling method, device, storage medium and electronic device, the main purpose of which is to improve the current related technology that during the process of repairing the current system, the associated system of the current system is prone to failure, the accuracy of fault location is low, resulting in a low success rate of fault handling, which in turn affects the technical problem of fault handling efficiency.

[0005] In a first aspect, the present application provides a fault handling method, comprising:

[0006] In response to detecting first alarm information of the first system, obtaining an alarm log corresponding to the first alarm information;

[0007] Based on the alarm log, obtaining second alarm information corresponding to a second system, where the second system is an associated system of the first system;

[0008] Determining a fault handling strategy for the first system according to the second alarm information and the first alarm information;

[0009] According to the fault handling strategy, the first system is repaired.

[0010] In a second aspect, the present application provides a fault handling device, comprising:

[0011] an acquisition module, configured to, in response to detecting first alarm information of a first system, acquire an alarm log corresponding to the first alarm information;

[0012] an acquisition module, configured to acquire second alarm information corresponding to a second system based on the alarm log, where the second system is an associated system of the first system;

[0013] a determination module, configured to determine a fault handling strategy for the first system according to the second alarm information and the first alarm information;

[0014] A repair module is configured to repair the first system according to the fault handling strategy.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the fault handling method described in the first aspect.

[0016] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the fault handling method described in the first aspect when executing the computer program.

[0017] In a fifth aspect, the present application provides a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the fault handling method described in the first aspect.

[0018] By means of the above technical scheme, the present application provides a fault handling method, device, storage medium and electronic device, which first responds to the detection of the first alarm information of the first system and obtains the alarm log corresponding to the alarm information; then obtains the second alarm information corresponding to the second system based on the alarm log, and the second system is the associated system of the first system; then determines the fault handling strategy of the first system according to the second alarm information and the first alarm information; finally, repairs the first system according to the fault handling strategy. The present application can infer the second alarm information of the second system associated with the first system based on the alarm log of the first system, and then determines the fault handling strategy of the first system according to the first alarm information of the first system and the second alarm information of the second system, so as to repair the first system, realize the fault mining between the associated systems, improve the accuracy of fault location, and thus improve the success rate and efficiency of fault handling.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram of a process flow of a fault handling method provided in an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of a process flow of a fault handling method provided in an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0026] Figure 5 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0027] Figure 6 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0028] Figure 7 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0029] Figure 8 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0030] Fig. 9 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0031] Fig.10 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0032] Fig.11 A schematic diagram of an example process provided by an embodiment of the present application is shown;

[0033] Fig.12 A schematic structural diagram of a fault handling device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0035] In order to improve the technical problem that in the process of repairing the current system, the associated system of the current system is prone to failure, the accuracy of fault location is low, resulting in a low success rate of fault handling, which in turn affects the efficiency of fault handling. This embodiment provides a fault handling method, such as Figure 1 As shown, the method includes:

[0036] Step 101: In response to detecting first alarm information of a first system, an alarm log corresponding to the first alarm information is obtained.

[0037] Among them, the first system may be the system where a fault occurs, and the first alarm information may be the alarm information of the first system; the alarm log may include timestamp, alarm type, alarm level, alarm content, alarm reason and other information. The alarm log provides a historical record of the system's operation. By analyzing the alarm log, historical alarm events can be traced and located, which helps to repair the current fault condition.

[0038] In some embodiments, after the first alarm information of the first system is detected, the alarm log corresponding to the first alarm information and the fault handling method can be viewed to repair the first alarm information.

[0039] Step 102: Based on the alarm log, obtain second alarm information corresponding to the second system, where the second system is an associated system of the first system.

[0040] In some embodiments, after detecting the first alarm information of the first system, the second system associated with the first system can be obtained, and then based on the alarm log of the first system, other possible faults of the first system and possible faults of the second system can be inferred, and the second alarm information that may appear in the second system can be obtained to repair the second system. In this way, when repairing the first system, the fault conditions of the associated systems can be considered, fault mining between the associated systems can be achieved, and the fault point can be located, thereby improving the accuracy of fault location, and the second alarm information associated with the first alarm information can be repaired, thereby improving the success rate and efficiency of fault handling. Among them, the second alarm information may include the alarm information of the second system.

[0041] Step 103: Determine a fault handling strategy for the first system according to the second alarm information and the first alarm information.

[0042] In some embodiments, the fault handling method corresponding to the second alarm information and the fault handling method corresponding to the first alarm information can be obtained, and the fault handling method corresponding to the second alarm information and the first alarm information respectively can be combined to determine the fault handling strategy of the first system, so that when the first system is repaired, the second system associated with the first system is repaired, thereby reducing the situation where the associated system fails, thereby improving the success rate of fault repair of the first system.

[0043] Step 104: Repair the first system according to the fault handling strategy.

[0044] Exemplarily, a task flow may be generated according to a fault handling strategy, and by executing the task flow, the second alarm information and the first alarm information are repaired in sequence, and finally the first system is repaired, thereby improving the repair success rate.

[0045] Compared with the current existing technology, this embodiment can first respond to the detection of the first alarm information of the first system and obtain the alarm log corresponding to the alarm information; then based on the alarm log, obtain the second alarm information corresponding to the second system, the second system is the associated system of the first system; then determine the fault handling strategy of the first system according to the second alarm information and the first alarm information; finally, repair the first system according to the fault handling strategy. In this way, the second alarm information of the second system associated with the first system can be inferred based on the alarm log of the first system, and then the fault handling strategy of the first system can be determined according to the first alarm information of the first system and the second alarm information of the second system to repair the first system, thereby realizing fault mining between associated systems, improving the accuracy of fault location, and thus improving the success rate and efficiency of fault handling.

[0046] In order to further illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 2 The specific method shown includes:

[0047] Step 201: In response to detecting first alarm information of a first system, an alarm log corresponding to the first alarm information is obtained.

[0048] Exemplarily, a fault diagnosis method and system based on improved federated learning may be provided, such as Figure 3 As shown, a schematic diagram of the overall framework is shown. The purpose of the system is to be able to analyze the alarm log corresponding to the first alarm information by calling the fault diagnosis model when the system detects an alarm message, so as to infer the cause of the alarm and generate a corresponding repair plan. Specifically, during the fault diagnosis process, when the system detects the first alarm message, it will immediately trigger the operation of calling the fault diagnosis model. The fault diagnosis model will analyze the alarm log information corresponding to the first alarm message, and use the improved federated learning method to infer the fault and generate a self-healing plan.

[0049] For example, in order to optimize the fault diagnosis model, the system can store key data such as alarm information and operation process in the server. By analyzing and mining these data, the accuracy and performance of the diagnosis model can be continuously optimized, thereby improving the effect and stability of the entire fault diagnosis system.

[0050] Step 202: Obtain a joint model corresponding to a preset time series, where the joint model includes a joint model corresponding to the second system and a joint model corresponding to the first system, and the second system is an associated system of the first system.

[0051] Optionally, step 202 may specifically include: dividing the log information of the second system and the log information of the first system according to a preset time series; generating a joint model corresponding to the preset time series based on the divided log information of the second system and the log information of the first system.

[0052] In some embodiments, the log information corresponding to the first system and the second system (such as service logs) can first be divided into multiple segments according to the time sequence corresponding to the preset time series. Through this cutting method, the amount of data for establishing a joint model (federal model) can be effectively reduced, thereby improving the efficiency and accuracy of the model.

[0053] Accordingly, a federal model can be established based on the cut log information. The federal model can be used to cover the intrinsic correlation between multiple businesses within a specific time period. Each federal model can represent the characteristics of system failures in the corresponding time period, reflecting the operating status of each business in the time period and possible failure conditions, so as to improve the fault repair effect of the current alarm.

[0054] Step 203: Fuse the joint model corresponding to the preset time series to generate a fault diagnosis model.

[0055] In some examples, aggregation algorithms such as average aggregation can be used to fuse multiple joint models, and labeling can be used to manually remove abnormal data to enhance the accuracy of the fault diagnosis model. Secondly, an error tolerance interval can be introduced to prevent model overfitting.

[0056] Optionally, step 203 may specifically include: based on a preset timing algorithm, fusing the joint model to generate a fault diagnosis model.

[0057] The preset time series algorithm may be an AdaBoost algorithm based on time series improvement. Exemplarily, after obtaining the joint model corresponding to the preset time series, the AdaBoost algorithm based on time series improvement may be used to fuse the joint model to obtain a fault diagnosis model, so as to effectively integrate the joint models of various services in different time periods. The system is divided according to the time series, and multiple sub-models are generated to achieve model aggregation based on the time series relationship, so as to obtain a more comprehensive and accurate system fault diagnosis result.

[0058] For example, Figure 4As shown, multiple joint models can be generated based on the service log, for example: joint model 1, joint model 2, ..., joint model N, etc., and then the multiple joint models are fused based on the time-series improved AdaBoost algorithm to generate a fault diagnosis model (fault model) to realize alarm detection. The fault diagnosis model modeling method has the characteristics of rapid modeling of massive data systems and optimization of existing models according to incremental data, which can effectively improve the modeling speed and model accuracy of the fault diagnosis model.

[0059] Optionally, based on a preset timing algorithm, the joint model is integrated to generate a fault diagnosis model, which may specifically include: determining a sample log from the log information of the second system and the log information of the first system; determining the weight of the joint model according to the time and occurrence frequency corresponding to the sample log; and combining the joint model according to the weight to generate a fault diagnosis model.

[0060] For example, Figure 5 As shown, a schematic diagram of the algorithm is shown, and the specific process is as follows:

[0061] First, the service logs can be sampled according to the time sequence, and sample logs can be determined from the log information of the second system and the log information of the first system to generate a training sample set D. Each training sample D i It can be defined as:

[0062] D i =(X i , Y i , T i )

[0063] Among them, X i It can represent the log information of the i-th sample, which may include error CODE, error information INFO and fault handling solution. i It can indicate whether it is an alarm message (for example: -1 means no, 1 means yes), T i It can represent the time corresponding to the sample.

[0064] At this time, the initial weight distribution D1(i) of the training sample set can be expressed as:

[0065] D1(i)=(w1,w2,w3,...w i ,...w N )

[0066] Assume that the current time is T now According to the characteristics of the fault model, the data samples closer to the current time and the data samples with higher occurrence frequency should account for a higher proportion in the weight distribution, so sample D i The weight W i It can be defined as:

[0067]

[0068] In the formula, count(X i ) can represent X i The number of occurrences, N can represent the number of samples in the sample set. By initializing the weights in this way, the impact of abnormal data on the system can be effectively reduced, the modeling speed of the fault model can be accelerated, and the modeling accuracy can be guaranteed.

[0069] Specifically, sample D i The weight W i Normalized, W i The normalized expression can be expressed as:

[0070]

[0071] For example, assuming the joint model is h, then the kth joint model h k In the distribution D k The error rate e k As shown below:

[0072]

[0073] When h k When it is the joint model with the lowest error rate, this model is regarded as the basic model H k At this time h k The weight α k The weights in the final fault model are as follows:

[0074]

[0075] When the model starts iterative training, the weight distribution D of the training samples k+1 The update expression is as follows:

[0076]

[0077] In the formula, y1H k (x i ) can indicate whether the sample is correctly classified. For example, when the sample is correctly classified, y i H k (x i )=1, then the weight distribution D k+1 can be simplified to:

[0078]

[0079] Correspondingly, when the sample classification is incorrect, y i H k (x i)=-1, then the weight distribution D k+1 can be simplified to:

[0080]

[0081] Finally, the joint models are combined according to the weight αk, and the final fault model H is obtained through the sign function. final , as shown below:

[0082]

[0083] For example, when the fault model H final When an upgrade is required, just add a new joint model H N+1 And access the corresponding log information in time sequence. In the specific upgrade process, you can use the generated joint model H1~joint model H N+1 Construct a new fault model H new , or directly use the old fault model H final and the joint model H N+1 Directly merge and generate a new fault model H new Both methods can realize the new fault model H new Compared with using full data to generate fault models, the generation time of the model is greatly shortened.

[0084] In this way, the fault sub-model corresponding to each time period of the system can be trained based on federated learning; then the AdaBoost algorithm is improved based on the time series, and the sub-models are integrated into a complete fault model, which ensures data security and supports massive data modeling and model hot upgrades.

[0085] Step 204: Infer the second alarm information corresponding to the second system by using the fault diagnosis model and the alarm log.

[0086] Optionally, the fault diagnosis model and the alarm log may be used to infer the second alarm information corresponding to the second system and other alarm information of the first system.

[0087] In some embodiments, after the fault detection system detects the first alarm information, the first system will generate a complete fault repair strategy based on the generated fault diagnosis model, the error CODE code in the alarm log corresponding to the first alarm information, the error information INFO and the fault handling plan, and the inherent connection between the faults of the associated systems, and infer the second alarm information corresponding to the second system, as well as other alarm information of the first system. The first alarm information may include the error code and error information corresponding to the first system, and the second alarm information may include the error code and error information corresponding to the second system.

[0088] For example, Figure 6 As shown, the second system that the first system (current system) depends on may include: associated system 1, associated system 2 and associated system 3, which are used to implement system functions. Among them, associated system 1 and associated system 2 are indirectly related to associated system 4, and associated system 3 and associated system 4 interact with the underlying system at the same time. When the current system detects error code CODE1 and error information INFO1, the systems can be inferred based on the fault model, and the error code code1 and error information info1 of the current system and the error code code2, error code code3 and error information info2 and error information info3 of the associated system can be indirectly obtained, thereby obtaining other alarm information that may be triggered by the first alarm information of the current system, which helps to improve the success rate of fault repair.

[0089] Step 205: Determine a fault handling strategy for the first system according to the second alarm information and the first alarm information.

[0090] Optionally, step 205 may specifically include: splicing the fault handling method of the first alarm information and the fault handling method corresponding to the second alarm information to generate a self-healing strategy for the first system; and determining the fault handling strategy for the first system from the self-healing strategy according to the self-healing success rate of the self-healing strategy.

[0091] Exemplarily, the fault diagnosis model is used to infer the second alarm information and other alarm information of the first system, and then integrated with the first alarm information to obtain complete error information, so as to achieve a complete analysis of the fault point where the first system fault occurred. Then, the fault handling method corresponding to each system alarm information can be spliced, and multiple fault self-healing solutions can be generated based on the global situation. The fault point in the entire call chain can be determined at the first time, and the corresponding optimal self-healing solution can be generated according to the combination of fault causes, which ensures the success rate of the self-healing solution. The expression of the fault self-healing solution can be:

[0092] M1=M a +M c +M e M2=M a +M c +M f ,....,M n =M b +M d +M f

[0094] In the formula, M1, M2, M n The first, second, and Nth fault self-healing schemes after splicing can be represented respectively, and a preset number (such as 3) of self-healing schemes with the highest previous self-healing success rates can be selected as the fault handling strategy for the first system, and self-healing can be attempted to improve the repair success rate.

[0095] Step 206: Repair the first system according to the fault handling strategy.

[0096] Optionally, step 206 may specifically include: generating a task flow corresponding to the fault handling strategy, and performing high-risk operation detection on the task flow; if it is detected that there is no high-risk operation in the task flow, repairing the first system according to the task flow; if it is detected that there is a high-risk operation in the task flow, manually approving the task flow.

[0097] For example, the specific process of attempting self-healing according to the fault handling strategy is as follows: Figure 7 As shown in the figure, first, generate an executable task flow corresponding to the fault handling strategy to complete the fault repair. Secondly, detect the script information in the task flow to identify whether high-risk operations occur: when a high-risk operation is detected in the task flow, the relevant person in charge can be notified for approval, and then the task flow can be executed to try self-healing. This ensures that the necessary manual review and authorization are obtained before executing high-risk operations, thereby reducing potential risks; when it is detected that there is no high-risk operation in the task flow, self-healing can be directly attempted. If the self-healing operation fails, the system will immediately notify the relevant person in charge to repair the fault to ensure that the fault can be resolved in a timely manner.

[0098] Specifically, Figure 8 The figure shows the process of generating task flows for the self-healing solution. The design and implementation of task flows can help the system respond quickly and automatically repair when a fault occurs, reducing manual intervention and delays, and improving the reliability and robustness of the system. First, according to the content of the self-healing solution, it can be cut into operation steps, and each operation step can be converted into a corresponding script. These scripts contain specific operation instructions and logic for automatic repair of faults. Each script is processed for a specific problem or fault to ensure the accuracy and effectiveness of self-healing.

[0099] Next, these scripts are arranged in sequence and connected in series to form an executable task flow. Each script in the task flow is executed in sequence according to a pre-defined order, ensuring that each operation step of the self-healing solution can be performed in the correct order, thereby achieving automatic repair of the fault.

[0100] Finally, by executing this task flow, the fault can be self-healed. The system will execute each script in sequence according to the instructions of the task flow, and make corresponding judgments and processing according to the execution results of each operation step. Through the execution of the task flow, the system can automatically diagnose and repair faults, improving the efficiency and accuracy of fault handling.

[0101] Accordingly, before executing the task flow, a high-risk detection can be performed on the task flow. The detection process involves command vault detection, execution account detection, and impact assessment. The specific process of high-risk detection is as follows: Fig. 9 As shown, first, the script content of each script in the task flow is verified through the instruction vault. The instruction vault is maintained by relevant operation and maintenance personnel, which contains high-risk instructions used in daily work. Instructions are verified using a variety of matching methods including but not limited to complete matching, fuzzy matching, regular expression matching, wildcard matching, and diversity matching.

[0102] Secondly, after the script content is verified, the execution account of the script is determined for permissions. According to the results of the command vault detection, the commands can be divided into low-risk, medium-risk and high-risk commands. Low-risk commands are directly released, and medium-risk and high-risk commands are determined by blacklist and whitelist. If the execution account is on the whitelist, the command is directly released, and if it is on the blacklist, the command is determined to be high-risk and needs to be approved by relevant operation and maintenance personnel before it can be executed.

[0103] Finally, it is necessary to comprehensively consider the nodes, businesses, and scheduled execution time associated with the script to assess the potential scope of impact and determine whether an audit is required. In cases where the scope of impact is wide, the situations that require review include but are not limited to:

[0104] 1. The script affects a large number of nodes, and the related nodes carry many businesses, and there is a lack of sufficient isolation between these businesses.

[0105] 2. The business that the script operates is critical or has a large number of users relying on it.

[0106] 3. The script is scheduled to be executed during the peak business period, or its execution may have a significant impact on ongoing business.

[0107] Compared with the current existing technology, this embodiment can obtain a joint model corresponding to a preset time series, ensure data security, support massive data modeling and model hot upgrade, generate a fault diagnosis model by integrating the joint model corresponding to the preset time series, and use the fault diagnosis model and the alarm log to infer the second alarm information corresponding to the second system, deeply mine the inherent fault connection between multiple related systems, infer the fault situation of the second system through the fault performance of the first system, and generate a global fault handling strategy from the perspective of the call chain to ensure the success rate of the fault handling strategy; and can determine the fault handling strategy of the first system from the self-healing strategy according to the self-healing success rate of the self-healing strategy, realize system fault self-inspection, and try self-healing according to the self-healing strategy, reduce operation and maintenance costs, enhance operation and maintenance processing capabilities, shorten fault response time, complete the self-healing process by arranging task flows, and access high-risk operation audits, and manually approve task flows with high-risk operations, thereby enhancing system security and improving user experience.

[0108] As a possible implementation, Fig.10 As shown in the figure, a fault model generation method based on improved federated learning is provided, which aims to complete the generation of the federated model through federated learning, and then complete the generation of the fault model through the AdaBoost algorithm based on timing improvement. The specific process is as follows:

[0109] First, according to the time series T0~T n , divide the alarm logs and fault handling methods in each system into n time periods for data splitting, so as to more effectively process large amounts of data and improve the efficiency of subsequent model generation. The key to this step is to reasonably segment the data in time, so as to ensure that the amount of data in each time period is moderate, so as to retain important information without being too large.

[0110] Secondly, in each time period, each system generates its own business model based on the alarm log and fault handling method of the business. These business models are based on the combination of system-specific fault error CODE codes, fault error information, and fault handling methods. By associating fault error CODE codes and error information, it can be ensured that each fault error corresponds to the correct handling method. In addition, in order to further improve processing efficiency, the handling methods of the same fault error are sorted, and the methods that appear more frequently and are earlier in time are given priority.

[0111] Next, in order to ensure data security, the related systems do not interact directly with each other, but generate a joint model by transmitting data to the terminal. This method can effectively prevent data leakage or tampering, and also provide more data sources for model generation.

[0112] Finally, according to the time series, the original data used in the establishment of some joint models are extracted as training samples. In this step, the AdaBoost algorithm based on time series improvement can be used to train the time series from T0 to T n The n joint models in this time period are aggregated to generate a fused fault model. This model combines the data characteristics and processing methods of different systems in each time period, and has higher prediction accuracy and practicality.

[0113] Correspondingly, such as Fig.11 As shown, a fault diagnosis system architecture diagram based on improved federated learning is provided, which may include a fault diagnosis module and an underlying service module. Through the service logs collected by the log collection system, fault diagnosis of the currently running system can be achieved, and a self-healing plan can be generated to attempt self-healing.

[0114] Among them, the fault diagnosis module constitutes the core of system diagnosis, which may include but is not limited to: fault model, self-healing strategy and task flow. Specifically, the fault model can be constructed based on the data obtained from the log collection system and the preset timing algorithm, which can be used to reflect the internal interconnection relationship between the current system and other systems. By analyzing the current system state, the operating status of the associated system can be inferred; the self-healing strategy can be generated based on the insight of the fault model, not only proposing local optimization solutions for each system state, but also combining these solutions to form a global optimization strategy; the task flow can be formulated according to the self-healing strategy, using the native operation interface of the underlying service, and converted into a series of executable tasks. These tasks can be executed serially or in parallel, and can be assigned to the corresponding responsible persons and business operation and maintenance personnel. In this way, the fault diagnosis module can not only accurately identify the problem, but also effectively guide the solution of the problem to ensure the efficient operation of the system.

[0115] Accordingly, the underlying service module can be used to ensure the robustness and security of the system, which may include but is not limited to: high-risk detection, message notification, host service and other modules. Specifically, the high-risk detection module can review all scripts in the task flow, with a wide range of detection, including but not limited to instruction security assessment, execution account authority verification, and analysis of the impact range of operations. When a high-risk item is detected, the system can activate the message notification module to ensure that the task flow is strictly approved before execution; the message notification module can ensure that the relevant business person in charge or operation and maintenance personnel receive the approval request in a timely manner through various means, such as email, voice message, SMS or direct call, so as to review the task flow in a timely manner. When the task flow (task flow) is approved, the system can try to execute the process; the host service module can play a key role by providing a native operation interface to implement system fault repair and other necessary maintenance operations. Among them, the host service may include but is not limited to functions such as system monitoring, resource management and automated deployment, ensuring that the task flow is not only safe, but also can be executed efficiently after obtaining the necessary approval, thereby supporting the continuity and reliability of the system.

[0116] In some examples, the method of this embodiment can be applied to complex scenarios where multiple systems call each other, such as modeling and analysis of systems and platforms that generate massive amounts of data during operation, effectively reducing the operation and maintenance costs of the system, quickly locating the cause of system failures and attempting self-healing while ensuring the data security of each system module, significantly shortening the duration of the impact of the failure and ensuring the customer's user experience; at the same time, it has significant advantages in processing massive data analysis and can greatly shorten the modeling time.

[0117] Further, as Figure 1 and Figure 2The specific implementation of the method shown in this embodiment provides a fault handling device, such as Fig.12 As shown, the device includes: an acquisition module 31, a determination module 32, and a repair module 33.

[0118] The acquisition module 31 is configured to acquire an alarm log corresponding to the first alarm information in response to detecting the first alarm information of the first system;

[0119] The acquisition module 31 is configured to acquire second alarm information corresponding to the second system based on the alarm log, where the second system is an associated system of the first system;

[0120] A determination module 32, configured to determine a fault handling strategy for the first system according to the second alarm information and the first alarm information;

[0121] The repair module 33 is configured to repair the first system according to the fault handling strategy.

[0122] In some examples of this embodiment, the acquisition module 31 is specifically configured to obtain a joint model corresponding to a preset time series, the joint model including a joint model corresponding to the second system and a joint model corresponding to the first system; fuse the joint models corresponding to the preset time series to generate a fault diagnosis model; and use the fault diagnosis model and the alarm log to infer the second alarm information corresponding to the second system.

[0123] In some examples of this embodiment, the acquisition module 31 is specifically configured to divide the log information of the second system and the log information of the first system according to a preset time series; based on the divided log information of the second system and the log information of the first system, a joint model corresponding to the preset time series is generated.

[0124] In some examples of this embodiment, the acquisition module 31 is specifically configured to generate a fault diagnosis model by fusing the joint model based on a preset timing algorithm.

[0125] In some examples of this embodiment, the acquisition module 31 is specifically configured to determine the sample log from the log information of the second system and the log information of the first system; determine the weight of the joint model according to the time and occurrence frequency corresponding to the sample log; and combine the joint model according to the weight to generate a fault diagnosis model.

[0126] In some examples of this embodiment, the determination module 32 is specifically configured to splice the fault handling method of the first alarm information and the fault handling method corresponding to the second alarm information to generate a self-healing strategy for the first system; and determine the fault handling strategy of the first system from the self-healing strategy according to the self-healing success rate of the self-healing strategy.

[0127] In some examples of this embodiment, the repair module 33 is specifically configured to generate a task flow corresponding to the fault handling strategy and perform high-risk operation detection on the task flow; if it is detected that there is no high-risk operation in the task flow, the first system is repaired according to the task flow; if it is detected that there is a high-risk operation in the task flow, the task flow is manually approved.

[0128] It should be noted that for other corresponding descriptions of the functional units involved in the fault handling device provided in this embodiment, reference can be made to Figure 1 and Figure 2 The corresponding description in will not be repeated here.

[0129] Based on the above Figure 1 and Figure 2 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 The method shown.

[0130] Based on the above Figure 1 and Figure 2 The method shown in the embodiment also provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 The method shown.

[0131] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the method of the implementation scenario of the present application.

[0132] Based on the above Figure 1 and Figure 2 The method shown, and Fig.12 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 2 The method shown.

[0133] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0134] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0135] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0136] Through the description of the above implementation methods, the technical personnel in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the technical solution of this embodiment, the joint model corresponding to the preset time series can be obtained, data security is guaranteed, massive data modeling and model hot upgrade are supported, and the fault diagnosis model is generated by integrating the joint model corresponding to the preset time series. The second alarm information corresponding to the second system is inferred by using the fault diagnosis model and the alarm log, and the inherent fault connection between multiple related systems is deeply excavated. The fault situation of the second system is inferred by the fault performance of the first system, and the global fault handling strategy is generated from the perspective of the call chain to ensure the success rate of the fault handling strategy; and the fault handling strategy of the first system can be determined from the self-healing strategy according to the self-healing success rate of the self-healing strategy, and the fault self-check of the system is realized, and self-healing is attempted according to the self-healing strategy, which reduces the operation and maintenance cost, enhances the operation and maintenance processing capability, shortens the fault response time, completes the self-healing process by arranging the task flow, and accesses the high-risk operation audit, and manually approves the task flow with high-risk operations, which enhances the system security and improves the user experience.

[0137] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0138] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

Claims

1. A fault handling method, characterized in that: include: In response to detecting first alarm information of the first system, obtaining an alarm log corresponding to the first alarm information; Based on the alarm log, obtaining second alarm information corresponding to a second system, where the second system is an associated system of the first system; Determining a fault handling strategy for the first system according to the second alarm information and the first alarm information; According to the fault handling strategy, the first system is repaired.

2. The method according to claim 1, characterized in that The acquiring, based on the alarm log, second alarm information corresponding to the second system includes: Acquire a joint model corresponding to a preset time series, wherein the joint model includes a joint model corresponding to the second system and a joint model corresponding to the first system; Fusion of the joint model corresponding to the preset time series to generate a fault diagnosis model; The fault diagnosis model and the alarm log are used to infer second alarm information corresponding to the second system.

3. The method according to claim 2, characterized in that The obtaining of the joint model corresponding to the preset time series includes: According to the preset time sequence, the log information of the second system and the log information of the first system are divided; Based on the divided log information of the second system and the log information of the first system, a joint model corresponding to the preset time series is generated.

4. The method according to claim 3, characterized in that The fusing of the joint model corresponding to the preset time series to generate a fault diagnosis model includes: Based on a preset timing algorithm, the joint model is integrated to generate the fault diagnosis model.

5. The method according to claim 4, characterized in that The method of fusing the joint model based on a preset timing algorithm to generate the fault diagnosis model includes: Determine a sample log from the log information of the second system and the log information of the first system; Determining the weight of the joint model according to the time and occurrence frequency corresponding to the sample log; The joint models are combined according to the weights to generate the fault diagnosis model.

6. The method according to claim 1, characterized in that The determining, according to the second alarm information and the first alarm information, a fault handling strategy of the first system includes: Combine the fault handling method of the first alarm information and the fault handling method corresponding to the second alarm information to generate a self-healing strategy for the first system; According to the self-healing success rate of the self-healing strategy, a fault handling strategy of the first system is determined from the self-healing strategy.

7. The method according to claim 1, characterized in that The repairing the first system according to the fault handling strategy includes: Generate a task flow corresponding to the fault handling strategy, and perform high-risk operation detection on the task flow; If it is detected that the task flow does not have the high-risk operation, repairing the first system according to the task flow; If it is detected that the task flow has the high-risk operation, the task flow is manually approved.

8. A fault handling device, characterized in that: include: an acquisition module, configured to, in response to detecting first alarm information of a first system, acquire an alarm log corresponding to the first alarm information; an acquisition module, configured to acquire second alarm information corresponding to a second system based on the alarm log, where the second system is an associated system of the first system; a determination module, configured to determine a fault handling strategy for the first system according to the second alarm information and the first alarm information; A repair module is configured to repair the first system according to the fault handling strategy.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product having a computer program stored thereon, characterized in that: When the computer program product is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images