System fault processing method and device, computer equipment and storage medium

Abstract

The invention discloses a system fault processing method and device, computer equipment and a storage medium, and is suitable for a fault processing system connected with a service system, and the fault processing system comprises an operation and maintenance monitoring platform and an operation and maintenance change platform. Log data of historical system faults are obtained from the operation and maintenance monitoring platform, first log data are formed, and a first log item set is obtained by calculating the frequent item set. And log data of a fault recovery operation matched with the historical system fault is obtained from the operation and maintenance change platform to form second log data, and a second log item set is obtained by calculating the frequent item set. And based on the two log item sets, constructing a training data set and training a fault prediction model. Obtaining to-be-processed log data, calculating a frequent item set of the to-be-processed log data, inputting the to-be-processed log item set into the fault prediction model, generating a fault prediction result, recognizing a system fault, and automatically repairing the system fault. According to the invention, the automatic fault processing capability and the fault recovery efficiency of the service system are improved.

Description

Technical Field

[0001] The present application belongs to the technical field of computer system fault processing, and specifically relates to a method, device, computer equipment and storage medium for processing system faults. Background Art

[0002] System fault recovery is an important concept in the field of engineering technology, especially in aeronautical engineering and computer technology. It aims to quickly and effectively restore the normal operation of the system after a system failure. System fault recovery uses a series of strategies, methods and techniques to isolate the fault and select another facility or method to return the system to the task point before the failure, so that the system can continue to work normally.

[0003] At present, when troubleshooting, the fault monitoring system usually generates an alarm to notify the operation and maintenance personnel, who need to log in manually to conduct analysis and operation processing, such as logging in manually to the system, checking logs one by one, analyzing data, and performing corresponding fault handling operations. This process is not only time-consuming and labor-intensive, but also highly dependent on the immediate response ability and experience judgment of the operation and maintenance personnel. It is easy to make misjudgments or delays in tense or complex fault situations, which is not conducive to the rapid recovery of system faults. In particular, for system faults during non-working hours, the operation and maintenance personnel need to use remote VPN to handle the faults, which not only increases the complexity and uncertainty of the operation, but also may prolong the impact of the fault due to network delays, VPN connection stability and other issues, seriously affecting business continuity and customer satisfaction. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for handling system failures, so as to solve the technical problems that existing fault handling solutions are time-consuming and labor-intensive, highly dependent on the immediate response capabilities and experience judgment of operation and maintenance personnel, prone to misjudgment or delays, and affect business continuity and customer satisfaction.

[0005] In order to solve the above technical problems, the present application provides a method for handling system failures, which adopts the following technical solutions:

[0006] A method for handling system failures, the method is applied to a failure handling system, the failure handling system is connected to a business system, the failure handling system includes an operation and maintenance monitoring platform and an operation and maintenance change platform, and the method includes:

[0007] Obtaining log data of historical system failures from the operation and maintenance monitoring platform to obtain first log data;

[0008] Calculate the frequent itemsets of the first log data to obtain the first log itemsets;

[0009] Obtaining log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match historical system faults;

[0010] Calculate the frequent itemsets of the second log data to obtain the second log itemsets;

[0011] Based on the first log item set and the second log item set, a training data set is constructed, and a fault prediction model is trained using the training data set;

[0012] Obtain the log data to be processed in real time, calculate the frequent item sets of the log data to be processed, and obtain the log item sets to be processed;

[0013] Input the log item set to be processed into the fault prediction model to obtain the fault prediction result;

[0014] Based on the fault prediction results, system faults of the business system are identified and repaired.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a system failure processing device, which adopts the following technical solution:

[0016] A system fault processing device, the processing device is used to run a fault processing system, the fault processing system is connected to a business system, the fault processing system includes an operation and maintenance monitoring platform and an operation and maintenance change platform, and the processing device includes:

[0017] A first log acquisition module is used to acquire log data of historical system failures from the operation and maintenance monitoring platform to obtain first log data;

[0018] A first frequent item set module, used to calculate the frequent item sets of the first log data to obtain the first log item set;

[0019] A second log acquisition module is used to acquire log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match historical system faults;

[0020] A second frequent item set module, used to calculate the frequent item sets of the second log data to obtain the second log item sets;

[0021] A prediction model training module, used to construct a training data set based on the first log item set and the second log item set, and use the training data set to train a fault prediction model;

[0022] The third frequent item set module is used to obtain the log data to be processed in real time, calculate the frequent item sets of the log data to be processed, and obtain the log item sets to be processed;

[0023] The system fault prediction module is used to input the log item set to be processed into the fault prediction model to obtain the fault prediction result;

[0024] The system fault repair module is used to identify system faults of the business system based on the fault prediction results and perform fault repair on the system faults of the business system.

[0025] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0026] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any one of the above-mentioned system failure processing methods when executing the computer-readable instructions.

[0027] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0028] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for processing a system failure as described in any one of the above.

[0029] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0030] The present application discloses a method, device, computer equipment and storage medium for processing system failures, which are applicable to a failure processing system connected to a business system, wherein the failure processing system includes an operation and maintenance monitoring platform and an operation and maintenance change platform. Log data of historical system failures are obtained from the operation and maintenance monitoring platform to form first log data, and a first log item set is obtained by calculating frequent item sets. Log data of fault recovery operations matching historical system failures are obtained from the operation and maintenance change platform to form second log data, and a second log item set is obtained by calculating frequent item sets. Based on these two log item sets, a training data set is constructed and a failure prediction model is trained. The log data to be processed is obtained and its frequent item sets are calculated, and the log item sets to be processed are input into the failure prediction model to generate a failure prediction result, identify system failures, and automatically repair system failures. The present application obtains the log data of system failures and the log data of failure recovery operations, calculates their frequent item sets, and then learns the features of the frequent item sets through a prediction model, so that the prediction model learns the correlation between the cause of the failure and the failure recovery operation, thereby obtaining the automated processing capability of predicting system failures and predicting failure recovery operations, significantly reducing the need for manual intervention, shortening the failure processing time, and improving the automated processing capability of business system failures and the failure recovery efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0033] Figure 2 A flow chart showing an embodiment of a method for processing a system failure according to the present application;

[0034] Figure 3 Shown above Figure 2 A flowchart of an embodiment of step S202;

[0035] Figure 4 Shown above Figure 2 A flowchart of an embodiment of step S204;

[0036] Figure 5 Shown above Figure 2 A flowchart of an embodiment of step S205;

[0037] Figure 6 A schematic structural diagram of an embodiment of a device for processing system failure according to the present application is shown;

[0038] Figure 7 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0042] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0043] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0044] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0045] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0046] It should be noted that the method for processing system failures provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the device for processing system failures is generally provided in the server / terminal device.

[0047] It should be understood that Figure 1 The numbers of terminal devices, networks and servers in the embodiment are merely illustrative. The above system may have any number of terminal devices, networks and servers according to implementation requirements.

[0048] The present application discloses a method for handling system failures, which is applied to a fault handling system, which is connected to a business system. The fault handling system includes an operation and maintenance monitoring platform and an operation and maintenance change platform. The operation and maintenance monitoring platform is mainly responsible for real-time monitoring of the operating status of the business system, collecting log data and alarm information generated during the operation of the system, and plays a role in monitoring the system status, identifying potential failures, and providing alarms. The operation and maintenance change platform is used to record and manage change operations implemented during system maintenance, including system recovery, fault repair and other operation steps, and saves recovery operation logs that match historical system failures. By combining the log data of the monitoring and change platforms for analysis and processing, fault monitoring, analysis, prediction and repair can be achieved, the duration of failures can be reduced, and the efficiency of fault handling can be improved.

[0049] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for handling a system failure according to the present application. The method for handling a system failure disclosed in the present application specifically comprises the following steps:

[0050] S201, obtaining log data of historical system failures from an operation and maintenance monitoring platform to obtain first log data.

[0051] Specifically, the operation and maintenance monitoring platform is the core module responsible for monitoring the operating status of the business system. The platform will record various log data during the system operation, including alarm information, error information, performance anomalies, etc., especially log data related to historical system failures, such as high server load, machine hardware anomalies, request network timeout alarms, etc. By collecting this data, the system can obtain comprehensive information related to historical failures, covering the time of occurrence of the failure, the scope of impact, system status, specific failure type, etc.

[0052] S202: Calculate frequent itemsets of first log data to obtain first log itemsets.

[0053] Specifically, calculating frequent item sets is a data mining technique that aims to find log items that often appear together from a large amount of log data. First, by analyzing historical system fault log data, we can find frequent combinations of specific log items (such as error codes, alarm types, etc.) when faults occur. These combinations reflect common fault modes and help the system quickly identify recurring fault features. Second, frequent item sets represent the correlation between fault events. By calculating frequent item sets, the system can understand how faults are triggered under certain specific conditions, thereby inferring the potential causes of different faults, which is helpful for fault prediction and location.

[0054] In this step, the system mines frequent itemsets for the first log data obtained from the operation and maintenance monitoring platform, and analyzes which log items frequently appear simultaneously in historical failures. These frequent itemsets reflect the potential associations between different system failures, helping the system identify common failure modes. By extracting frequent itemsets, the system can summarize the combination of conditions that may trigger failures. The calculation of frequent itemsets is usually based on association rule algorithms, such as April or FP-growth algorithms, to ensure that valuable information can be efficiently extracted from massive log data.

[0055] S203, obtaining log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match historical system faults.

[0056] Specifically, the operation and maintenance change platform is responsible for recording the change operation logs executed during the system maintenance process, including recovery operations for system failures, such as restarting abnormal application instances or stopping processes to isolate abnormal application instances, modifying IP configurations to switch network links, etc. These recovery operations are closely related to the specific steps performed by operation and maintenance personnel when handling historical failures. By obtaining historical failure recovery operation log data from the change platform, the system can trace back to how the previously identified failures were resolved. The second log data is the operation steps, execution time, operation effects, and other information related to failure recovery.

[0057] S204: Calculate frequent itemsets of the second log data to obtain second log itemsets.

[0058] Specifically, by analyzing historical fault recovery operation log data, the system can find the combination of recovery operation steps that are frequently used when handling different faults. These frequent item sets represent common repair solutions and reflect which operation steps are most effective in historical fault handling. Secondly, the second log item set provides the correlation between the repair operation and the fault mode. By combining the second log item set with the first log item set (fault trigger mode), the system can train a more intelligent fault prediction model, thereby more accurately predicting faults and automatically matching corresponding repair solutions.

[0059] In this step, the system analyzes the historical fault recovery operation log data obtained from the operation and maintenance change platform to mine the frequently occurring recovery operation step combinations, namely the second log item set. By calculating the frequent item sets, the system can identify the recovery operation combinations that operation and maintenance personnel often take when dealing with similar faults. These frequent item sets reflect the correlation between fault recovery operations and specific fault scenarios. For example, if a certain type of fault triggers the same recovery operation steps multiple times, the system can solidify this correlation into a fault recovery rule to ensure that the corresponding repair operation is automatically triggered when similar faults occur in the future.

[0060] For example, the following frequent itemsets are found by the frequent itemset algorithm:

[0061] {CPU usage > 90%, memory usage > 80%, operation restart process changed}: This indicates that the system resources are insufficient and need to be recovered by restarting the application instance.

[0062] {Error code 500, application log connect time out, operation switching network link change}: This indicates that there is a network connection problem on the server side and the primary and backup network links need to be switched for recovery.

[0063] S205 , constructing a training data set based on the first log item set and the second log item set, and using the training data set to train a fault prediction model.

[0064] Specifically, the first log item set represents the frequent item set of historical system failures, and the second log item set corresponds to the frequent item set of failure recovery operations. By combining these two types of item sets, the system constructs a training data set for training the failure prediction model. The construction of the training data set takes into account the triggering conditions of the failure and its corresponding recovery measures to ensure that the model can learn the association between failure and repair. By using machine learning algorithms (such as decision trees, random forests, or deep learning models), the system learns the best recovery solutions under different failure modes in the model.

[0065] S206, obtaining the log data to be processed in real time, calculating the frequent itemsets of the log data to be processed, and obtaining the log item sets to be processed.

[0066] Specifically, after completing the training of the fault prediction model, the system will obtain the log data to be processed from the current business system in real time. These log data represent various types of information generated during the operation of the current system. The system analyzes these log data and calculates their frequent item sets, similar to the method for processing historical log data mentioned above. The log item sets to be processed reflect the potential problems and alarms in the current system. By calculating the frequent item sets, the system can quickly identify feature combinations similar to historical faults.

[0067] S207, inputting the log item set to be processed into the fault prediction model to obtain a fault prediction result.

[0068] Specifically, the system inputs the set of log items to be processed calculated by step S206 into the trained fault prediction model. Based on the previously trained fault prediction model, the system can analyze the current set of log items to be processed and predict the possible fault types and fault recovery operations, i.e., fault repair measures, in the system. The fault prediction model uses the fault modes and repair measures summarized in the historical data to determine whether the feature combination in the current log item set matches certain known faults. If the match is successful, the system will generate the corresponding fault prediction results to help operation and maintenance personnel or automation systems identify potential faults in advance.

[0069] S208: Identify the system failure of the business system based on the failure prediction result, and perform fault repair on the system failure of the business system.

[0070] Specifically, after obtaining the fault prediction results, the system will identify the specific fault type and location in the business system based on the prediction results. Then, the system will generate a corresponding fault repair plan based on the fault recovery operation in the fault prediction results, and automatically perform repair operations according to the fault repair plan to repair the business system fault.

[0071] In the above embodiment, the present application discloses a method for handling system failures, by obtaining log data of system failures and log data of failure recovery operations, and calculating their frequent item sets, and then learning the features of the frequent item sets through a prediction model, so that the prediction model learns the correlation between the cause of the failure and the failure recovery operation, thereby obtaining the automated processing capability of predicting system failures and predicting failure recovery operations, significantly reducing the need for manual intervention, shortening the failure processing time, and improving the automated processing capability of business system failures and the failure recovery efficiency.

[0072] It should be noted that for new alarm information detected by the operation and maintenance monitoring platform, the intelligent fault handling system generates a processing plan based on the analysis of historical similar alarms and fault recovery operations, and recommends it to the operation and maintenance manager, requesting the operation and maintenance personnel to authorize the change operation to perform fault self-healing. At the same time, in order to ensure that false alarms trigger erroneous operations and affect normal services, prerequisite restrictions can be placed on fault self-healing recovery operations. For example, when an application instance is closed and isolated, it is necessary to ensure that the number of remaining surviving instances is greater than or equal to 50% of the total number of instances in the system.

[0073] For further information, please refer to Figure 3 , calculating the frequent itemsets of the first log data, and obtaining the first log itemsets, specifically includes:

[0074] S301, identifying all transactions in a first log item set to obtain a first log transaction;

[0075] S302, dividing the first log data according to a preset division rule to generate a plurality of first candidate item sets;

[0076] S303, traversing each first candidate item set, and counting the number of occurrences of the first log transaction in each first candidate item set;

[0077] S304, calculating the support of each first candidate item set according to the number of occurrences of the first log transaction in each first candidate item set;

[0078] S305 , based on the support of each first candidate item set and a preset first support threshold, screen a plurality of first candidate item sets to obtain a first log item set.

[0079] In this embodiment, first, by identifying all transactions in the first log item set, the system first obtains the first log transaction, which includes all records appearing in the historical fault log. For example, in a system fault log, a transaction may include all log entries, error codes, warning information, etc. at a specific time point. Then, according to the preset division rules (such as time period, fault type, etc.), the first log data is divided into several first candidate item sets, and these candidate item sets represent the potential frequent patterns in the log data. Then, the system traverses each first candidate item set and counts the occurrence frequency of the first log transaction in each first candidate item set. The transaction occurrence frequency is a key indicator for measuring the frequency of the item set. By calculating the support of each candidate item set (i.e., the proportion of the occurrence frequency of the first log transaction in each item set), the system can evaluate the importance of each item set. The screening of the support is implemented based on the preset first support threshold. This first support threshold is used to screen out the candidate item set whose occurrence frequency of the first log transaction is higher than the first support threshold, and the screened candidate item set is used as the first log item set.

[0080] Through the above steps, the system can accurately identify frequent failure modes in log data, filter out log item sets that are of significant significance for failure analysis, and improve the recognition accuracy of failure modes.

[0081] For further information, please refer to Figure 4 , calculating the frequent itemsets of the second log data, and obtaining the steps of the second log itemsets, specifically comprising:

[0082] S401, identifying all transactions in a second log item set to obtain a second log transaction;

[0083] S402, dividing the second log data according to a preset division rule to generate a plurality of second candidate item sets;

[0084] S403, traversing each second candidate item set, and counting the number of occurrences of the second log transaction in each second candidate item set;

[0085] S404, calculating the support degree of each second candidate item set according to the number of occurrences of the second log transaction in each second candidate item set;

[0086] S405 , based on the support of each second candidate item set and a preset second support threshold, screen a plurality of second candidate item sets to obtain a second log item set.

[0087] In this embodiment, when processing the second log data, the system first identifies all transactions in the second log item set, which represent records or events of historical fault recovery operations, and each transaction usually contains specific recovery steps and their corresponding context information. Then, according to the preset division rules (such as the type of recovery operation, processing time, etc.), the system divides the second log data into a number of second candidate item sets, which represent possible recovery operation modes in the log data. The system traverses each second candidate item set and counts the number of occurrences of the second log transaction in each second candidate item set. This process helps to identify common operation step combinations in the fault handling process. Next, according to the number of occurrences of the second log transaction in each second candidate item set, the support of each second candidate item set is calculated, and the support is a standard for measuring the frequency of the item set. Finally, based on the support of each second candidate item set and the preset second support threshold, the system screens the second candidate item set, and screens out those candidate item sets whose occurrence frequency is higher than the second support threshold, and obtains the second log item set.

[0088] Through the above steps, by calculating the frequent item sets of the second log data, common recovery operation modes are effectively identified, and recovery step combinations with high frequency are screened out.

[0089] For further information, please refer to Figure 5 The steps of constructing a training data set based on the first log item set and the second log item set, and using the training data set to train a fault prediction model specifically include:

[0090] S501, obtaining a log transaction matching result for transactions in the first log item set and the second log item set;

[0091] S502, filtering a log item set pair according to the log transaction matching result, wherein the log item set pair includes a first log item set and a second log item set;

[0092] S503, constructing a training data set based on the log item set pairs;

[0093] S504: Import the training data set into a preset initial prediction model to obtain a fault prediction model.

[0094] In this embodiment, the system first matches the transactions in the first log item set and the second log item set to obtain the log transaction matching result, which reflects the relationship between the fault mode in the fault log data and the corresponding repair operation. Through comparison and matching, the system can identify the matching pair of the fault mode and the recovery operation. Then, according to the log transaction matching result, the system screens out the log item set pairs that meet the conditions, and the log item set pairs include an item set of the first log item set and an item set of the second log item set. The matching criteria usually include the matching frequency, support and preset threshold of the item set pairs. Subsequently, based on these screened log item set pairs, the system constructs training data sets, which contain the patterns of historical faults and their repair operations. Finally, the constructed training data set is imported into the preset initial prediction model, and a fault prediction model is generated through training. This model can learn the relationship between the fault mode and the repair measures, so that when similar faults are encountered in the future, it can more accurately predict and recommend repair operations.

[0095] Through the above steps, the system can build a high-quality fault prediction model, which is trained based on actual fault and repair data, improving the accuracy and practicality of the prediction, reducing dependence on manual experience, and improving the level of automation in fault handling, so that the system can respond to and handle future fault events more quickly.

[0096] Furthermore, the fault prediction model includes a fault type prediction sub-model, the initial prediction model includes a first prediction sub-model, and the step of importing the training data set into the preset initial prediction model to obtain the fault prediction model specifically includes:

[0097] Performing fault classification annotation on the first log data to generate a fault classification label;

[0098] Identify the first log item set in the training data set, obtain the fault classification label that matches the first log item set in the training data set, and obtain the target fault classification label;

[0099] Perform feature extraction on the first log item set in the training data set to obtain the first log item set features;

[0100] Importing the target fault classification label and the first item set feature into a first prediction sub-model, wherein the first prediction sub-model is a clustering model;

[0101] In the first prediction sub-model, clustering operation is performed on the features of the first item set based on the target fault classification label;

[0102] During the clustering operation, the first prediction sub-model is used to learn clustering features to obtain a fault type prediction sub-model.

[0103] In this scheme, the first log data is first labeled with fault classification to generate corresponding fault classification labels, which represent the specific classification information of different fault types. Next, the system identifies the first log item set in the training data set and obtains the fault classification labels that match these item sets, thereby obtaining the target fault classification labels. This step ensures that the system can associate the actual fault mode with the classification labels. Subsequently, the system extracts features from the first log item set in the training data set to obtain the first log item set features, which represent the important attributes and patterns in the fault log. Subsequently, the target fault classification labels and the extracted features are imported into the first prediction sub-model, wherein the prediction sub-model is a clustering model for clustering features. In the clustering process, the model learns how to effectively group and cluster features according to the fault classification labels. By learning the relationship between features and classification labels, it helps the system to more accurately classify and predict fault types in practical applications, thereby generating a fault type prediction sub-model that can identify different fault types.

[0104] Through the above steps, the system can build an accurate fault type prediction sub-model, which can effectively identify and classify different fault types through clustering learning.

[0105] It should be noted that when the fault type prediction sub-model predicts a possible fault, in order to avoid misoperation in the early stage, the operation and maintenance team can conduct a second confirmation and approval of the disposal plan recommended by the system. After approval, the recovery plan will be automatically executed in advance to deal with it, such as increasing resource allocation, optimizing service configuration, restarting services, network switching, etc., to avoid or reduce the impact of the fault.

[0106] As time goes by, the operation and maintenance team needs to continuously collect new log data and update the fault type prediction sub-model to improve the accuracy and timeliness of the prediction. When the fault prediction accuracy meets the business needs, it can automatically handle the predicted possible faults in advance without approval, and notify the operation and maintenance team of the handling results for confirmation, without the need for human intervention.

[0107] Furthermore, the fault prediction model further includes a fault recovery operation prediction sub-model, the initial prediction model includes a second prediction sub-model, and the step of importing the training data set into the preset initial prediction model to obtain the fault prediction model further includes:

[0108] Perform feature extraction on the training data set to obtain the features of the second item set;

[0109] Importing the second item set features into the second prediction sub-model, wherein the second prediction sub-model is a deep convolutional network, and the deep convolutional network includes an embedding layer, a convolutional layer, and a fully connected layer;

[0110] Importing the second item set features into the second prediction sub-model;

[0111] Mapping the features of the second item set into a preset feature space through an embedding layer, and obtaining a feature distribution map from the feature space;

[0112] Perform convolution operation on the feature distribution map through the convolution layer to obtain the local feature vector;

[0113] Through the fully connected layer, the fault recovery operation prediction is performed based on the local feature vector to obtain the fault recovery operation prediction result;

[0114] Based on the fault recovery operation prediction result and the preset standard result, the second prediction sub-model is iteratively updated using the back propagation algorithm until the model is fitted, thereby obtaining the fault recovery operation prediction sub-model.

[0115] In this scheme, the system first extracts features from the training data set to obtain the features of the second log item set. These features represent the key attributes and patterns in the fault log data, including the correlation between the repair operation and the fault mode. Subsequently, these features are input into the second prediction sub-model. The second prediction sub-model is a deep convolutional network, which consists of three main parts: an embedding layer, a convolutional layer, and a fully connected layer. First, the embedding layer maps the features of the second log item set into a preset feature space and generates a feature distribution map, which shows the distribution of the features in the feature space. Then, the convolutional layer performs a convolution operation on the feature distribution map to extract local feature vectors, which capture the local patterns and important information of the features. Finally, the fully connected layer predicts the fault recovery operation based on the extracted local feature vectors and generates a prediction result. In order to improve the accuracy and generalization ability of the model, the system uses the back propagation algorithm to iteratively update the second prediction sub-model. By continuously adjusting the model parameters until the model fits the preset standard, an efficient fault recovery operation prediction sub-model is obtained.

[0116] Through the above steps, the system can build an accurate fault recovery operation prediction sub-model, which uses the powerful feature extraction and learning capabilities of the deep convolutional network to improve the prediction accuracy of the fault recovery operation.

[0117] Further, based on the fault recovery operation prediction result and the preset standard result, the second prediction sub-model is iteratively updated using the back propagation algorithm until the model is fitted to obtain the fault recovery operation prediction sub-model, which specifically includes:

[0118] Performing operation classification marking on the second log data to generate an operation classification label;

[0119] Identify a second log item set in the training data set, obtain an operation classification label that matches the second log item set in the training data set, and obtain a target operation classification label;

[0120] Combine the target fault classification label and the target operation classification label to obtain a standard result;

[0121] Calculate the prediction error between the predicted result of the failure recovery operation and the standard result;

[0122] Use the back-propagation algorithm to propagate the prediction error in the deep convolutional network and identify the errors of each network layer in the deep convolutional network;

[0123] Determine whether the error of each network layer is less than or equal to a preset error threshold;

[0124] If not, the parameters of each network layer are adjusted until the errors of each network layer are less than or equal to the error threshold, thereby obtaining a fault recovery operation prediction sub-model.

[0125] In this scheme, the system first performs operation classification annotation on the second log data to generate operation classification labels, which identify various operation types or steps. Next, the system identifies the second log item set in the training data set and obtains the operation classification labels that match these item sets to obtain the target operation classification labels. The system combines these target operation classification labels with the previous target fault classification labels to obtain standard results, which are used to compare with the predicted results. Then, the system calculates the prediction error between the fault recovery operation prediction result and the standard result to evaluate the accuracy of the model prediction. Next, the prediction error is propagated in the deep convolutional network using the backpropagation algorithm to identify and analyze the errors of each network layer. Finally, the system checks whether the errors of each layer are less than or equal to the preset error threshold. If not, the parameters of the network layer are adjusted and iterative updates are continued until the errors of each network layer are less than or equal to the error threshold, and the fault recovery operation prediction sub-model is obtained.

[0126] Through the above steps, the system can accurately optimize the parameters in the deep convolutional network, improve the accuracy of fault recovery operation prediction, and the final generated prediction sub-model can more effectively match fault and operation classification.

[0127] In this embodiment, the method for handling system failures is executed on the electronic device (eg Figure 1 The server shown in the figure can receive instructions or obtain data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0129] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0130] Further references Figure 6 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a system failure processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0131] like Figure 6 As shown, the system fault processing device 600 described in this embodiment is used to run a fault processing system, the fault processing system is connected to the business system, the fault processing system includes an operation and maintenance monitoring platform and an operation and maintenance change platform, and the system fault processing device 600 includes:

[0132] The first log acquisition module 601 is used to acquire log data of historical system failures from the operation and maintenance monitoring platform to obtain first log data;

[0133] A first frequent item set module 602, used to calculate frequent item sets of first log data to obtain a first log item set;

[0134] The second log acquisition module 603 is used to acquire log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match historical system faults;

[0135] A second frequent item set module 604, used to calculate frequent item sets of second log data to obtain second log item sets;

[0136] A prediction model training module 605 is used to construct a training data set based on the first log item set and the second log item set, and use the training data set to train a fault prediction model;

[0137] The third frequent item set module 606 is used to obtain the log data to be processed in real time, calculate the frequent item sets of the log data to be processed, and obtain the log item sets to be processed;

[0138] The system fault prediction module 607 is used to input the log item set to be processed into the fault prediction model to obtain the fault prediction result;

[0139] The system fault repair module 608 is used to identify the system fault of the business system based on the fault prediction result and perform fault repair on the system fault of the business system.

[0140] Furthermore, the first frequent item set module 602 is specifically used for:

[0141] Identify all transactions in a first log item set to obtain first log transactions; partition the first log data according to a preset partitioning rule to generate a plurality of first candidate item sets; traverse each first candidate item set and count the number of occurrences of the first log transaction in each first candidate item set; calculate the support of each first candidate item set based on the number of occurrences of the first log transaction in each first candidate item set; screen a plurality of first candidate item sets based on the support of each first candidate item set and a preset first support threshold to obtain a first log item set.

[0142] Furthermore, the second frequent itemset module 604 is specifically used for:

[0143] Identify all transactions in the second log item set to obtain second log transactions; divide the second log data according to a preset division rule to generate a plurality of second candidate item sets; traverse each second candidate item set and count the number of occurrences of the second log transaction in each second candidate item set; calculate the support of each second candidate item set according to the number of occurrences of the second log transaction in each second candidate item set; screen a plurality of second candidate item sets based on the support of each second candidate item set and a preset second support threshold to obtain a second log item set.

[0144] Furthermore, the prediction model training module 605 is specifically used for:

[0145] For transactions in the first log item set and the second log item set, a log transaction matching result is obtained; according to the log transaction matching result, a log item set pair is screened, wherein the log item set pair includes a first log item set and a second log item set; a training data set is constructed based on the log item set pair; and the training data set is imported into a preset initial prediction model to obtain a fault prediction model.

[0146] Further, the fault prediction model includes a fault type prediction sub-model, the initial prediction model includes a first prediction sub-model, and the prediction model training module 605 is further used to:

[0147] Perform fault classification annotation on the first log data to generate a fault classification label; identify the first log item set in the training data set, obtain the fault classification label that matches the first log item set in the training data set, and obtain a target fault classification label; perform feature extraction on the first log item set in the training data set to obtain a first log item set feature; import the target fault classification label and the first log item set feature into a first prediction sub-model, wherein a prediction sub-model is a clustering model; in the first prediction sub-model, perform a clustering operation on the first log item set feature based on the target fault classification label; during the clustering operation, use the first prediction sub-model to learn the clustering features to obtain a fault type prediction sub-model.

[0148] Furthermore, the fault prediction model further includes a fault recovery operation prediction sub-model, the initial prediction model includes a second prediction sub-model, and the prediction model training module 605 is further used to:

[0149] Extract features from the training data set to obtain features of the second item set; import the features of the second item set into the second prediction sub-model, wherein the second prediction sub-model is a deep convolutional network, and the deep convolutional network includes an embedding layer, a convolutional layer, and a fully connected layer;

[0150] Import the features of the second item set into the second prediction sub-model; map the features of the second item set into a preset feature space through an embedding layer, and obtain a feature distribution map from the feature space; perform a convolution operation on the feature distribution map through a convolution layer to obtain a local feature vector; perform a fault recovery operation prediction based on the local feature vector through a fully connected layer to obtain a fault recovery operation prediction result; based on the fault recovery operation prediction result and the preset standard result, use a back propagation algorithm to iteratively update the second prediction sub-model until the model is fitted, and obtain a fault recovery operation prediction sub-model.

[0151] Furthermore, the prediction model training module 605 is also used for:

[0152] The second log data is annotated with operation classification to generate operation classification labels; the second log item set in the training data set is identified, and the operation classification labels matching the second log item set in the training data set are obtained to obtain target operation classification labels; the target fault classification labels and the target operation classification labels are combined to obtain standard results; the prediction error between the fault recovery operation prediction result and the standard result is calculated; the prediction error is propagated in the deep convolutional network using the back propagation algorithm, and the errors of each network layer in the deep convolutional network are identified; it is determined whether the errors of each network layer are less than or equal to a preset error threshold; if not, the parameters of each network layer are adjusted until the errors of each network layer are less than or equal to the error threshold, thereby obtaining a fault recovery operation prediction sub-model.

[0153] In the above embodiment, the present application discloses a system fault processing device, which obtains the log data of the system fault and the log data of the fault recovery operation, calculates their frequent item sets, and then learns the features of the frequent item sets through a prediction model, so that the prediction model learns the correlation between the fault cause and the fault recovery operation, thereby obtaining the automated processing capability of predicting system faults and predicting fault recovery operations, significantly reducing the need for manual intervention, shortening the fault processing time, and improving the automated processing capability of business system faults and the fault recovery efficiency.

[0154] To solve the above technical problems, the present application also provides a computer device. Figure 7 , Figure 7 This is a basic structural block diagram of the computer device in this embodiment.

[0155] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 7 having a memory 71, a processor 72, and a network interface 73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0156] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0157] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 7. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions of a method for handling system failures, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0158] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run computer-readable instructions or process data stored in the memory 71, such as computer-readable instructions for running the method for processing the system failure.

[0159] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0160] In the above embodiment, the present application discloses a computer device, which obtains log data of system failures and log data of failure recovery operations, calculates their frequent item sets, and then learns the features of the frequent item sets through a prediction model, so that the prediction model learns the correlation between the cause of the failure and the failure recovery operation, thereby obtaining the automated processing capability of predicting system failures and predicting failure recovery operations, significantly reducing the need for manual intervention, shortening the failure processing time, and improving the automated processing capability of business system failures and the failure recovery efficiency.

[0161] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the system failure processing method as described above.

[0162] In the above embodiments, the present application discloses a computer-readable storage medium, which obtains log data of system failures and log data of failure recovery operations, calculates their frequent item sets, and then learns the features of the frequent item sets through a prediction model, so that the prediction model learns the correlation between the cause of the failure and the failure recovery operation, thereby obtaining the automated processing capability of predicting system failures and predicting failure recovery operations, significantly reducing the need for manual intervention, shortening the failure processing time, and improving the automated processing capability of business system failures and the failure recovery efficiency.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0164] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for handling system failure, characterized in that: The processing method is applied to a fault processing system, the fault processing system is connected to a business system, the fault processing system includes an operation and maintenance monitoring platform and an operation and maintenance change platform, and the processing method includes: Obtaining log data of historical system failures from the operation and maintenance monitoring platform to obtain first log data; Calculate the frequent itemsets of the first log data to obtain a first log item set; Acquire log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match the historical system faults; Calculate the frequent itemsets of the second log data to obtain a second log itemset; Based on the first log item set and the second log item set, construct a training data set, and use the training data set to train a fault prediction model; Acquire the log data to be processed in real time, calculate the frequent item sets of the log data to be processed, and obtain the log item sets to be processed; Inputting the to-be-processed log item set into the fault prediction model to obtain a fault prediction result; A system fault of the business system is identified based on the fault prediction result, and the system fault of the business system is repaired.

2. The method for handling system failure according to claim 1, characterized in that: The step of calculating the frequent itemsets of the first log data to obtain the first log itemsets specifically includes: Identify all transactions in the first log item set to obtain a first log transaction; Performing data division on the first log data according to a preset division rule to generate a plurality of first candidate item sets; Traversing each of the first candidate item sets, and counting the number of occurrences of the first log transaction in each of the first candidate item sets; Calculating the support of each of the first candidate item sets according to the number of occurrences of the first log transaction in each of the first candidate item sets; Based on the support of each of the first candidate item sets and a preset first support threshold, a plurality of the first candidate item sets are screened to obtain the first log item set.

3. The method for handling system failure according to claim 1, characterized in that: The step of calculating the frequent itemsets of the second log data to obtain the second log itemsets specifically includes: Identify all transactions in the second log item set to obtain second log transactions; Performing data division on the second log data according to a preset division rule to generate a plurality of second candidate item sets; Traversing each of the second candidate item sets, and counting the number of occurrences of the second log transaction in each of the second candidate item sets; Calculate the support of each second candidate item set according to the number of occurrences of the second log transaction in each second candidate item set; Based on the support of each of the second candidate item sets and a preset second support threshold, a plurality of the second candidate item sets are screened to obtain the second log item set.

4. The method for handling system failure according to any one of claims 2 or 3, characterized in that: The step of constructing a training data set based on the first log item set and the second log item set, and using the training data set to train a fault prediction model specifically includes: Obtaining a log transaction matching result for transactions in the first log item set and the second log item set; Filtering a log item set pair according to the log transaction matching result, wherein the log item set pair includes the first log item set and the second log item set; Constructing the training data set based on the log item set pairs; The training data set is imported into a preset initial prediction model to obtain the fault prediction model.

5. The method for handling system failure according to claim 4, characterized in that: The fault prediction model includes a fault type prediction sub-model, the initial prediction model includes a first prediction sub-model, and the step of importing the training data set into a preset initial prediction model to obtain the fault prediction model specifically includes: Performing fault classification marking on the first log data to generate a fault classification label; Identify a first log item set in the training data set, obtain a fault classification label that matches the first log item set in the training data set, and obtain a target fault classification label; Performing feature extraction on the first log item set in the training data set to obtain features of the first log item set; Importing the target fault classification label and the first item set feature into the first prediction sub-model, wherein the first prediction sub-model is a clustering model; In the first prediction sub-model, clustering operation is performed on the first item set features based on the target fault classification label; During the clustering operation, the first prediction sub-model is used to learn clustering features to obtain the fault type prediction sub-model.

6. The method for handling system failure according to claim 5, characterized in that: The fault prediction model further includes a fault recovery operation prediction sub-model, the initial prediction model includes a second prediction sub-model, and the step of importing the training data set into a preset initial prediction model to obtain the fault prediction model further includes: Extract features from the training data set to obtain features of a second item set; Importing the second item set features into the second prediction sub-model, wherein the second prediction sub-model is a deep convolutional network, and the deep convolutional network includes an embedding layer, a convolutional layer, and a fully connected layer; Importing the second item set features into the second prediction sub-model; Mapping the second item set features into a preset feature space through the embedding layer, and acquiring a feature distribution map from the feature space; Performing a convolution operation on the feature distribution map through the convolution layer to obtain a local feature vector; By using the fully connected layer and based on the local feature vector, a fault recovery operation prediction is performed to obtain a fault recovery operation prediction result; Based on the fault recovery operation prediction result and the preset standard result, the second prediction sub-model is iteratively updated using a back propagation algorithm until the model is fitted, thereby obtaining the fault recovery operation prediction sub-model.

7. The method for handling system failure according to claim 6, characterized in that: The step of iteratively updating the second prediction sub-model based on the fault recovery operation prediction result and the preset standard result by using a back propagation algorithm until the model is fitted to obtain the fault recovery operation prediction sub-model specifically includes: Performing operation classification marking on the second log data to generate an operation classification label; Identify a second log item set in the training data set, obtain an operation classification label that matches the second log item set in the training data set, and obtain a target operation classification label; Combining the target fault classification label and the target operation classification label to obtain the standard result; Calculating a prediction error between the fault recovery operation prediction result and the standard result; Propagate the prediction error in the deep convolutional network using the back propagation algorithm and identify errors in each network layer in the deep convolutional network; Determine whether the error of each network layer is less than or equal to a preset error threshold; If not, the parameters of each network layer are adjusted until the errors of each network layer are less than or equal to the error threshold, thereby obtaining the fault recovery operation prediction sub-model.

8. A system failure processing device, characterized in that: The processing device is used to run a fault processing system, the fault processing system is connected to a business system, the fault processing system includes an operation and maintenance monitoring platform and an operation and maintenance change platform, and the processing device includes: A first log acquisition module is used to acquire log data of historical system failures from the operation and maintenance monitoring platform to obtain first log data; A first frequent item set module, used to calculate the frequent item sets of the first log data to obtain a first log item set; A second log acquisition module is used to acquire log data of historical fault recovery operations from the operation and maintenance change platform to obtain second log data, wherein the historical fault recovery operations are fault recovery operations that match the historical system faults; A second frequent item set module, used to calculate the frequent item sets of the second log data to obtain a second log item set; A prediction model training module, used to construct a training data set based on the first log item set and the second log item set, and use the training data set to train a fault prediction model; The third frequent item set module is used to obtain the log data to be processed in real time, calculate the frequent item sets of the log data to be processed, and obtain the log item sets to be processed; A system fault prediction module, used for inputting the set of log items to be processed into the fault prediction model to obtain a fault prediction result; A system fault repair module is used to identify the system fault of the business system based on the fault prediction result, and perform fault repair on the system fault of the business system.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the method for processing system failure according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the method for processing a system failure according to any one of claims 1 to 7 are implemented.

Images