Abnormal log processing method and device
By preprocessing and classifying exception logs, combined with pre-created solution libraries, quickly locate and resolve system failures, the fault location and resolution efficiency of operation and maintenance personnel is improved.
Patent Information
- Application Number
- PCT/CN2024/136147
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-11-30
- Filing Date
- 2024-12-02
- Publication Date
- 2025-06-05
AI Technical Summary
When the operation and maintenance personnel analyze abnormal logs to locate system failures, there is a problem of low fault positioning efficiency.
By collecting exception logs generated during the operation of the business system, pre-processing, and classifying the exception logs using the pre-trained log classification model, searching for pre-created solution libraries to query solutions that match the fault type, and pushing the optimal solution to the operation and maintenance personnel.
It improves the efficiency of operation and maintenance staff in solving faults, allowing operation and maintenance personnel to quickly deal with faults that occur during system operation.
Smart Images

Figure CN2024136147_05062025_PF_FP_ABST
Abstract
Description
Abnormal log processing method and device Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method and device for processing abnormal logs. Background Art
[0002] With the rapid development of information technology, systems and applications have been widely used. During the operation of these systems and applications, massive amounts of log data are generated, including a large number of exception logs. By analyzing these exception logs, we can locate the location and cause of system failures.
[0003] However, when operations and maintenance personnel analyze exception logs to determine the fault location, they usually adopt the following three solutions: first, try various solutions directly in the system based on the exception log; second, ask other experienced operations and maintenance personnel; third, search the operations and maintenance logs or the Internet for solutions. Regardless of which method is chosen, the fault location efficiency is low. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide a method and device for processing abnormal logs to solve at least part of the above technical problems. The technical solutions provided are as follows:
[0005] In a first aspect, an embodiment of the present application provides a method for processing abnormal logs, including:
[0006] Collect exception logs generated during the operation of business systems;
[0007] Preprocess exception logs;
[0008] Based on the pre-trained log classification model, the pre-processed abnormal logs are classified to obtain the corresponding fault type;
[0009] A pre-created solution library is searched to query for a solution that matches the fault type, wherein the solution library includes the fault type and the association relationship between the fault type and the corresponding solution.
[0010] In a second aspect, an embodiment of the present application further provides an abnormal log processing device, comprising:
[0011] Log collection module, used to collect abnormal logs generated during the operation of the business system;
[0012] A log management module is used to pre-process the abnormal logs and classify the pre-processed abnormal logs into corresponding fault categories by calling a log classification model;
[0013] The solution recommendation module is used to search a pre-created solution library to query for solutions that match the fault category.
[0014] Compared with the existing technology, the above technical solution provided by this application has the following advantages: the exception log processing method provided by this application collects exception logs during the operation of the system (microservice or application) and uses a pre-trained log classification model to quickly classify the exception logs. It further searches for solutions that match the classification category of the exception log from a pre-created solution library, and pushes the optimal solution to the operation and maintenance personnel, allowing them to quickly resolve or handle faults that occur during system operation, thereby improving the efficiency of the operation and maintenance staff in resolving faults. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0016] FIG1 is a flow chart of an abnormal log processing method provided by an embodiment of the present application;
[0017] FIG2 is a flow chart of another abnormal log processing method provided by an embodiment of the present application;
[0018] FIG3 is a schematic diagram of the structure of an abnormal log processing device provided in an embodiment of the present application;
[0019] FIG4 is a schematic diagram of the structure of another abnormal log processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0020] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0021] To make the description of the following embodiments clear and concise, a brief introduction to the related technologies is first given:
[0022] Natural Language Processing (NLP): NLP technology can transform text data into a form that computers can understand and process. In log classification, NLP can be applied to tasks such as text segmentation, part-of-speech tagging, and named entity recognition to extract useful feature information.
[0023] Machine Learning: Machine learning algorithms can be used to identify and classify different types of logs by training models. Common machine learning algorithms include Naive Bayes classifiers, support vector machines (SVMs), decision trees, and random forests. These algorithms can learn the characteristic patterns of log text and infer the log's category.
[0024] Feature Engineering: Feature engineering involves constructing effective feature representations from raw data to improve the performance of classification algorithms. In log classification, feature engineering can include techniques such as word frequency statistics, TF-IDF (Term Frequency-Inverse Document Frequency) calculations, and word embedding.
[0025] Deep learning: Deep learning techniques such as convolutional neural networks (CNN) and recurrent neural networks (RNN) are also used in log classification. These models can automatically learn features from data and perform classification and prediction through multi-layer neural networks.
[0026] Incremental learning: Since the amount of log data is huge and constantly growing, the use of incremental learning algorithms can achieve rapid classification of new logs and update models without retraining the entire model.
[0027] Please refer to FIG1 , which shows a flow chart of an abnormal log processing method provided by an embodiment of the present application. The method is applied to an operation and maintenance system and may include the following steps:
[0028] S101, collecting abnormal logs generated during the operation of the business system.
[0029] Collect exception logs generated during the operation of each system (microservice or application). The collection targets are mainly exception logs, including logs of different types, levels, and events. For example, the levels of exception logs may include:
[0030] Debug: Records debugging-related information for development and troubleshooting.
[0031] Warning: Records warning information, indicating possible problems or potential errors.
[0032] Error: Records error information, indicating that a specific error or failure has occurred.
[0033] Critical: Records serious error information, indicating that a very serious failure has occurred, which may cause the system to crash or not work properly.
[0034] Fatal: Records fatal error information, indicating that an unrecoverable error has occurred and the system cannot continue to run.
[0035] In one exemplary embodiment, a corresponding agent collection program can be deployed based on the system or microservice logging framework. The collected log levels can be configured, such as warning, error, critical, and fatal. When the system or microservice log reaches the configured collection level, an exception log collection action is triggered. For example, if a log contains an error, the log is collected.
[0036] S102: Perform data preprocessing on the abnormal log.
[0037] Log preprocessing is crucial for the effectiveness of classification algorithms. Preprocessing steps can include removing stop words, standardizing text formats (i.e., log formatting), and handling missing values and outliers to reduce noise interference and improve classification accuracy.
[0038] In the embodiment of the present application, the purpose of log formatting is to standardize the collected log data and to organize and sort the original log information into a certain format for subsequent processing, storage and analysis.
[0039] In an exemplary embodiment, the log data formatting method may include the following:
[0040] (1) Timestamp
[0041] Add timestamp information to the log to record the specific time when the event occurred. The timestamp can use a standard date and time format, such as "2023-06-21T10:30:00Z", or other custom formats.
[0042] (2) Log level
[0043] Assign a level to each log entry to indicate its importance or severity. Common log levels include debug, warning, error, critical, fatal, etc.
[0044] (3) Log source
[0045] Record the name or identifier of the module, component, or application that generates the log to facilitate tracing the source of the log.
[0046] (4) Message content
[0047] Record the specific content of the log, including descriptive text information, error stack trace, exception information, etc. Make sure the message content is concise and clear and can clearly convey the required information.
[0048] (5) Contextual information
[0049] Add contextual information related to the log as needed. This information may include user ID, device ID, request parameters, response code, etc., to better understand and analyze the log.
[0050] (6) Separators and format specifications
[0051] Use appropriate delimiters and format specifications to ensure readability and parsability of log data. For example, use delimiters such as commas and tabs, or adopt common log formats such as Apache log format or JSON format.
[0052] (7) Visualization
[0053] As needed, the formatted log data can be displayed in graphical or tabular form to provide more intuitive observation and analysis methods.
[0054] S103: extract the feature vector of the pre-processed abnormal log and input it into a pre-trained log classification model for classification to obtain a fault category.
[0055] The log classification model is a model based on a machine learning algorithm. The log classification model is designed and trained with training data so that the model automatically learns the log classification method from the training data, and finally obtains a trained log classification model.
[0056] In one embodiment, the process of obtaining log classification using the log classification model may include:
[0057] (1) Feature extraction
[0058] Useful features can be extracted from the formatted exception log data, such as timestamp, log level, component name, keyword, etc. In addition, for text features, operations such as word segmentation, stop word removal, and stemming can be performed.
[0059] For example, the log data "10.20.20.10; [2018-07-16 13:12:57]; GET / online / sample HTTP / 1.1; 200" is segmented based on punctuation or special symbols such as ;:<>[]{} / \n\t\r. After segmentation, the log data becomes: 10.20.20.10 2018-07-16 13 12 57GET online sample HTTP 1.1 200. After segmentation, stemming can be performed. The extracted stem for this log is: get online sample http.
[0060] (2) Feature vectorization
[0061] Convert the extracted features into numerical representations. For example, you can use vector conversion tools such as one-hot encoding, bag of words, and TF-IDF (Term Frequency-Inverse Document Frequency) to convert text data into sparse or dense vectors.
[0062] (3) Input the vector corresponding to the log data into the log classification model, and output the category label corresponding to the log through the prediction function of the model.
[0063] S104: Search a pre-established solution library to obtain the optimal solution corresponding to the fault category, and push the optimal solution.
[0064] In one embodiment, a relationship between faults and solutions is established based on a rule matching model to form a solution library. For example, the solution library can be in the form of the following Table 1:
[0065] Table 1
[0066] Based on the category labels of the abnormal logs obtained by the log classification model analysis, the existing solution library is searched to query solutions that match the fault manifestations and causes of the current abnormal logs. For example, you can search based on the category labels, keywords or fault descriptions of the abnormal logs to quickly find relevant solutions.
[0067] For example, if the current fault is determined to be a database fault and caused by a lock conflict based on the exception log, the category of the exception log is "database fault-lock conflict", and a solution matching "database fault-lock conflict" is further searched from the solution library.
[0068] Furthermore, after searching the solution library for a solution that matches the anomaly log, we can evaluate whether the solution matches the fault. For example, we can determine whether the solution matches the fault based on the confidence level between the two. A higher confidence level indicates a better match between the solution and the fault, thereby ensuring that the steps and suggestions in the searched solution are applicable to the current fault situation, thereby improving the accuracy of the searched solution. For example, the confidence level between the solution and the fault can be calculated using entropy. The entropy value is calculated using a certain calculation logic, and the lower the entropy value, the higher the confidence level.
[0069] In an exemplary embodiment, the optimal solution can be sent to the terminal device used by the operation and maintenance personnel, or the optimal solution can be displayed on the client of the operation and maintenance system. This application does not specifically limit the specific method of pushing the optimal solution.
[0070] In addition, in other embodiments of the present application, after the searched solution is pushed to the operation and maintenance personnel, the operation and maintenance personnel will provide feedback on the solution, that is, whether the pushed solution can solve the fault problem corresponding to the abnormal log, and the log classification model will be further strengthened and trained based on this feedback result.
[0071] In addition, during the actual application process, the solution library is continuously optimized and updated to include the latest and most applicable solutions, thereby improving the accuracy and applicability of the solution library.
[0072] The exception log processing method provided in this embodiment collects exception logs from the operation of the system (microservice or application) and uses a pre-trained log classification model to quickly classify the exception logs. It then searches a pre-created solution library for solutions that match the classification of the exception logs and pushes the optimal solution to operations and maintenance personnel, enabling them to quickly resolve or handle faults that occur during system operation, thereby improving their troubleshooting efficiency.
[0073] Please refer to Figure 2, which shows a flowchart of another abnormal log processing method provided by an embodiment of the present application. This embodiment will explain the abnormal log processing process in detail and focus on the training process of the log classification model. As shown in Figure 2, the method may include the following steps:
[0074] S201, collecting abnormal logs generated during the operation of the business system.
[0075] S202: Pre-process the abnormal log.
[0076] The implementation process of S201 to S202 is the same as that of S101 to S102 and will not be repeated here.
[0077] S203, use the pre-trained log classification model to classify the pre-processed exception log; if the model can automatically classify the exception log, execute S204; if it cannot automatically classify the exception log, execute S205, that is, use the exception log data that cannot be automatically classified to conduct targeted training on the log classification model, thereby improving the applicability of the log classification model.
[0078] The feature vector is extracted from the abnormal log and input into the trained log classification model. The model predicts the category label of the abnormal log, obtains the predicted category labels, and sorts them from high to low according to the confidence of each predicted category label. Finally, the category label with the highest confidence is determined as the fault category of the abnormal log.
[0079] The process of extracting the feature vector of the abnormal log can be found in the relevant content of S103 in the embodiment shown in FIG1 , and will not be described in detail here.
[0080] S204: Search a pre-created solution library, query for a solution that matches the fault category of the exception log, and push it.
[0081] S205, construct a fault classification system.
[0082] A fault classification system categorizes faults according to specific rules and standards to facilitate better system management and maintenance. The classification system is constructed based on multiple dimensions, including but not limited to anomaly classification, performance classification, and configuration issue classification.
[0083] Log data can be classified in different ways. In some embodiments, the log data classification methods may include the following:
[0084] (1) Classification by log level
[0085] Categorize logs based on their importance or severity, such as warning, error, critical, and fatal. This classification helps operations personnel (or developers) better understand the meaning and importance of logs and quickly locate problems.
[0086] (2) Classification by log type
[0087] Logs are categorized according to their function or purpose, such as application logs, system logs, security logs, performance logs, access logs, etc. Each type of log records different aspects of information, which helps analyze and troubleshoot specific issues.
[0088] (3) Classification by log source
[0089] Categorize logs based on the module, component, or application that generates them, such as database logs, network logs, server logs, application logs, etc. This classification method helps track and filter relevant logs in complex systems.
[0090] (4) Classification by time period
[0091] Categorize logs by time period, such as daily, weekly, monthly, or annually. This classification helps implement log storage and archiving strategies.
[0092] (5) Classification by log format
[0093] Log data is classified according to its format, such as text logs, JSON logs, XML logs, etc. Different formats are suitable for different application scenarios and analysis requirements.
[0094] (6) Classification by keywords
[0095] Classify by keywords: Classify logs according to keywords or specific content in the logs, such as error logs, warning logs, exception logs, information logs, etc.
[0096] In practical applications, at least two of the above classification methods can be selected, and different classification methods can be selected for different application scenarios.
[0097] S206: Mark the historical abnormality log data to obtain fault sample data.
[0098] In some embodiments, the fault sample data is manually labeled, for example, the fault type and labeling content of the fault sample data (ie, historical abnormal log data) can be determined based on the fault classification system constructed in the previous step to obtain labeled data.
[0099] S207, design a log classification model.
[0100] Select a classification model, such as a naive Bayes model, a support vector machine (SVM) model, a decision tree model, a random forest model, etc. Depending on the specific situation, you can also use a deep learning model, such as a convolutional neural network (CNN) module or a recurrent neural network (RNN) model.
[0101] S208: training a log classification model.
[0102] The labeled fault sample data is divided into a training set and a test set. Appropriate evaluation metrics are selected, and the established log classification model is trained using the training set. The specific training process may include: extracting the feature vector of the fault sample data and inputting it into the log classification model for classification to obtain a category prediction result. Then, based on the loss function, the accuracy, precision, and recall of the category prediction result and the labeled fault type corresponding to the fault sample data are calculated. If the metrics are not within the allowable range, the log classification model parameters are adjusted as needed until all metrics are within the allowable range, resulting in a usable log classification model.
[0103] In addition, the trained log classification model is tested using a test dataset. The feature vectors of the test dataset are extracted and fed into the log classification model to be tested. The model's prediction function then outputs the corresponding category labels for each test data point. Metrics such as accuracy, precision, and recall are then calculated. If all metrics are within the acceptable range, the trained log classification model's accuracy and generalization performance are considered suitable for the application.
[0104] The trained log classification model is used to classify the collected abnormal log data to obtain a category label. That is, the trained log classification model can be used to execute S203.
[0105] The abnormal log processing method provided in this embodiment obtains fault sample data by annotating historical abnormal log data, further extracts the feature vector of the fault sample data, and inputs the feature vector into the initial log classification model. The initial log classification model is used to classify the fault sample data to obtain the corresponding category label. Based on the categories obtained by the model and the categories annotated with the fault sample data, indicators such as accuracy and recall are calculated. Finally, the model parameters are adjusted according to the indicator data until each indicator meets the requirements. The model training process is simple and fast, and the accuracy and efficiency of the log classification model are superior to the log classification method based on rules and keywords. Therefore, the use of this abnormal log processing method can improve the efficiency of abnormal log classification. Moreover, the method can directly search for solutions that match the fault type of the abnormal log and push the optimal solution to the operation and maintenance personnel, so that the operation and maintenance personnel can quickly solve or handle faults that occur during system operation, thereby improving the efficiency of the operation and maintenance staff in resolving faults.
[0106] Please refer to FIG3 , which shows a schematic diagram of the structure of an abnormal log processing device provided by an embodiment of the present application. As shown in FIG3 , the device may include:
[0107] The log collection module 101 is used to collect abnormal logs generated during the operation of the business system.
[0108] The log management module 102 manages the collected exception logs, such as mainly log formatting and log classification.
[0109] The purpose of log formatting is to standardize the collected log data. The original log information will be sorted and organized according to a certain format to facilitate subsequent processing, storage and analysis.
[0110] Log classification is to classify log data according to different classification methods. The log classification of this module will be quickly classified by calling the log classification algorithm.
[0111] The log classification model 103 mainly uses training data to train the classification model, automatically learns and generates a model for log classification from the training data, and applies the learned model to unknown log data for classification, thereby achieving rapid log classification.
[0112] The solution library 104 is a knowledge base used to organize and store problem-solving methods, techniques, best practices, and experiences. Solutions are categorized and classified by multiple dimensions, such as problem type, technical field, and product module. This classification intersects and correlates with the log classification, providing a basis for solution recommendations.
[0113] The solution recommendation module 105 is used to quickly search for the optimal solution that matches the fault type in the abnormal log from the existing solutions in the solution library 104 and push it to the operation and maintenance personnel.
[0114] The exception log processing device provided in this embodiment collects exception logs during the operation of the system (microservice or application) through the log collection module and sends them to the log management module for further processing. After the log management module pre-processes the exception logs, it can quickly classify the exception logs based on the log classification model. The solution recommendation module further searches for solutions that match the classification category of the exception log from the pre-created solution library, and pushes the optimal solution to the operation and maintenance personnel, so that the operation and maintenance personnel can quickly resolve or handle faults that occur during the operation of the system, thereby improving the efficiency of the operation and maintenance staff in resolving faults.
[0115] Please refer to FIG4 , which shows a schematic diagram of the structure of another abnormal log processing device provided in an embodiment of the present application. As shown in FIG4 , the device can further include the following on the basis of the embodiment shown in FIG3 :
[0116] The fault classification system construction module 201 is used to construct a fault classification system.
[0117] A fault classification system categorizes faults according to specific rules and standards to facilitate better system management and maintenance. The classification system is constructed based on multiple dimensions, including but not limited to anomaly classification, performance classification, and configuration issue classification.
[0118] The data annotation module 202 is used to annotate historical abnormality log data to obtain fault sample data.
[0119] The fault type and annotation content of the fault sample data (i.e., historical abnormal log data) can be determined according to the constructed fault classification system to obtain the annotation data.
[0120] The classification model training module 203 is used to train the initial log classification model using fault sample data.
[0121] The labeled fault sample data is divided into a training set and a test set. Appropriate evaluation metrics are selected, and the established log classification model is trained using the training set. The specific training process may include: extracting the feature vector of the fault sample data and inputting it into the log classification model for classification to obtain a category prediction result. Then, based on the loss function, the accuracy, precision, and recall of the category prediction result and the labeled fault type corresponding to the fault sample data are calculated. If the metrics are not within the allowable range, the log classification model parameters are adjusted as needed until all metrics are within the allowable range, resulting in a usable log classification model.
[0122] The abnormal log processing device provided in this embodiment obtains fault sample data by annotating historical abnormal log data, further extracts the feature vector of the fault sample data, and inputs the feature vector into the initial log classification model, and uses the initial log classification model to classify the fault sample data to obtain the corresponding category label. Based on the categories obtained by the model and the categories annotated with the fault sample data, indicators such as accuracy and recall are calculated, and finally the model parameters are adjusted according to the indicator data until each indicator meets the requirements. The model training process is simple and fast, and the accuracy and efficiency of the log classification model are superior to the log classification method based on rules and keywords. Therefore, the use of this abnormal log processing method can improve the efficiency of abnormal log classification. Moreover, the method can directly search for solutions that match the fault type of the abnormal log and push the optimal solution to the operation and maintenance personnel, so that the operation and maintenance personnel can quickly solve or handle faults that occur during system operation, thereby improving the efficiency of the operation and maintenance staff in resolving faults.
[0123] The present application provides a computing device including a processor and a memory, wherein the memory stores a program executable on the processor. When the processor executes the program stored in the memory, the above-mentioned exception log processing method embodiment is implemented.
[0124] The present application also provides a storage medium executable by a computing device, wherein the storage medium stores a program, and when the program is executed by the computing device, the above-mentioned abnormal log processing method is implemented.
[0125] It should be noted that the technical features described in the various embodiments in this specification can be replaced or combined with each other, and can also be adjusted in order, merged, and deleted according to actual needs. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0126] The modules or submodules described as separate components may or may not be physically separate, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules may be selected to achieve the purpose of this embodiment according to actual needs.
[0127] In addition, each functional module or submodule in each embodiment of the present application may be integrated into a processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into a single module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or software functional modules or submodules.
[0128] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.
[0129] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for processing abnormal logs, characterized in that: include: Collect abnormal logs generated during the operation of business systems; Preprocess the abnormal logs; Based on the pre-trained log classification model, the pre-processed abnormal logs are classified to obtain the corresponding fault types; A pre-created solution library is searched to query a solution matching the fault type, wherein the solution library includes the fault type and an association relationship between the fault type and the corresponding solution.
2. The method according to claim 1, characterized in that The pre-trained log classification model is used to classify the pre-processed abnormal logs to obtain corresponding fault types, including: Extract the feature vector of the preprocessed abnormal log; The feature vector is input into a pre-trained log classification model for classification to obtain the fault type corresponding to the abnormal log.
3. The method according to claim 1, characterized in that The method further comprises: If the log classification model cannot obtain the fault type corresponding to the abnormal log, the log classification model is strengthened and trained based on the abnormal log.
4. The method according to any one of claims 1 to 3, characterized in that: The method further includes: pushing the searched solution to operation and maintenance personnel.
5. The method according to claim 4, characterized in that The method further comprises: Receive feedback from operation and maintenance personnel on the pushed solutions; If the feedback result is that the solution cannot solve the fault problem of the abnormal log, the log classification model is trained based on the abnormal log.
6. The method according to claim 1, characterized in that The preprocessing of the abnormal log includes: Stop words, missing values and abnormal values in the abnormal log are removed, and the abnormal log is formatted to obtain a standard format log.
7. The method according to any one of claims 1 to 3, characterized in that: The training process of the log classification model includes: Construct a fault classification system; Annotate historical abnormal logs to obtain fault sample data; Extracting a feature vector of the fault sample data, and inputting the feature vector into an initial log classification model for classification to obtain a fault category prediction result; Based on the fault category prediction result and the labeled category corresponding to the historical abnormal log, obtaining the performance index of the initial log classification model; If the performance index is not within the allowable range, the model parameters of the initial log classification model are adjusted, and the historical abnormal log data are classified using the adjusted log classification model to obtain a fault category prediction result, until the performance index is within the allowable range, and the log classification model is obtained.
8. An abnormal log processing device, characterized in that: include: The log collection module is used to collect abnormal logs generated during the operation of the business system; A log management module, used to pre-process the abnormal logs, and call the log classification model to classify the pre-processed abnormal logs to obtain corresponding fault categories; The solution recommendation module is used to search a pre-created solution library to query for solutions matching the fault category.
9. The device according to claim 8, characterized in that The solution recommendation module is also used to push the searched solutions to operation and maintenance personnel.
10. The device according to claim 8 or 9, characterized in that Also includes: The log classification model retraining module is used to receive feedback from operation and maintenance personnel on the pushed solutions; If the feedback result is that the solution cannot solve the fault problem of the abnormal log, the log classification model is trained based on the abnormal log.
Citation Information
Patent Citations
Multi-source data fault processing method and device, electronic equipment and storage medium
CN113190372A
Exception handling method, device and equipment for application program and storage medium
CN113626241A
Abnormal log processing method and device
CN117648214A
Identifying log anomaly resolution from anomalous system logs
US20230273849A1
Cited By
Fault processing method, electronic equipment and storage medium
CN120498979A
Operation and maintenance log self-learning analysis system and method based on large language model
CN120763484A
Log analysis method and electronic equipment
CN122451298A