Fault diagnosis method and device

Abstract

The present disclosure provides a fault diagnosis method and apparatus, comprising: determining alarm log data corresponding to a target alarm feature, and recalling the alarm log data to obtain suspected fault alarm log data; determining an alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to the alarm log template of a preset category from the suspected fault alarm log data; extracting the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information; performing secondary classification on the first fault alarm log data to generate a fault type, and determining first target fault alarm log data whose fault type is a preset fault type; and determining fault information based on the preset fault type and the first target fault alarm log data. This eliminates the need for manual analysis by operation and maintenance experts, thereby improving the efficiency of fault diagnosis.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a fault diagnosis method and device. Background Art

[0002] The content of alarm log data can reflect the operating status of the system. For diagnosing cache faults, existing techniques use fuzzy matching algorithms. This algorithm searches the log system for cache-related keywords and performs fuzzy matching on alarm logs from different time periods. Operations and maintenance personnel then locate the problem based on the core content of the alarm, similar information, expert experience, and received alarm records. However, this approach relies heavily on operations and maintenance experts, resulting in low diagnostic efficiency. Summary of the Invention

[0003] The present disclosure provides a method and apparatus for fault diagnosis, which eliminates the need for manual analysis by operation and maintenance experts and improves the efficiency of fault diagnosis.

[0004] In a first aspect, the present disclosure provides a fault diagnosis method, comprising:

[0005] Determine the alarm log data corresponding to the target alarm feature, and recall the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data;

[0006] Determining an alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data;

[0007] Extracting and processing the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information;

[0008] Performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is first target fault alarm log data of a preset fault type;

[0009] Fault information is determined based on the preset fault type and the first target fault alarm log data.

[0010] According to the fault diagnosis method provided by the present disclosure, after generating the fault type, the method further includes:

[0011] Determining second target fault alarm log data whose fault type is non-preset fault type, and merging and caching the second target fault alarm log data within a preset time period;

[0012] Inputting the merged cached second target fault alarm log data into a binary classification model for model training, and outputting the second target fault alarm log data corresponding to the non-downtime type fault;

[0013] Fault information is determined based on the second target fault alarm log data corresponding to the non-downtime type fault.

[0014] According to the fault diagnosis method provided by the present disclosure, before determining the alarm log data corresponding to the target alarm feature, the method includes:

[0015] Determining historical alarm log data, processing the historical alarm log data to generate a structured log template and determining an alarm log template corresponding to each piece of the historical alarm log data;

[0016] Acquire real-time alarm log data, match the real-time alarm log data with the historical alarm log data based on the structured log template, aggregate the matched real-time alarm log data, and generate structured online alarm log data;

[0017] Inputting the structured online alarm log data into an abnormal alarm scoring model to perform periodicity analysis and rarity analysis to obtain the alarm features corresponding to each structured online alarm log data;

[0018] Alarm log data corresponding to a target alarm feature is determined based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

[0019] According to the fault diagnosis method provided by the present disclosure, the alarm log data corresponding to the target alarm feature is recalled to obtain suspected fault alarm log data including:

[0020] Input the alarm log data corresponding to the target alarm feature into the alarm fault recall model to perform anomaly detection, and obtain the abnormal point and the alarm log data corresponding to the abnormal point;

[0021] The abnormal points are processed using a multi-layer abnormality detection voting mechanism to obtain a fault score for the abnormal points;

[0022] Sorting the fault scores of the abnormal points to obtain target abnormal points corresponding to the fault scores that meet preset conditions;

[0023] The alarm log data corresponding to the target abnormal point is used as suspected fault alarm log data.

[0024] According to the fault diagnosis method provided by the present disclosure, the preset category is the cache category;

[0025] Obtaining initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data includes:

[0026] Determine the suspected fault alarm log data corresponding to the cache class alarm log template in the suspected fault alarm log data by using a log template matching technology;

[0027] The suspected fault alarm log data corresponding to the cache class alarm log template is divided to obtain the authentication class suspected fault alarm log data, and the authentication class suspected fault alarm log data is used as the initial fault alarm log data.

[0028] According to the fault diagnosis method provided by the present disclosure, the extraction and processing of the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data includes:

[0029] The initial fault alarm log data is input into a named entity recognition model to generate basic information corresponding to the initial fault alarm log data, wherein the named entity recognition model is trained based on the initial fault alarm log sample data and the basic information labels corresponding to the initial fault alarm log sample data.

[0030] According to the fault diagnosis method provided by the present disclosure, after obtaining basic information corresponding to the initial fault alarm log data, the method further includes:

[0031] Determining second fault alarm log data based on the basic information;

[0032] An alarm application corresponding to the second fault alarm log data is determined, and a version upgrade process is performed on the alarm application so that the second fault alarm log data is converted into first fault alarm log data.

[0033] According to the fault diagnosis method provided by the present disclosure, the secondary classification of the first fault alarm log data to generate a fault type includes:

[0034] The first fault alarm log data is input into a multi-classification model to generate a fault type, wherein the multi-classification model is trained based on the first fault alarm log sample data and the fault type label corresponding to the first fault alarm log sample data.

[0035] In a second aspect, the present disclosure provides a fault diagnosis apparatus, comprising:

[0036] A processing module, configured to determine the alarm log data corresponding to the target alarm feature, perform recall processing on the alarm log data corresponding to the target alarm feature, and obtain suspected fault alarm log data;

[0037] an acquisition module, configured to determine an alarm log template corresponding to the suspected fault alarm log data, and acquire initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data;

[0038] an extraction module, configured to extract the initial fault alarm log data, obtain basic information corresponding to the initial fault alarm log data, and obtain first fault alarm log data based on the basic information;

[0039] a classification module, configured to perform secondary classification on the first fault alarm log data to determine first target fault alarm log data that meets a preset fault type;

[0040] A determination module is used to determine fault information based on the preset fault type and the first target fault alarm log data.

[0041] In a third aspect, the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the fault diagnosis method as described in any one of the above items when executing the program.

[0042] In a fourth aspect, the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the fault diagnosis method as described in any one of the above items.

[0043] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the fault diagnosis method as described in any one of the above items.

[0044] The present disclosure provides a fault diagnosis method and apparatus. The method comprises determining alarm log data corresponding to a target alarm feature, recalling the alarm log data corresponding to the target alarm feature, and obtaining suspected fault alarm log data. The method further comprises determining an alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to a preset alarm log template from the suspected fault alarm log data. This method reduces the amount of alarm log data required for troubleshooting. The method then directly extracts and processes the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtains first fault alarm log data based on the basic information. The method further comprises performing secondary classification on the first fault alarm log data to generate a fault type, determining that the fault type is first target fault alarm log data of a preset fault type, and determining fault information based on the preset fault type and the first target fault alarm log data. The method eliminates the need for manual analysis by operation and maintenance experts during the entire fault diagnosis process, directly analyzing and processing the alarm log data to ultimately diagnose the fault in the alarm log data, thereby improving diagnostic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is one of the flowcharts of a fault diagnosis method provided by an embodiment of the present disclosure;

[0047] Figure 2 This is a second flowchart of a fault diagnosis method provided by an embodiment of the present disclosure;

[0048] Figure 3 is a schematic diagram of the Bert binary classification model provided by an embodiment of the present disclosure;

[0049] Figure 4 is a schematic diagram of an abnormal alarm scoring model provided by an embodiment of the present disclosure;

[0050] Figure 5 is a schematic diagram of the structure of a named entity recognition model provided by an embodiment of the present disclosure;

[0051] Figure 6 It is a schematic diagram of the structure of the Bert multi-classification model provided by the embodiment of the present disclosure;

[0052] Figure 7is a block diagram of a fault diagnosis method provided by an embodiment of the present disclosure;

[0053] Figure 8 is a structural diagram of a fault diagnosis device provided by an embodiment of the present disclosure;

[0054] Figure 9 It is a structural diagram of the electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of this disclosure more clear, the technical solutions of this disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this disclosure, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of this disclosure without creative effort shall fall within the scope of protection of this disclosure.

[0056] Reference Figure 1 , which is one of the flowcharts of a fault diagnosis method provided by an embodiment of the present disclosure, including:

[0057] 110 , determining the alarm log data corresponding to the target alarm feature, and performing a recall process on the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data.

[0058] In this step, the alarm log data refers to the information data generated by the network device or application due to a certain failure, which may include the identification of the network device that generates the alarm log, the alarm type (used to indicate the failure of the network device) and the timestamp of the generation of the alarm log.

[0059] The target warning feature can be understood as an alarm feature with high abnormal sensitivity.

[0060] 120 , determining the alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to the alarm log template of a preset category from the suspected fault alarm log data.

[0061] In this step, the main function of the alarm log template is to extract the log template online from the existing alarm log in order to automatically adapt to various log formats. By extracting the log template, the logs can be clustered so that the same type of logs belong to the same template.

[0062] 130 , extracting and processing the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information.

[0063] In this step, basic information includes IP information and keyword information.

[0064] Correspondingly, extracting and processing the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data can be understood as extracting corresponding IP information and keyword information for each piece of initial fault alarm log data.

[0065] 140 , performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is first target fault alarm log data of a preset fault type.

[0066] In this step, secondary classification can be achieved by using the BERT multi-classification model. The preset fault types can be Client-Pool, Server-Refused, and Server-Reset.

[0067] 150. Determine fault information based on the preset fault type and the first target fault alarm log data.

[0068] In this step, when the preset fault types are Client-Pool, Server-Refused, and Server-Reset, the fault information can be clearly determined in the first target fault alarm log data.

[0069] The present disclosure provides a fault diagnosis method that determines alarm log data corresponding to a target alarm feature, recalls the alarm log data corresponding to the target alarm feature, and obtains suspected fault alarm log data; determines an alarm log template corresponding to the suspected fault alarm log data, and obtains initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data. This method can reduce the amount of alarm log data that needs to be analyzed during fault troubleshooting. Subsequently, the initial fault alarm log data is directly extracted and processed to generate basic information corresponding to the initial fault alarm log data, and first fault alarm log data is obtained based on the basic information; the first fault alarm log data is secondary classified to generate a fault type, and the fault type is determined to be a first target fault alarm log data of a preset fault type; and fault information is determined based on the preset fault type and the first target fault alarm log data. During the entire fault diagnosis process, there is no need to rely on manual analysis by operation and maintenance experts. The alarm log data is directly analyzed and processed to ultimately diagnose the fault in the alarm log data, thereby improving diagnostic efficiency.

[0070] Reference Figure 2 , which is a second flow chart of a fault diagnosis method provided by an embodiment of the present disclosure, including:

[0071] 210 , determining second target fault alarm log data whose fault type is a non-preset fault type, and merging and caching the second target fault alarm log data within a preset time period.

[0072] In this step, the non-preset fault type can be Unknown. The preset time period can be set to 2 minutes. It is understood that the preset content in the embodiment of the present disclosure can be set according to the specific application scenario, and the present disclosure is not limited to this.

[0073] 220 , input the merged and cached second target fault alarm log data into a binary classification model for model training, and output the second target fault alarm log data corresponding to the non-downtime type fault.

[0074] In this step, the binary classification model can be a Bert binary classification model. Figure 3 , which is a schematic diagram of the Bert binary classification model provided in an embodiment of the present disclosure.

[0075] In the figure, [CLS] indicates the start flag, Tok refers to Token, which is the identifier preceding the request data, None indicates None, and Softmax indicates the normalized exponential function. In the figure, the fault type is downtime, and cache merging is performed from the alarm log data Java.net.SocketTimeoutExcepttion:Readtimedout10.221.109.69:5032---MOVED that does not carry node information to the alarm log data Connectionproblem:172.25.239.189:5092java.net.SocketTimeException:Read time out that does not carry node information.

[0076] Specifically, the second target fault alarm log data is merged and cached with a 2-minute window, and the merged and cached second target fault alarm log data is input into the Bert binary classification model. The output results include the second target fault alarm log data corresponding to non-downtime type faults and the second target fault alarm log data corresponding to downtime type faults.

[0077] Specifically, the second target fault alarm log data corresponding to the downtime type fault can be processed through an automated process.

[0078] 230 : Determine fault information based on the second target fault alarm log data corresponding to the non-downtime type fault.

[0079] In this step, the second target fault alarm log data corresponding to the non-downtime type fault can directly specify the specific fault information.

[0080] Based on any of the above embodiments, before step 110, the following steps 111 to 114 are included:

[0081] Step 111 : determine historical alarm log data, process the historical alarm log data to generate a structured log template, and determine an alarm log template corresponding to each piece of the historical alarm log data.

[0082] In this step, a large amount of historical alarm log data is first determined, these historical alarm log data are parsed and processed, and structured log templates are trained and generated to determine the alarm log template corresponding to each historical alarm log data.

[0083] Step 112: acquiring real-time alarm log data, matching the real-time alarm log data with the historical alarm log data based on the structured log template, aggregating the matched real-time alarm log data, and generating structured online alarm log data.

[0084] In this step, the log template generated by historical alarm log template training is matched with the real-time alarm log data, so that the real-time alarm log data is matched to the corresponding alarm log template. After the matching is completed, the real-time alarm log is aggregated to generate structured online alarm log data.

[0085] Step 113: Input the structured online alarm log data into an abnormal alarm scoring model to perform periodicity analysis and rarity analysis, and obtain the alarm feature corresponding to each structured online alarm log data.

[0086] In this step, structured online alarm log data is fed into an abnormal alarm scoring model for periodicity and rarity analysis. Rare and non-periodic alarms are assigned high anomaly scores, while periodic and high-volume routine alarms have their anomaly scores reduced. The anomaly levels of different alarms are described and weighted. The model outputs the alarm signature for each structured online alarm log data entry, which serves as a weight for the subsequent alarm fault recall model.

[0087] Specifically, refer to Figure 4 This is a schematic diagram of the abnormal alarm scoring model provided by an embodiment of the present disclosure. The diagram includes some specific content from the alarm log data, specifically column names, types, and descriptions. For example, the column name created_date, type string, describes the creation time; the column name app_name, type string, describes the application name; the column name alarm_app_name, type string, describes the alarm application name; and the column name theme, type string, describes the theme.

[0088] Step 114 : determining alarm log data corresponding to a target alarm feature based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

[0089] In this step, the target alarm feature can be understood as an alarm feature with high abnormal sensitivity, specifically a feature with a high abnormality score and obvious time series and state features.

[0090] Based on any of the above embodiments, step 110 specifically includes the following steps 1 to 4:

[0091] Step 1: Input the alarm log data corresponding to the target alarm feature into the alarm fault recall model to perform anomaly detection, and obtain anomaly points and the alarm log data corresponding to the anomaly points.

[0092] In this step, the alarm fault recall model can be a multi-layer unsupervised anomaly detection model, and the multi-layer unsupervised anomaly detection model consists of four models: local anomaly factor detection (LOF), isolation forest (IForest), nearest neighbor node algorithm (KNN) and angle-based outlier detection algorithm (ABOD).

[0093] Step 2: Process the abnormal point using a multi-layer abnormality detection voting mechanism to obtain a fault score for the abnormal point.

[0094] In this step, the multi-layer anomaly detection voting mechanism is used to process the anomaly point, including first performing multi-layer anomaly detection voting and then performing multi-dimensional weighted fault scoring, thereby obtaining the fault score of the anomaly point.

[0095] Step 3: sort the fault scores of the abnormal points to obtain target abnormal points corresponding to the fault scores that meet preset conditions.

[0096] In this step, the fault scores of the anomalies are arranged in ascending or descending order, and preset conditions are set according to the specific application scenario. For example, the first three anomalies with higher fault scores can be obtained and these three anomalies can be used as target anomalies.

[0097] Step 4: Use the alarm log data corresponding to the target abnormal point as suspected fault alarm log data.

[0098] In this step, anomaly detection is performed using the alarm fault recall model. The obtained results include two types of data: suspected fault alarm log data and non-fault alarm log data. The non-fault alarm log data is not processed.

[0099] Based on any of the above embodiments, the preset category is a cache category;

[0100] The step 120 specifically includes the following steps 121 to 122:

[0101] Step 121 : Determine the suspected fault alarm log data corresponding to the cache alarm log template in the suspected fault alarm log data by using a log template matching technology.

[0102] In this step, the log template matching technology can be implemented through the FT-Tree & label tree matching technology.

[0103] Step 122 : divide the suspected fault alarm log data corresponding to the cache class alarm log template, obtain authentication class suspected fault alarm log data, and use the authentication class suspected fault alarm log data as initial fault alarm log data.

[0104] In this step, the suspected fault alarm log data corresponding to the cache alarm log template is divided into authentication suspected fault alarm log data and non-authentication suspected fault alarm log data. The non-authentication suspected fault alarm log data is not processed.

[0105] The present disclosure uses log template matching technology to identify cache alarm log templates and distinguish them into authentication-type suspected fault alarm log data and non-authentication-type suspected fault alarm log data, thereby reducing the amount of logs that need to be analyzed during problem troubleshooting and directly locking non-authentication-type suspected fault alarm log data.

[0106] Based on any of the above embodiments, step 130 specifically includes:

[0107] The initial fault alarm log data is input into a named entity recognition model to generate basic information corresponding to the initial fault alarm log data, wherein the named entity recognition model is trained based on the initial fault alarm log sample data and the basic information labels corresponding to the initial fault alarm log sample data.

[0108] In this step, refer to Figure 5 , which is a schematic diagram of the structure of the named entity recognition model provided by the embodiments of the present disclosure. In this figure, BiLSTM and CRF represent two different layers in the named entity recognition model, B, I, and O represent entity labels, B-IP represents IP information, B-Keywords represents keyword information, and w0, w1, w2, and w3 represent the four parts of the input data.

[0109] The authentication suspected fault alarm log data (initial fault alarm log data) is input into the named entity recognition model (BiLSTM-CRF) to generate the corresponding basic information, which includes IP information and keyword information.

[0110] Compared with the rule-based method, the named entity recognition method has faster calculation speed, higher coverage and higher algorithm accuracy.

[0111] Based on any of the above embodiments, first fault alarm log data and second fault alarm log data can be obtained based on basic information. The first fault alarm log data refers to fault alarm log data carrying node information.

[0112] Based on any of the above embodiments, after obtaining basic information corresponding to the initial fault alarm log data in step 130, the method further includes:

[0113] Second fault alarm log data is determined based on the basic information.

[0114] Specifically, the second fault alarm log data refers to fault alarm log data that does not carry node information.

[0115] An alarm application corresponding to the second fault alarm log data is determined, and a version upgrade process is performed on the alarm application so that the second fault alarm log data is converted into first fault alarm log data.

[0116] Specifically, for fault alarm log data that does not carry node information, obtain the alarm application corresponding to the data and the version corresponding to the alarm application, promote research and development to upgrade the version, complete the node information in the alarm log data, and convert the fault alarm log data that does not carry node information into fault alarm log data that carries node information.

[0117] Based on any of the above embodiments, step 140 specifically includes:

[0118] The first fault alarm log data is input into a multi-classification model to generate a fault type, wherein the multi-classification model is trained based on the first fault alarm log sample data and the fault type label corresponding to the first fault alarm log sample data.

[0119] Specifically, refer to Figure 6 , which is a structural diagram of the Bert multi-classification model provided in an embodiment of the present disclosure.

[0120] In this figure, the fault type is Server-Refused, Softmax represents the normalized exponential function, and Java.net.SocketTimeoutExcepttion refused represents fault alarm log data that carries node information.

[0121] The first fault alarm log (fault alarm log data carrying node information) is input into the Bert multi-classification model, and the fault alarm log data carrying node information and the corresponding fault type are output.

[0122] Fault types include Client-Pool, Server-Refused, Server-Reset, and Unknown.

[0123] Further, the following supplementary explanations are given for the embodiments of the present disclosure:

[0124] Reference Figure 7 , is a block diagram of the fault diagnosis method provided by the embodiment of the present disclosure, which includes an alarm feature system construction module and an alarm fault recall model module. The alarm feature system construction module mainly pre-processes historical alarm log data to obtain alarm features. Specifically, it includes the following steps (1)-(4):

[0125] (1) Determine historical alarm log data, process the historical alarm log data to generate a structured log template, and determine the alarm log template corresponding to each historical alarm log data.

[0126] (2) Obtaining real-time alarm log data, matching the real-time alarm log data with the historical alarm log data based on the structured log template, and aggregating the matched real-time alarm log data to generate structured online alarm log data.

[0127] (3) Input the structured online alarm log data into the abnormal alarm scoring model for periodicity analysis and rarity analysis to obtain the alarm features corresponding to each structured online alarm log data.

[0128] (4) Determine the alarm log data corresponding to the alarm feature with high abnormal sensitivity based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

[0129] The alarm fault recall model module mainly inputs the alarm log data corresponding to the alarm features with high abnormal sensitivity into the alarm fault recall model based on the alarm features, and outputs the suspected fault alarm log data and non-fault alarm log data. Specifically including (5)-(8):

[0130] (5) Input the alarm log data corresponding to the alarm features with high abnormal sensitivity into the alarm fault recall model for abnormality detection, and obtain the abnormal points and the alarm log data corresponding to the abnormal points.

[0131] (6) The multi-layer anomaly detection voting mechanism is used to first perform multi-layer anomaly detection voting on the anomaly point, and then perform multi-dimensional weighted fault scoring processing to obtain the fault score of the anomaly point.

[0132] (7) Sort the fault scores of the abnormal points to obtain the target abnormal points corresponding to the fault scores that meet the preset conditions.

[0133] (8) The alarm log data corresponding to the target abnormal point is used as the suspected fault alarm log data.

[0134] After obtaining the suspected fault alarm log data, the suspected fault alarm log data is subsequently processed, specifically including steps (9) to (15):

[0135] (9) Using log template matching technology, the suspected fault alarm log data corresponding to the cache alarm log template is determined in the suspected fault alarm log data.

[0136] (10) The suspected fault alarm log data corresponding to the cache class alarm log template is divided to obtain the authentication class suspected fault alarm log data and the non-authentication class suspected fault alarm log data.

[0137] (11) The non-authentication suspected fault alarm log data is input into the named entity recognition model, and the IP information and keyword information in each non-authentication alarm log data are extracted. Then, the non-authentication alarm log data is divided into fault alarm log data carrying node information and fault alarm log data not carrying node information.

[0138] (12) The fault alarm log data carrying node information is input into the Bert multi-classification model to generate fault alarm log data corresponding to the fault types of Client-Pool, Server-Refused, Server-Reset, and Unknown.

[0139] (13) The fault alarm log data corresponding to the Client-Pool, Server-Refused, and Server-Reset types can clearly identify the specific fault information.

[0140] (14) For the fault alarm log data corresponding to the Unknown type, the second target fault alarm log data is merged and cached with a window of 2 minutes, and the merged and cached second target fault alarm log data is input into the Bert binary classification model. The output results include the second target fault alarm log data corresponding to non-downtime type faults and the second target fault alarm log data corresponding to downtime type faults.

[0141] (15) The second target fault alarm log data corresponding to the downtime type fault can be processed through an automated process; the second target fault alarm log data corresponding to the non-downtime type fault can directly clarify the specific fault information.

[0142] The fault diagnosis method provided by the present disclosure does not need to rely on manual analysis by operation and maintenance experts during the entire fault diagnosis process. It directly analyzes and processes the alarm log data and finally diagnoses the fault in the alarm log data, thereby improving the efficiency of diagnosis.

[0143] The fault diagnosis device provided by the present disclosure is described below. The fault diagnosis device described below and the fault diagnosis method described above can be referenced to each other.

[0144] Reference Figure 8 , which is a structural diagram of a fault diagnosis device provided by an embodiment of the present disclosure, including:

[0145] The processing module 810 is configured to determine the alarm log data corresponding to the target alarm feature, and perform recall processing on the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data;

[0146] An acquisition module 820 is configured to determine an alarm log template corresponding to the suspected fault alarm log data, and acquire initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data;

[0147] An extraction module 830 is configured to extract the initial fault alarm log data, obtain basic information corresponding to the initial fault alarm log data, and obtain first fault alarm log data based on the basic information;

[0148] A classification module 840 is configured to perform secondary classification on the first fault alarm log data to determine first target fault alarm log data that meets a preset fault type;

[0149] The determination module 850 is configured to determine fault information based on the preset fault type and the first target fault alarm log data.

[0150] Based on any of the above embodiments, the device further includes:

[0151] a merging and caching module, configured to determine second target fault alarm log data whose fault type is not a preset fault type, and merge and cache the second target fault alarm log data within a preset time period;

[0152] A model training module is used to input the second target fault alarm log data after the merged cache processing into a binary classification model for model training, and output the second target fault alarm log data corresponding to the non-downtime type fault;

[0153] The non-downtime type fault determination module is configured to determine fault information based on second target fault alarm log data corresponding to the non-downtime type fault.

[0154] Based on any of the above embodiments, before processing module 810, the following steps are included:

[0155] A generating unit, configured to determine historical alarm log data, process the historical alarm log data to generate a structured log template, and determine an alarm log template corresponding to each piece of the historical alarm log data;

[0156] a matching unit, configured to obtain real-time alarm log data, match the real-time alarm log data with the historical alarm log data based on the structured log template, aggregate the matched real-time alarm log data, and generate structured online alarm log data;

[0157] An analysis unit, configured to input the structured online alarm log data into an abnormal alarm scoring model to perform periodicity analysis and rarity analysis, and obtain an alarm feature corresponding to each structured online alarm log data;

[0158] The first determining unit is configured to determine alarm log data corresponding to a target alarm feature based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

[0159] Based on any of the above embodiments, the processing module 810 is specifically configured to:

[0160] Input the alarm log data corresponding to the target alarm feature into the alarm fault recall model to perform anomaly detection, and obtain the abnormal point and the alarm log data corresponding to the abnormal point;

[0161] The abnormal points are processed using a multi-layer abnormality detection voting mechanism to obtain a fault score for the abnormal points;

[0162] Sorting the fault scores of the abnormal points to obtain target abnormal points corresponding to the fault scores that meet preset conditions;

[0163] The alarm log data corresponding to the target abnormal point is used as suspected fault alarm log data.

[0164] Based on any of the above embodiments, the preset category is a cache category;

[0165] The acquisition module 820 is specifically configured to:

[0166] Determine the suspected fault alarm log data corresponding to the cache class alarm log template in the suspected fault alarm log data by using a log template matching technology;

[0167] The suspected fault alarm log data corresponding to the cache class alarm log template is divided to obtain the authentication class suspected fault alarm log data, and the authentication class suspected fault alarm log data is used as the initial fault alarm log data.

[0168] Based on any of the above embodiments, the extraction module 830 is specifically configured to:

[0169] The initial fault alarm log data is input into a named entity recognition model to generate basic information corresponding to the initial fault alarm log data, wherein the named entity recognition model is trained based on the initial fault alarm log sample data and the basic information labels corresponding to the initial fault alarm log sample data.

[0170] Based on any of the above embodiments, after the extraction module 830, the apparatus further includes:

[0171] A second determining unit, configured to determine second fault alarm log data based on the basic information;

[0172] The third determining unit is configured to determine an alarm application corresponding to the second fault alarm log data, and perform version upgrade processing on the alarm application so as to convert the second fault alarm log data into first fault alarm log data.

[0173] Based on any of the above embodiments, the classification module 840 is specifically configured to:

[0174] The first fault alarm log data is input into a multi-classification model to generate a fault type, wherein the multi-classification model is trained based on the first fault alarm log sample data and the fault type label corresponding to the first fault alarm log sample data.

[0175] The present disclosure provides a fault diagnosis device that determines alarm log data corresponding to a target alarm feature, recalls the alarm log data corresponding to the target alarm feature, and obtains suspected fault alarm log data; determines an alarm log template corresponding to the suspected fault alarm log data, and obtains initial fault alarm log data corresponding to a preset alarm log template from the suspected fault alarm log data. This reduces the amount of alarm log data that needs to be analyzed during fault troubleshooting. Subsequently, the initial fault alarm log data is directly extracted and processed to generate basic information corresponding to the initial fault alarm log data, and first fault alarm log data is obtained based on the basic information; the first fault alarm log data is secondary classified to generate a fault type, and the fault type is determined to be a first target fault alarm log data of a preset fault type; and fault information is determined based on the preset fault type and the first target fault alarm log data. During the entire fault diagnosis process, there is no need to rely on manual analysis by operation and maintenance experts. The alarm log data is directly analyzed and processed, and the fault is ultimately diagnosed in the alarm log data, thereby improving diagnostic efficiency.

[0176] Figure 9 An example of a physical structure diagram of an electronic device is shown below. Figure 9 As shown, the electronic device may include: a processor (processor) 910, a communication interface (Communications Interface) 920, a memory (memory) 930 and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call the logic instructions in the memory 930 to execute a fault diagnosis method, including: determining the alarm log data corresponding to the target alarm feature, recalling the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data; determining the alarm log template corresponding to the suspected fault alarm log data, and obtaining the initial fault alarm log data corresponding to the alarm log template of a preset category in the suspected fault alarm log data; extracting the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information; performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is the first target fault alarm log data of a preset fault type; and determining fault information based on the preset fault type and the first target fault alarm log data.

[0177] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0178] On the other hand, the present disclosure also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a fault diagnosis method provided by the above methods, including: determining the alarm log data corresponding to the target alarm feature, recalling the alarm log data corresponding to the target alarm feature, and obtaining suspected fault alarm log data; determining the alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to the alarm log template of a preset category in the suspected fault alarm log data; extracting the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information; performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is the first target fault alarm log data of a preset fault type; determining fault information based on the preset fault type and the first target fault alarm log data.

[0179] On the other hand, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a fault diagnosis method provided by the above-mentioned methods, including: determining the alarm log data corresponding to the target alarm feature, recalling the alarm log data corresponding to the target alarm feature, and obtaining suspected fault alarm log data; determining the alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to the alarm log template of a preset category in the suspected fault alarm log data; extracting the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information; performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is the first target fault alarm log data of a preset fault type; and determining fault information based on the preset fault type and the first target fault alarm log data.

[0180] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A fault diagnosis method, characterized in that: include: Determine the alarm log data corresponding to the target alarm feature, and recall the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data; Determining an alarm log template corresponding to the suspected fault alarm log data, and obtaining initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data; Extracting and processing the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data, and obtaining first fault alarm log data based on the basic information; Performing secondary classification on the first fault alarm log data to generate a fault type, and determining that the fault type is first target fault alarm log data of a preset fault type; Determining fault information based on the preset fault type and the first target fault alarm log data; Before determining the alarm log data corresponding to the target alarm feature, the method includes: Determining historical alarm log data, processing the historical alarm log data to generate a structured log template and determining an alarm log template corresponding to each piece of the historical alarm log data; Acquire real-time alarm log data, match the real-time alarm log data with the historical alarm log data based on the structured log template, aggregate the matched real-time alarm log data, and generate structured online alarm log data; Inputting the structured online alarm log data into an abnormal alarm scoring model to perform periodicity analysis and rarity analysis to obtain the alarm features corresponding to each structured online alarm log data; Alarm log data corresponding to a target alarm feature is determined based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

2. The fault diagnosis method according to claim 1, characterized in that: After generating the fault type, the method further includes: Determining second target fault alarm log data whose fault type is non-preset fault type, and merging and caching the second target fault alarm log data within a preset time period; Inputting the merged cached second target fault alarm log data into a binary classification model for model training, and outputting the second target fault alarm log data corresponding to the non-downtime type fault; Fault information is determined based on the second target fault alarm log data corresponding to the non-downtime type fault.

3. The fault diagnosis method according to claim 1, characterized in that: Recall the alarm log data corresponding to the target alarm feature to obtain suspected fault alarm log data including: Input the alarm log data corresponding to the target alarm feature into the alarm fault recall model to perform anomaly detection, and obtain the abnormal point and the alarm log data corresponding to the abnormal point; The abnormal points are processed using a multi-layer abnormality detection voting mechanism to obtain a fault score for the abnormal points; Sorting the fault scores of the abnormal points to obtain target abnormal points corresponding to the fault scores that meet preset conditions; The alarm log data corresponding to the target abnormal point is used as suspected fault alarm log data.

4. The fault diagnosis method according to claim 1, characterized in that: The preset category is the cache category; Obtaining initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data includes: Determine the suspected fault alarm log data corresponding to the cache class alarm log template in the suspected fault alarm log data by using a log template matching technology; The suspected fault alarm log data corresponding to the cache class alarm log template is divided to obtain the authentication class suspected fault alarm log data, and the authentication class suspected fault alarm log data is used as the initial fault alarm log data.

5. The fault diagnosis method according to claim 4, characterized in that: The extracting and processing of the initial fault alarm log data to generate basic information corresponding to the initial fault alarm log data includes: The initial fault alarm log data is input into a named entity recognition model to generate basic information corresponding to the initial fault alarm log data, wherein the named entity recognition model is trained based on the initial fault alarm log sample data and the basic information labels corresponding to the initial fault alarm log sample data.

6. The fault diagnosis method according to claim 5, characterized in that: After obtaining basic information corresponding to the initial fault alarm log data, the method further includes: Determining second fault alarm log data based on the basic information; An alarm application corresponding to the second fault alarm log data is determined, and a version upgrade process is performed on the alarm application so that the second fault alarm log data is converted into first fault alarm log data.

7. The fault diagnosis method according to claim 6, characterized in that: The secondary classification of the first fault alarm log data to generate a fault type includes: The first fault alarm log data is input into a multi-classification model to generate a fault type, wherein the multi-classification model is trained based on the first fault alarm log sample data and the fault type label corresponding to the first fault alarm log sample data.

8. A fault diagnosis device, characterized in that: include: A processing module, configured to determine the alarm log data corresponding to the target alarm feature, perform recall processing on the alarm log data corresponding to the target alarm feature, and obtain suspected fault alarm log data; an acquisition module, configured to determine an alarm log template corresponding to the suspected fault alarm log data, and acquire initial fault alarm log data corresponding to an alarm log template of a preset category from the suspected fault alarm log data; an extraction module, configured to extract the initial fault alarm log data, obtain basic information corresponding to the initial fault alarm log data, and obtain first fault alarm log data based on the basic information; a classification module, configured to perform secondary classification on the first fault alarm log data to determine first target fault alarm log data that meets a preset fault type; a determination module, configured to determine fault information based on the preset fault type and the first target fault alarm log data; Before processing the module, include: A generating unit, configured to determine historical alarm log data, process the historical alarm log data to generate a structured log template, and determine an alarm log template corresponding to each piece of the historical alarm log data; a matching unit, configured to obtain real-time alarm log data, match the real-time alarm log data with the historical alarm log data based on the structured log template, aggregate the matched real-time alarm log data, and generate structured online alarm log data; An analysis unit, configured to input the structured online alarm log data into an abnormal alarm scoring model to perform periodicity analysis and rarity analysis, and obtain an alarm feature corresponding to each structured online alarm log data; The first determining unit is configured to determine alarm log data corresponding to a target alarm feature based on the alarm feature, wherein the alarm feature includes an abnormality score feature, a time series feature, and a state feature.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the fault diagnosis method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the fault diagnosis method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the fault diagnosis method according to any one of claims 1 to 7 are implemented.

Images