Fault root cause positioning method and device based on machine learning, equipment and medium

Abstract

The invention particularly relates to a fault root cause positioning method and device based on machine learning, equipment and a medium, and the method comprises the steps: if an obtained data monitoring dynamic threshold value of an operation and maintenance platform within a first preset time, data monitoring dynamic threshold values in different business scenes have first alarm information and / or log data have second alarm information, determining that the data monitoring dynamic threshold values have first alarm information; if yes, obtaining fault data information of the first alarm information and / or the second alarm information within a second preset time, starting a fault root cause positioning model, and constructing a data monitoring dynamic threshold value, a data monitoring dynamic threshold value under different service scenes and a correlation analysis link of log data; and generating an abnormal link dependency relationship of the operation and maintenance platform in combination with the fault data information, and then performing fault propagation path reverse tracking from a fault starting node to position a fault source of the operation and maintenance platform. Therefore, the problems of false information report, missing report, low efficiency, low accuracy and the like caused by fault root cause positioning based on a fixed threshold value and / or artificial experience are solved.

Description

Technical Field

[0001] The present application relates to the field of information fault management technology, and in particular to a method, device, equipment and medium for locating the root cause of a fault based on machine learning. Background Art

[0002] In the modern information technology environment, the complexity and scale of systems continue to expand, placing higher demands on fault management and operation and maintenance of machine learning. Therefore, accurate root cause location and recovery are very necessary.

[0003] In related technologies, fault root cause location and recovery usually rely on fixed thresholds and manual experience judgment.

[0004] However, judgment based on fixed thresholds and manual experience has the following disadvantages: (1) Fixed thresholds cannot adapt to changing business needs and are prone to generating a large number of false positives or omissions; (2) Root cause location of faults is highly dependent on manual experience, which can easily miss important abnormal information, is inefficient, and prone to errors; (3) It is difficult to effectively filter out noise information in a large amount of monitoring data, which affects the timeliness and accuracy of fault handling and needs to be solved urgently. Summary of the Invention

[0005] The present application provides a method, apparatus, device, and medium for locating the root cause of a fault based on machine learning to address issues such as false positives, missed positives, low efficiency, and low accuracy that may arise from locating the root cause of a fault based on fixed thresholds and / or manual experience.

[0006] The first embodiment of the present application provides a method for locating the root cause of a fault based on machine learning, comprising the following steps:

[0007] Obtaining data monitoring dynamic thresholds of the operation and maintenance platform within a first preset time, data monitoring dynamic thresholds under different business scenarios within the first preset time, and log data within the first preset time;

[0008] If the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios has a first alarm information, and / or the log data has a second alarm information, then obtain the fault data information of the first alarm information and / or the second alarm information within the second preset time, and start the fault root cause location model;

[0009] Construct the data monitoring dynamic threshold, the data monitoring dynamic threshold under the different business scenarios and the correlation analysis link of the log data, and generate the abnormal link dependency of the operation and maintenance platform based on the fault data information and the correlation analysis link, and based on the abnormal link dependency, reversely trace the fault propagation path from the fault starting node to locate the fault source of the operation and maintenance platform.

[0010] According to one embodiment of the present application, before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, the method further includes:

[0011] Obtaining historical performance data and historical business characteristic data of the operation and maintenance platform;

[0012] Performing self-learning on the historical performance data and the historical service feature data based on a preset machine learning algorithm to obtain a dynamic threshold value of historical data of the operation and maintenance platform;

[0013] Cluster analysis is performed on the historical data dynamic thresholds to obtain historical data dynamic thresholds under different business scenarios.

[0014] According to one embodiment of the present application, after cluster analysis is performed on the historical data dynamic thresholds to obtain the historical data dynamic thresholds under different business scenarios, the method further includes:

[0015] Based on the historical data dynamic threshold, the preset machine learning algorithm is used to extract the time series characteristics of the historical data dynamic threshold, and based on the time series characteristics and the historical data dynamic threshold, the data monitoring dynamic threshold of the operation and maintenance platform within a first preset time and the data monitoring dynamic threshold under the different business scenarios are generated;

[0016] Obtain historical log data of the operation and maintenance platform, use preset language processing technology to extract log features in the historical log data, train according to the log features to obtain non-abnormal log data and abnormal log data, and obtain the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data.

[0017] According to one embodiment of the present application, after obtaining the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data, the method further includes:

[0018] Determining whether the data monitoring dynamic threshold within the first preset time matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold under different business scenarios within the first preset time matches the historical data dynamic threshold under the different business scenarios;

[0019] If the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios within the first preset time does not match the historical data dynamic threshold under the different business scenarios, generating the first alarm information;

[0020] If it is determined based on the historical log database that the log data within the first preset time belongs to the abnormal log data, the second alarm information is generated.

[0021] According to one embodiment of the present application, before determining, based on the historical log database, that the log data within the first preset time period belongs to the abnormal log data, the method further includes:

[0022] The log data within the first preset time is converted into a log feature vector, and the log feature vector is clustered and analyzed. The cluster analysis result is input into the historical log database, and it is determined whether the log data within the first preset time belongs to the abnormal log data based on the output result.

[0023] According to one embodiment of the present application, before starting the fault root cause location model, the method further includes:

[0024] performing data preprocessing on the first alarm information and the second alarm information;

[0025] The pre-processed first alarm information and the second alarm information are subjected to alarm noise reduction by using a preset data processing mechanism to obtain valid alarm information in the first alarm information and the second alarm information, and the fault root cause location model is started based on the valid alarm information.

[0026] According to the machine learning-based fault root cause location method of the embodiment of the present application, if the data monitoring dynamic threshold value obtained for the operation and maintenance platform within the first preset time, the data monitoring dynamic threshold value under different business scenarios has a first alarm information and / or the log data has a second alarm information, then the fault data information of the first alarm information and / or the second alarm information within the second preset time is obtained, and the fault root cause location model is started, and the correlation analysis link of the data monitoring dynamic threshold value, the data monitoring dynamic threshold value under different business scenarios and the log data is constructed, and the abnormal link dependency relationship of the operation and maintenance platform is generated in combination with the fault data information, and then the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform. In this way, the problems of false alarms, missed alarms, low efficiency, low accuracy, etc. that exist in fault root cause location due to fixed thresholds and / or manual experience are solved.

[0027] A second embodiment of the present application provides a device for locating a root cause of a fault based on machine learning, comprising:

[0028] A first acquisition module is used to obtain the data monitoring dynamic threshold value of the operation and maintenance platform within a first preset time, the data monitoring dynamic threshold value under different business scenarios within the first preset time, and the log data within the first preset time;

[0029] A second acquisition module is used to obtain fault data information of the first alarm information and / or the second alarm information within a second preset time if the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios has a first alarm information, and / or the log data has a second alarm information, and start the fault root cause location model;

[0030] A fault location module is used to construct the data monitoring dynamic threshold, the data monitoring dynamic threshold under the different business scenarios and the correlation analysis link of the log data, and generate the abnormal link dependency of the operation and maintenance platform based on the fault data information and the correlation analysis link, and based on the abnormal link dependency, reversely trace the fault propagation path from the fault starting node to locate the fault source of the operation and maintenance platform.

[0031] According to one embodiment of the present application, before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, the first obtaining module further includes:

[0032] An acquisition unit, configured to acquire historical performance data and historical service feature data of the operation and maintenance platform;

[0033] A self-learning unit, configured to perform self-learning on the historical performance data and the historical service feature data based on a preset machine learning algorithm to obtain a dynamic threshold value of historical data of the operation and maintenance platform;

[0034] The cluster analysis unit is used to perform cluster analysis on the historical data dynamic threshold value to obtain the historical data dynamic threshold value under different business scenarios.

[0035] According to one embodiment of the present application, after performing cluster analysis on the historical data dynamic thresholds to obtain the historical data dynamic thresholds under different business scenarios, the cluster analysis unit further includes:

[0036] a generating subunit, configured to extract, based on the historical data dynamic threshold, a time series feature of the historical data dynamic threshold using the preset machine learning algorithm, and generate, based on the time series feature and the historical data dynamic threshold, a data monitoring dynamic threshold of the operation and maintenance platform within a first preset time and a data monitoring dynamic threshold under the different business scenarios;

[0037] An acquisition subunit is used to obtain historical log data of the operation and maintenance platform, use a preset language processing technology to extract log features in the historical log data, train according to the log features to obtain non-abnormal log data and abnormal log data, and obtain a historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data.

[0038] According to one embodiment of the present application, after obtaining the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data, the obtaining subunit further includes:

[0039] a judgment component, configured to judge whether the data monitoring dynamic threshold within the first preset time matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold under different business scenarios within the first preset time matches the historical data dynamic threshold under the different business scenarios;

[0040] a first generating component, configured to generate the first alarm information if the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios within the first preset time does not match the historical data dynamic threshold under the different business scenarios;

[0041] The second generating component is configured to generate the second alarm information if it is determined based on the historical log database that the log data within the first preset time period belongs to the abnormal log data.

[0042] According to one embodiment of the present application, before determining, based on the historical log database, that the log data within the first preset time period belongs to the abnormal log data, the second generating component further includes:

[0043] A cluster analysis sub-component is used to convert the log data within the first preset time into a log feature vector, perform cluster analysis on the log feature vector, input the cluster analysis results into the historical log database, and determine whether the log data within the first preset time belongs to the abnormal log data based on the output results.

[0044] According to one embodiment of the present application, before starting the fault root cause location model, the second acquisition module further includes:

[0045] a data processing unit, configured to perform data preprocessing on the first alarm information and the second alarm information;

[0046] An alarm denoising unit is used to perform alarm denoising on the preprocessed first alarm information and the second alarm information using a preset data processing mechanism, obtain valid alarm information in the first alarm information and the second alarm information, and start the fault root cause location model based on the valid alarm information.

[0047] According to the machine learning-based fault root cause location device of the embodiment of the present application, if the data monitoring dynamic threshold value obtained within the first preset time of the operation and maintenance platform and the data monitoring dynamic threshold value under different business scenarios contain the first alarm information and / or the log data contains the second alarm information, then the fault data information of the first alarm information and / or the second alarm information within the second preset time is obtained, and the fault root cause location model is started, and the correlation analysis link of the data monitoring dynamic threshold value, the data monitoring dynamic threshold value under different business scenarios and the log data is constructed, and the abnormal link dependency relationship of the operation and maintenance platform is generated in combination with the fault data information, and then the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform. In this way, the problems of false alarms, missed alarms, low efficiency, low accuracy, etc. that exist in fault root cause location due to fixed thresholds and / or manual experience are solved.

[0048] An embodiment of the third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the machine learning-based fault root cause location method as described in the above embodiment.

[0049] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the machine learning-based fault root cause location method as described in the above embodiment.

[0050] The fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the machine learning-based fault root cause location method described in the above embodiment.

[0051] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0053] Figure 1 This is a flowchart of a method for locating the root cause of a fault based on machine learning according to an embodiment of the present application;

[0054] Figure 2 This is an example diagram of a fault root cause location device based on machine learning according to an embodiment of the present application;

[0055] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0057] The following describes the method, device, equipment and medium for locating the root cause of a fault based on machine learning in an embodiment of the present application with reference to the accompanying drawings. In response to the problems of false alarms, missed alarms, low efficiency and low accuracy mentioned in the above background technology due to the fixed threshold and / or manual experience in locating the root cause of a fault, the present application provides a method for locating the root cause of a fault based on machine learning. In this method, if the data monitoring dynamic threshold of the operation and maintenance platform within a first preset time and the data monitoring dynamic threshold under different business scenarios have a first alarm information and / or the log data has a second alarm information, then the fault data information of the first alarm information and / or the second alarm information within the second preset time is obtained, and the root cause of the fault location model is started, and the correlation analysis link of the data monitoring dynamic threshold, the data monitoring dynamic threshold under different business scenarios and the log data is constructed. The abnormal link dependency relationship of the operation and maintenance platform is generated in combination with the fault data information, and then the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform. Thus, the problems of false alarms, missed alarms, low efficiency and low accuracy due to the fixed threshold and / or manual experience in locating the root cause of the fault are solved.

[0058] Specifically, Figure 1 A flowchart of a method for locating the root cause of a fault based on machine learning is provided in an embodiment of the present application.

[0059] like Figure 1 As shown, the fault root cause location method based on machine learning includes the following steps:

[0060] In step S101, the data monitoring dynamic threshold value within the first preset time, the data monitoring dynamic threshold value under different business scenarios within the first preset time, and the log data within the first preset time of the operation and maintenance platform are obtained.

[0061] According to one embodiment of the present application, before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, it also includes: obtaining the historical performance data and historical business feature data of the operation and maintenance platform; self-learning the historical performance data and historical business feature data based on a preset machine learning algorithm to obtain the historical data dynamic threshold of the operation and maintenance platform; performing cluster analysis on the historical data dynamic threshold to obtain the historical data dynamic threshold under different business scenarios.

[0062] Among them, the first preset time and the preset machine learning algorithm can be set by technical personnel in this field according to actual monitoring needs, and are not specifically limited here.

[0063] Specifically, in order to solve the problems of false alarms, missed alarms, low efficiency, low accuracy, etc. caused by the use of fixed thresholds and / or manual experience in related technologies to locate the root cause of faults, the embodiments of the present application mainly realize the automation and intelligence of fault management by building an intelligent operation and maintenance platform, and ultimately achieve the purpose of improving system stability, reducing operation and maintenance costs, and improving user experience.

[0064] Specifically, first, obtain the historical performance data and historical business characteristic data of the operation and maintenance platform. Among them, the historical performance data mainly includes the operating status and performance indicators of the operation and maintenance platform collected in the past period of time, such as the CPU usage, memory usage, disk load, network traffic and other performance indicators of the operation and maintenance platform in different periods such as yesterday, last week, last month, and last year. The historical business characteristic data mainly includes the characteristic data related to the business in the past period of time, such as the transaction data, user behavior data, order data and other business-related characteristic data of the operation and maintenance platform in different periods such as yesterday, last week, last month, and last year.

[0065] Secondly, based on the historical performance data and historical business feature data obtained above, preset machine learning algorithms are used, such as ARIMA (Auto Regressive Integrated Moving Average) and LSTM (Long Short-Term Memory), to perform self-learning based on historical performance data and historical business feature data to obtain dynamic thresholds of historical data that change over time on the operation and maintenance platform. These dynamic thresholds of historical data can better adapt to business changes and avoid false alarms or omissions caused by fixed thresholds.

[0066] Finally, by performing cluster analysis on the dynamic thresholds of historical data, such as K-means, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), etc., the dynamic thresholds of historical data under different business scenarios can be obtained. That is to say, after performing cluster analysis on the dynamic thresholds of historical data, the business characteristics of the behavior patterns under different business scenarios can be obtained, and the corresponding dynamic thresholds of historical data can be matched according to the business characteristics. For example, during business peak periods, the CPU (Central Processing Unit) utilization rate may naturally increase, and the dynamic threshold will be relaxed accordingly to avoid false alarms. During business off-peak periods, the dynamic threshold will be tightened to ensure that any abnormal fluctuations can be captured in a timely manner. Therefore, different dynamic thresholds can be applied in different time periods to ensure the accuracy of the alarm.

[0067] According to one embodiment of the present application, after performing cluster analysis on the dynamic thresholds of historical data to obtain the dynamic thresholds of historical data under different business scenarios, it also includes: based on the dynamic thresholds of historical data, using a preset machine learning algorithm to extract the time series characteristics of the dynamic thresholds of historical data, and generating the data monitoring dynamic thresholds of the operation and maintenance platform within a first preset time and the data monitoring dynamic thresholds under different business scenarios based on the time series characteristics and the dynamic thresholds of historical data; obtaining historical log data of the operation and maintenance platform, using a preset language processing technology to extract log features in the historical log data, so as to train according to the log features to obtain non-abnormal log data and abnormal log data, and obtaining the historical log database of the operation and maintenance platform based on the non-abnormal log data and abnormal log data.

[0068] Specifically, after clustering analysis is performed on the dynamic thresholds of historical data to obtain the dynamic thresholds of historical data under different business scenarios, the embodiment of the present application needs to further monitor the dynamic thresholds of historical data over a period of time in the past, so as to obtain the data monitoring dynamic thresholds within the first preset time and the data monitoring dynamic thresholds under different business scenarios based on the monitoring of the dynamic thresholds of historical data. That is to say, by monitoring the dynamic thresholds of historical data, the dynamic thresholds in different scenarios within the first preset time can be accurately predicted to provide early warning of potential risks. For example, the dynamic thresholds in the next day, week or month can be predicted to better adapt to business changes, reduce false alarms and missed reports, manage resources more intelligently, improve operation and maintenance efficiency, and ensure the stability and reliability of the system.

[0069] Specifically, a preset machine learning algorithm is used to extract the time series characteristics of the dynamic threshold of historical data. For example, based on the dynamic threshold of historical data obtained yesterday, last week, last month, and last year, that is, the historical change trend, the dynamic threshold of historical data corresponding to yesterday, last week, last month, and last year in different scenarios can be obtained based on the time series characteristics. For example, based on the network traffic usage rate, the dynamic threshold of historical data related to the network traffic usage rate of yesterday, last week, last month, and last year can be obtained, and the dynamic threshold of historical data is monitored to obtain the data monitoring dynamic threshold in the future period.

[0070] Furthermore, historical log data of the operation and maintenance platform is obtained, where the historical log data mainly includes detailed information such as the operating status, error information, user operation records, etc. of the operation and maintenance platform, which can reflect the behavioral changes within the operation and maintenance platform. Then, preset language processing technologies, such as TF-IDF (Term Frequency-Inverse Document Frequency), word vectors, etc., are used to analyze and learn the historical logs, extract common log patterns and log features, and train according to the log features to obtain non-abnormal log data and abnormal log data, and build a historical log database of the operation and maintenance platform based on the non-abnormal log data and abnormal log data.

[0071] According to one embodiment of the present application, after obtaining the historical log database of the operation and maintenance platform based on non-abnormal log data and abnormal log data, it also includes: judging whether the data monitoring dynamic threshold within the first preset time matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold in different business scenarios within the first preset time matches the historical data dynamic threshold under different business scenarios; if the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold in different business scenarios within the first preset time does not match the historical data dynamic threshold, then a first alarm message is generated; if it is determined based on the historical log database that the log data within the first preset time is abnormal log data, then a second alarm message is generated.

[0072] According to one embodiment of the present application, before determining whether the log data within the first preset time is abnormal log data based on the historical log database, it also includes: converting the log data within the first preset time into a log feature vector, and performing cluster analysis on the log feature vector, inputting the cluster analysis results into the historical log database, and determining whether the log data within the first preset time is abnormal log data based on the output results.

[0073] Specifically, after constructing the historical log database of the operation and maintenance platform, the embodiment of the present application predicts the data dynamic threshold for a future period of time based on the monitoring of the historical data dynamic threshold, that is, obtains the data monitoring dynamic threshold within the first preset time, and determines whether the data monitoring dynamic threshold within the first preset time matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold in different business scenarios within the first preset time matches the historical data dynamic threshold under different business scenarios. If the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold in different business scenarios within the first preset time does not match the historical data dynamic threshold, then it indicates that the predicted data monitoring dynamic threshold within the first preset time and / or the data monitoring dynamic threshold in different business scenarios within the first preset time are abnormal, and a first alarm information about the data monitoring dynamic threshold and / or the data monitoring dynamic threshold in different business scenarios is generated.

[0074] Furthermore, based on the historical log data, a prediction is made for a period of time in the future, that is, the log data within a first preset time, and then the log data within the first preset time is converted into a log feature vector, and the log feature vector is clustered and analyzed, such as K-means, DBSCAN, etc. The cluster analysis results are input into the historical log database and compared with the historical log data in the historical log database to determine whether the predicted log data belongs to abnormal log data. If the log data is found to be abnormal log data according to the comparative analysis, it means that there is an abnormality in the log data, and a second alarm information about the abnormal log data is generated at this time.

[0075] In step S102, if there is a first alarm message in the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios, and / or there is a second alarm message in the log data, then the fault data information of the first alarm message and / or the second alarm message within the second preset time is obtained, and the fault root cause location model is started.

[0076] According to one embodiment of the present application, before starting the fault root cause location model, it also includes: performing data preprocessing on the first alarm information and the second alarm information; using a preset data processing mechanism to perform alarm noise reduction on the preprocessed first alarm information and the second alarm information to obtain valid alarm information in the first alarm information and the second alarm information, and starting the fault root cause location model based on the valid alarm information.

[0077] The second preset time can be set by those skilled in the art according to actual monitoring requirements and is not specifically limited here.

[0078] Specifically, in an embodiment of the present application, if a first alarm message exists in the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios, and / or a second alarm message exists in the log data, then the fault data information of the first alarm message and / or the second alarm message within the second preset time is obtained, that is, the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios is in an abnormal state, and / or the log data is in an abnormal state. At this time, the fault data information of the first alarm message and / or the second alarm message within the second preset time is obtained, that is, the abnormal changes before and after the fault occurs in the historical time period are obtained. At the same time, in order to quickly identify the key alarm information from a large amount of alarm information, it is necessary to perform data preprocessing on the first alarm message and the second alarm message, such as data cleaning and priority sorting on the first alarm message and the second alarm message, and use the preset data processing mechanism to perform alarm noise reduction on the preprocessed first alarm message and the second alarm message to obtain the effective alarm information in the first alarm message and the second alarm message, that is, obtain the key alarm information and can be processed in a timely manner. At the same time, after obtaining the effective alarm information, the root cause location model of the fault is started to locate the root cause of the fault.

[0079] In step S103, data monitoring dynamic thresholds, data monitoring dynamic thresholds under different business scenarios, and correlation analysis links of log data are constructed, and based on the fault data information and the correlation analysis links, the abnormal link dependency of the operation and maintenance platform is generated. Based on the abnormal link dependency, the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform.

[0080] Specifically, by combining monitoring data such as Metrics, Logs, and Tracing, and correlating various types of alarm information, the system uses dynamic thresholds for data monitoring, dynamic thresholds for data monitoring in different business scenarios, and log data to build correlation analysis links, that is, to build a fault propagation map of the operation and maintenance platform. Then, based on the fault data information and the correlation analysis links, the abnormal link dependencies of the operation and maintenance platform are generated, and the correlation analysis links and abnormal link dependencies are visualized. At the same time, the abnormal link dependencies between various components and their state changes are recorded. Once a fault is detected, the fault root cause location model is started, and based on the abnormal link dependencies, the fault propagation path is traced back to the starting node until the original fault source is found, which can help operation and maintenance personnel quickly solve the problem.

[0081] Furthermore, after tracing the fault propagation path backward from the fault starting node to the fault source on the operation and maintenance platform based on the fault root cause location model and abnormal link dependencies, pre-customized fault recovery strategies, such as restarting services, rolling back versions, switching to backup nodes, etc., can be used. When a fault occurs, the optimal recovery plan is automatically selected based on the fault type and impact scope, and the operation and maintenance platform system automatically performs the corresponding operations, thereby achieving rapid recovery of the fault.

[0082] Therefore, based on the discussion of the above specific embodiments, the embodiments of the present application can achieve the following beneficial effects:

[0083] (1) By comparing the dynamic thresholds of historical data monitoring with the dynamic thresholds of future data monitoring, data anomalies can be quickly identified to improve the accuracy of alarms; (2) Through the generalized log mode, error log types that have not appeared before can be quickly alarmed, and problems can be discovered before the business; (3) Through intelligent noise reduction, the sending of alarm information is reduced, so that the alarm receiver receives a small amount of effective alarm information, avoiding the flood of alarm information that leads to the loss of important alarm information, and can also reduce the cost of alarm SMS; (4) By locating the root cause of the fault, the fault can be quickly located and the corresponding fault recovery strategy can be implemented to improve system stability.

[0084] According to the machine learning-based fault root cause location method of the embodiment of the present application, if the data monitoring dynamic threshold value obtained for the operation and maintenance platform within the first preset time, the data monitoring dynamic threshold value under different business scenarios has a first alarm information and / or the log data has a second alarm information, then the fault data information of the first alarm information and / or the second alarm information within the second preset time is obtained, and the fault root cause location model is started, and the correlation analysis link of the data monitoring dynamic threshold value, the data monitoring dynamic threshold value under different business scenarios and the log data is constructed, and the abnormal link dependency relationship of the operation and maintenance platform is generated in combination with the fault data information, and then the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform. In this way, the problems of false alarms, missed alarms, low efficiency, low accuracy, etc. that exist in fault root cause location due to fixed thresholds and / or manual experience are solved.

[0085] Next, a device for locating the root cause of a fault based on machine learning proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0086] Figure 2 4 is a block diagram of a device for locating the root cause of a fault based on machine learning according to an embodiment of the present application.

[0087] like Figure 2 As shown, the machine learning-based fault root cause location device 10 includes: a first acquisition module 100 , a second acquisition module 200 and a fault location module 300 .

[0088] The first acquisition module 100 is configured to acquire data monitoring dynamic thresholds of the operation and maintenance platform within a first preset time, data monitoring dynamic thresholds under different business scenarios within the first preset time, and log data within the first preset time;

[0089] The second acquisition module 200 is used to obtain fault data information of the first alarm information and / or the second alarm information within a second preset time if the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios has a first alarm information, and / or the log data has a second alarm information, and activate the fault root cause location model;

[0090] The fault location module 300 is used to construct data monitoring dynamic thresholds, data monitoring dynamic thresholds under different business scenarios and log data correlation analysis links, and generate abnormal link dependencies of the operation and maintenance platform based on the fault data information and the correlation analysis links, and based on the abnormal link dependencies, reversely trace the fault propagation path from the fault starting node to locate the fault source of the operation and maintenance platform.

[0091] According to one embodiment of the present application, before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, the first obtaining module 100 further includes:

[0092] An acquisition unit, used to obtain historical performance data and historical business feature data of the operation and maintenance platform;

[0093] A self-learning unit is used to self-learn historical performance data and historical business feature data based on a preset machine learning algorithm to obtain dynamic thresholds of historical data for the operation and maintenance platform;

[0094] The cluster analysis unit is used to perform cluster analysis on the dynamic thresholds of historical data to obtain the dynamic thresholds of historical data under different business scenarios.

[0095] According to one embodiment of the present application, after performing cluster analysis on the dynamic thresholds of historical data to obtain the dynamic thresholds of historical data under different business scenarios, the cluster analysis unit further includes:

[0096] A generating subunit is configured to extract, based on the historical data dynamic threshold, the time series characteristics of the historical data dynamic threshold using a preset machine learning algorithm, and generate, based on the time series characteristics and the historical data dynamic threshold, the data monitoring dynamic threshold of the operation and maintenance platform within a first preset time and the data monitoring dynamic threshold under different business scenarios;

[0097] The acquisition subunit is used to obtain the historical log data of the operation and maintenance platform, use the preset language processing technology to extract the log features in the historical log data, train according to the log features to obtain non-abnormal log data and abnormal log data, and obtain the historical log database of the operation and maintenance platform based on the non-abnormal log data and abnormal log data.

[0098] According to one embodiment of the present application, after obtaining the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data, the acquisition subunit further includes:

[0099] a judgment component, configured to judge whether the data monitoring dynamic threshold within the first preset time period matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold under different business scenarios within the first preset time period matches the historical data dynamic threshold under different business scenarios;

[0100] a first generating component, configured to generate a first alarm message if the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios within the first preset time does not match the historical data dynamic threshold under different business scenarios;

[0101] The second generating component is configured to generate second alarm information if it is determined based on the historical log database that the log data within the first preset time is abnormal log data.

[0102] According to one embodiment of the present application, before determining, based on the historical log database, that log data within the first preset time period is abnormal log data, the second generating component further includes:

[0103] The cluster analysis sub-component is used to convert the log data within the first preset time into log feature vectors, perform cluster analysis on the log feature vectors, input the cluster analysis results into the historical log database, and determine whether the log data within the first preset time is abnormal log data based on the output results.

[0104] According to one embodiment of the present application, before starting the fault root cause location model, the second acquisition module 200 further includes:

[0105] A data processing unit, configured to perform data preprocessing on the first alarm information and the second alarm information;

[0106] The alarm noise reduction unit is used to use a preset data processing mechanism to perform alarm noise reduction on the preprocessed first alarm information and the second alarm information, obtain valid alarm information in the first alarm information and the second alarm information, and start the fault root cause location model based on the valid alarm information.

[0107] According to the machine learning-based fault root cause location device of the embodiment of the present application, if the data monitoring dynamic threshold value obtained within the first preset time of the operation and maintenance platform and the data monitoring dynamic threshold value under different business scenarios contain the first alarm information and / or the log data contains the second alarm information, then the fault data information of the first alarm information and / or the second alarm information within the second preset time is obtained, and the fault root cause location model is started, and the correlation analysis link of the data monitoring dynamic threshold value, the data monitoring dynamic threshold value under different business scenarios and the log data is constructed, and the abnormal link dependency relationship of the operation and maintenance platform is generated in combination with the fault data information, and then the fault propagation path is reversely traced from the fault starting node to locate the fault source of the operation and maintenance platform. In this way, the problems of false alarms, missed alarms, low efficiency, low accuracy, etc. that exist in fault root cause location due to fixed thresholds and / or manual experience are solved.

[0108] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0109] Memory 301 , processor 302 , and computer programs stored in the memory 301 and executable on the processor 302 .

[0110] When the processor 302 executes the program, the fault root cause location method based on machine learning provided in the above embodiment is implemented.

[0111] Furthermore, the electronic device further includes:

[0112] The communication interface 303 is used for communication between the memory 301 and the processor 302 .

[0113] The memory 301 is used to store computer programs that can be run on the processor 302 .

[0114] The memory 301 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0115] If the memory 301, processor 302, and communication interface 303 are implemented independently, the communication interface 303, memory 301, and processor 302 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0116] Optionally, in a specific implementation, if the memory 301, the processor 302 and the communication interface 303 are integrated on a chip, the memory 301, the processor 302 and the communication interface 303 can communicate with each other through an internal interface.

[0117] The processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0118] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for locating the root cause of a fault based on machine learning.

[0119] This embodiment also provides a computer program product, including a computer program, which is executed to implement the machine learning-based fault root cause location method of the above embodiment.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0121] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0123] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0124] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0125] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0126] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0127] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for locating the root cause of a fault based on machine learning, characterized in that: The following steps are involved: Obtaining data monitoring dynamic thresholds of the operation and maintenance platform within a first preset time, data monitoring dynamic thresholds under different business scenarios within the first preset time, and log data within the first preset time; If the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios has a first alarm information, and / or the log data has a second alarm information, then obtain the fault data information of the first alarm information and / or the second alarm information within the second preset time, and start the fault root cause location model; Construct the data monitoring dynamic threshold, the data monitoring dynamic threshold under the different business scenarios and the correlation analysis link of the log data, and generate the abnormal link dependency of the operation and maintenance platform based on the fault data information and the correlation analysis link, and based on the abnormal link dependency, reversely trace the fault propagation path from the fault starting node to locate the fault source of the operation and maintenance platform.

2. The method according to claim 1, characterized in that Before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, the method further includes: Obtaining historical performance data and historical business characteristic data of the operation and maintenance platform; Performing self-learning on the historical performance data and the historical service feature data based on a preset machine learning algorithm to obtain a dynamic threshold value of historical data of the operation and maintenance platform; Cluster analysis is performed on the historical data dynamic thresholds to obtain historical data dynamic thresholds under different business scenarios.

3. The method according to claim 2, characterized in that After performing cluster analysis on the historical data dynamic thresholds to obtain the historical data dynamic thresholds under different business scenarios, the method further includes: Based on the historical data dynamic threshold, the preset machine learning algorithm is used to extract the time series characteristics of the historical data dynamic threshold, and based on the time series characteristics and the historical data dynamic threshold, the data monitoring dynamic threshold of the operation and maintenance platform within a first preset time and the data monitoring dynamic threshold under the different business scenarios are generated; Obtain historical log data of the operation and maintenance platform, use preset language processing technology to extract log features in the historical log data, train according to the log features to obtain non-abnormal log data and abnormal log data, and obtain the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data.

4. The method according to claim 3, characterized in that After obtaining the historical log database of the operation and maintenance platform based on the non-abnormal log data and the abnormal log data, the method further includes: Determining whether the data monitoring dynamic threshold within the first preset time matches the historical data dynamic threshold, and / or whether the data monitoring dynamic threshold under different business scenarios within the first preset time matches the historical data dynamic threshold under the different business scenarios; If the data monitoring dynamic threshold within the first preset time does not match the historical data dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios within the first preset time does not match the historical data dynamic threshold under the different business scenarios, generating the first alarm information; If it is determined based on the historical log database that the log data within the first preset time belongs to the abnormal log data, the second alarm information is generated.

5. The method according to claim 4, characterized in that Before determining, based on the historical log database, that the log data within the first preset time period belongs to the abnormal log data, the method further includes: The log data within the first preset time is converted into a log feature vector, and the log feature vector is clustered and analyzed. The cluster analysis result is input into the historical log database, and it is determined whether the log data within the first preset time belongs to the abnormal log data based on the output result.

6. The method according to claim 1, characterized in that Before starting the fault root cause location model, the method further includes: performing data preprocessing on the first alarm information and the second alarm information; The pre-processed first alarm information and the second alarm information are subjected to alarm noise reduction by using a preset data processing mechanism to obtain valid alarm information in the first alarm information and the second alarm information, and the fault root cause location model is started based on the valid alarm information.

7. A fault root cause location device based on machine learning, characterized in that: include: A first acquisition module is used to obtain the data monitoring dynamic threshold value of the operation and maintenance platform within a first preset time, the data monitoring dynamic threshold value under different business scenarios within the first preset time, and the log data within the first preset time; A second acquisition module is used to obtain fault data information of the first alarm information and / or the second alarm information within a second preset time if the data monitoring dynamic threshold and / or the data monitoring dynamic threshold under different business scenarios has a first alarm information, and / or the log data has a second alarm information, and start the fault root cause location model; A fault location module is used to construct the data monitoring dynamic threshold, the data monitoring dynamic threshold under the different business scenarios and the correlation analysis link of the log data, and generate the abnormal link dependency of the operation and maintenance platform based on the fault data information and the correlation analysis link, and based on the abnormal link dependency, reversely trace the fault propagation path from the fault starting node to locate the fault source of the operation and maintenance platform.

8. The device according to claim 7, characterized in that Before obtaining the data monitoring dynamic threshold of the operation and maintenance platform within the first preset time and the historical log database of the operation and maintenance platform within the first preset time, the first obtaining module further includes: An acquisition unit, configured to acquire historical performance data and historical service feature data of the operation and maintenance platform; A self-learning unit, configured to perform self-learning on the historical performance data and the historical service feature data based on a preset machine learning algorithm to obtain a dynamic threshold value of historical data of the operation and maintenance platform; The cluster analysis unit is used to perform cluster analysis on the historical data dynamic threshold value to obtain the historical data dynamic threshold value under different business scenarios.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for locating a root cause of a fault based on machine learning as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for locating the root cause of a fault based on machine learning as described in any one of claims 1 to 6.

Images