An interpretable graph-based root cause localization method and apparatus

Abstract

The application discloses an explainable graph-based root cause positioning method and device, which comprises the following steps: acquiring monitoring indexes and corresponding index dependency graphs in an online service system component; performing multi-exception mode exception detection on the monitoring indexes at the current fault time and the monitoring indexes at each historical fault time to obtain detection results of whether the monitoring indexes are abnormal; calculating index matching distances and graph matching distances of each pair of similar components for the current fault and each historical fault, and obtaining the distances of each pair of similar components; calculating optimal component allocation, calculating fault distances between the current fault and each historical fault according to the allocation results of the component distances of each pair of components; obtaining the most similar historical fault based on the fault distances of the current fault and each historical fault, and positioning the root cause of the current fault based on the most similar historical fault. The application can accurately find similar historical faults and perform root cause positioning, and can solve the challenge brought by a large number of indexes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer application, and particularly relates to an interpretable graph-based root cause positioning method and device. BACKGROUND

[0002] Large-scale online service systems (e.g., search engines and online shopping platforms) have played a vital role in our daily life. Such systems will contain hundreds of distributed components (e.g., services, containers and servers) to support a large number of concurrent users. However, due to the huge scale of online service systems and the complex dependency relationship between components, faults are difficult to avoid, causing huge economic losses. Therefore, when a fault occurs in an online service system, it is very important to quickly and accurately locate the root cause.

[0003] In order to quickly diagnose the fault, it is necessary to locate the root cause at a fine granularity, that is, the specific location and type of the root cause. For this purpose, we can use monitoring indicators as the most direct and fine-grained root cause clues. In order to closely monitor the system state, operation and maintenance personnel will often collect a variety of monitoring indicators (e.g., CPU usage, memory usage, request per second, etc.) 24*7. For example, the combination of high memory usage and high request volume occurring on a server indicates that the server has encountered a memory shortage problem caused by explosive requests.

[0004] Existing solutions can be divided into three categories: unsupervised root cause localization through heuristic rules. For example, the MonitorRank (Kim et al. 2013) algorithm considers that among the neighbors of an abnormal service, the neighbor that is most likely to cause it to be abnormal should be the one with the largest indicator similarity. The CRD (Cheng et al. 2016) algorithm assumes that a fault will cause the Granger causality to disappear, and by comparing the causality graph at normal time and the causality graph at fault time, based on the assumption that adjacent nodes have similar states (normal or abnormal), the most likely root cause node on the graph is calculated. Using supervised machine learning models to learn the relationship between fault root cause and fault features from historical fault data. For example, the DejaVu (Li et al. 2022) algorithm uses a graph neural network to input the monitoring indicators of different components in the system, and outputs the root cause score of each possible fault location and type. MEPFL (Zhou et al. 2019) inputs the features of the call trace data, and outputs the fault service and fault type. Root cause localization based on unsupervised similar historical fault search. This kind of method searches for similar faults in history, and locates the root cause of the current fault based on the root cause of the similar historical fault. All of them match similar faults by summarizing the characteristics of the alarms in the system. iSQUAD (Ma et al. 2020) and Fingerprint (Brandon et al. 2020) match the states of all indicators in the system. The above methods are not based on online service systems and cannot handle the complex dependency relationships between components in online service systems. Brandom et al. proposed representing the relationship between components in the system through a graph, and searching for similar faults by calculating the graph similarity. On the one hand, it is not based on indicator data but on specific component attributes; on the other hand, it can only be used to locate the same type of fault that has occurred at the same location in history, which has great limitations.

[0005] The disadvantages of the prior art are: the main disadvantage of the unsupervised root cause localization method through heuristic rules is that the heuristic rules they use often lack universality and cannot be universally applicable to various systems and various faults, so the root cause localization accuracy is relatively low. The disadvantage of the supervised method is that it relies on a large amount of manual root cause labeling for model training, which is difficult to apply in actual systems. It can only match the same type of historical fault that occurs at the same location. However, due to the large scale of the system, we cannot expect a type of fault to occur on every component. When matching similar faults, the interdependence of different components in the system is not considered. SUMMARY

[0006] The present application aims to at least partially solve one of the problems in the related art.

[0007] To this end, the present application aims to provide an interpretable graph-based root cause positioning method, device, equipment and storage medium, which represents the dependency relationship of components in an online service system as a graph, and matches similar historical faults and root cause positioning through the abnormal pattern of the indicators of each component on the graph, matches similar faults occurring at different locations through component allocation, and accurately finds similar historical faults and performs root cause positioning through multi-pattern anomaly detection, thereby solving the challenge brought by a large number of indicators.

[0008] To achieve the above-mentioned purpose, one aspect of the present application provides an interpretable graph-based root cause positioning method, comprising:

[0009] obtaining monitoring indicators in components of an online service system and corresponding indicator dependency graphs; wherein the monitoring indicators include monitoring indicators at a current fault time and at historical fault times;

[0010] performing multi-abnormal-pattern anomaly detection on the monitoring indicators at the current fault time and the monitoring indicators at each historical fault time to obtain detection results of whether the monitoring indicators are abnormal;

[0011] calculating indicator matching distances and graph matching distances of each pair of similar components based on the detection results for the current fault and each historical fault, and obtaining distances of each pair of similar components; calculating an optimal component allocation based on the distances of each pair of similar components, and calculating fault distances between the current fault and each historical fault according to the allocation results of the component distances of each pair of components;

[0012] obtaining the most similar historical fault based on the fault distances of the current fault and each historical fault, and positioning the root cause of the current fault based on the most similar historical fault.

[0013] The interpretable graph-based root cause positioning method according to the embodiments of the present application can further have the following additional technical features:

[0014] Further, in one embodiment of the present application, the monitoring indicators in the components include server monitoring indicators and container monitoring indicators, wherein the server monitoring indicators include at least one of the following indicators: CPU-related indicators, memory-related indicators and network-related indicators.

[0015] Further, in one embodiment of the present application, obtaining the detection results of whether the monitoring indicators are abnormal includes: determining an abnormal score of the monitoring indicators at a preset fault time according to the monitoring indicators at the preset fault time and a preset time window length parameter; determining abnormal detection results of spike and trough abnormal patterns according to the abnormal score and a preset threshold value; and calculating an indicator value corresponding to the monitoring indicators at the preset fault time, and determining abnormal detection results of sustained rise and sustained decline abnormal patterns according to the calculation results of the indicator value.

[0016] Further, in one embodiment of the present application, the obtaining of the most similar historical fault based on the fault distance between the current fault and each historical fault and the locating of the root cause of the current fault based on the most similar historical fault comprises: determining the root cause of each historical fault according to the component where the fault is located and the fault category, and obtaining an optimal mapping function according to the calculation of the fault distance between the current fault and each historical fault; obtaining the most similar historical fault based on the root cause of each historical fault and the optimal mapping function, and calculating a recommended root cause corresponding to the most similar historical fault and a corresponding recommended weight; sorting the recommended root cause according to the size of the recommended weight, and locating the root cause of the current fault according to the sorting result.

[0017] To achieve the above object, the present application provides an interpretable graph-based root cause locating device, comprising:

[0018] an index obtaining module, configured to obtain monitoring indexes in components of an online service system and a corresponding index dependency graph; wherein the monitoring indexes comprise monitoring indexes at a current fault time and at historical fault times;

[0019] an anomaly detection module, configured to perform multi-anomaly mode anomaly detection on the monitoring indexes at the current fault time and the monitoring indexes at each historical fault time, to obtain a detection result of whether the monitoring indexes are abnormal;

[0020] a distance calculation module, configured to calculate an index matching distance and a graph matching distance of each pair of same-type components based on the detection result of the current fault and each historical fault, and to obtain a distance of each pair of same-type components; and to calculate an optimal component allocation based on the distance of each pair of same-type components, and to calculate a fault distance between the current fault and each historical fault according to the allocation result of the component distance of each pair of components;

[0021] a root cause locating module, configured to obtain the most similar historical fault based on the fault distance between the current fault and each historical fault, and to locate the root cause of the current fault based on the most similar historical fault.

[0022] The third aspect of the present application provides a computer device, comprising a processor and a memory;

[0023] The processor runs a program corresponding to executable program code stored in the memory by reading the executable program code, so as to implement the interpretable graph-based root cause locating method.

[0024] The fourth aspect of the present application provides a non-transitory computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the interpretable graph-based root cause locating method.

[0025] The explainable graph-based root cause positioning method, device, equipment and storage medium according to the embodiments of the present application can utilize massive monitoring indicators and historical fault data, and accurately find similar historical faults and perform root cause positioning by graph representation of complex dependency relationships in a complex and large-scale online service system, and can match faults occurring at different positions, thereby improving the generalization of the method, and can effectively process massive monitoring indicators on each component and accurately match similar components.

[0026] Additional aspects and advantages of the present application will be described in the following description and part will become apparent from the following description or will be learned by practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of the explainable graph-based root cause positioning method according to the embodiments of the present application;

[0029] Figure 2 An architectural diagram of the explainable graph-based root cause positioning according to the embodiments of the present application;

[0030] Figure 3 A structural schematic diagram of the explainable graph-based root cause positioning device according to the embodiments of the present application;

[0031] Figure 4 A computer device according to the embodiments of the present application. DETAILED DESCRIPTION

[0032] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0033] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the protection scope of the present application.

[0034] The explainable graph-based root cause positioning method, device, equipment and storage medium according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0035] Figure 1is a flowchart of an interpretable graph-based root cause localization method of an embodiment of the present invention.

[0036] As shown in Figure 1 , the method includes but is not limited to the following steps:

[0037] S1, obtaining monitoring indicators in online service system components and corresponding indicator dependency graphs; wherein the monitoring indicators include monitoring indicators at the current fault time and historical fault times;

[0038] S2, performing multi-exception mode exception detection on the monitoring indicators at the current fault time and the monitoring indicators at each historical fault time to obtain detection results of whether the monitoring indicators are abnormal;

[0039] S3, calculating indicator matching distances and graph matching distances of each pair of similar components based on the detection results for the current fault and each historical fault, and obtaining distances of each pair of similar components; calculating an optimal component allocation based on the distances of each pair of similar components, and calculating fault distances between the current fault and each historical fault according to the allocation results of the component distances of each pair of components;

[0040] S4, obtaining the most similar historical fault based on the fault distances of the current fault and each historical fault, and positioning the root cause of the current fault based on the most similar historical fault.

[0041] Specifically, as shown in Figure 2 , when a fault occurs, the fault localization method of the present invention is triggered. The monitoring indicators on all components and the dependency relationships between components are used as inputs, and then the recommended root cause is output. The method of the present invention is divided into two parts: similar historical fault finding and root cause recommendation. In the similar fault finding part, the distance between the current fault and each historical fault is calculated. The smaller the distance, the more similar the historical fault and the current fault. In the root cause recommendation part, the corresponding recommended root cause is weighted according to the distance of all similar historical faults, and finally the ranking of all recommended root causes is obtained.

[0042] The interpretable graph-based root cause localization method of the embodiment of the present invention is described in detail below.

[0043] First, for the current fault (denoted as t1) and any historical fault (denoted as t2), the present invention needs to calculate the distance (denoted as D(t1, t2)) between them. For this purpose, the present invention will first calculate the distance between individual components and individual components, then based on the distance, similar components will be allocated as a pair, and finally the total distance of each pair of similar components will be calculated.

[0044] For t1 and t2, the present invention uses two graphs to represent the dependency relationships of the components in the system at their corresponding times, denoted as and wherein and are two sets of nodes, representing the time instants t1 and t2 respectively, of all components in the system. and are two sets of edges, representing the dependency between all components in the system at time instants t1 and t2 respectively. For example, if there is a dependency between component c1 and component c2 at time instant t1, then

[0045] All components can be classified into different component categories. For example, server A and server B belong to the same category of servers. The present invention assumes that there are the same monitoring indicators on servers in the same category. Therefore, components in the same category can be compared. Components in different categories obviously perform completely different functions, and therefore cannot be compared. Therefore, the present invention only needs to calculate the distance between a single component and a single component in the same category, and only needs to assign a component to a component in the same category.

[0046] The present invention divides the components in into different sets, each of which corresponds to a component category. For any component category, the present invention records it as C o . The present invention considers the components corresponding to it at time instants t1 and t2, respectively recorded as and Without loss of generality, the present invention can assume that |C1|≥|C2|, that is, the number of components corresponding to the component category at time instant t1 is not less than the number of components at time instant t2. For any pair of components (recorded as c1 and c2) in C1 and C2, the present invention calculates the component distance (recorded as D C (c1,c2;t1,t2) between them, which will be explained later), where the parameters t1 and t2 represent that the two components correspond to faults t1 and t2 respectively). Then, the present invention uses the Jonker-Volgement algorithm to solve the following assignment problem:

[0047]

[0048] where m is a mapping function that maps each component in C1 to each component in C2 without repetition, and then the remaining components are mapped to a virtual component null, whose indicators are all assumed to be normal. Then the present invention sums the component distances of each component in C1 (i.e. c in the formula) and the mapped component (i.e. m(c) in the formula). The present invention solves such an m that minimizes the total component distance. Its meaning is to assign as many similar components as possible to a pair without repetition, and the remaining components that cannot be assigned are considered to be normal and should be matched with a normal component.

[0049] The present application calculates the minimum sum component distance in all failure categories, and the sum is the D(t1, t2) required by the present application.

[0050] Further, the present application describes how to calculate the component distance D C (c1, c2; t1, t2), where c1 and c2 are two components of the same category at failure times t1 and t2, respectively. The present application calculates the component distance in two parts. First, the present application matches whether the corresponding monitoring indicators of components c1 and c2 at failure times t1 and t2, respectively, are similar; second, the present application matches whether components c1 and c2 have similar positions on the failure dependency graph. The sum of the distances obtained by the two parts is the component distance to be calculated by the present application:

[0051] D C (c1, c2; t1, t2) = D M (c1, c2; t1, t2) + λ · D G (c1, c2; t1, t2)

[0052] where D M (c1, c2; t1, t2) represents the indicator matching distance, D G (c1, c2; t1, t2) represents the graph matching distance, and λ is a given parameter (usually set to 0.125).

[0053] Specifically, multi-abnormal mode anomaly detection is performed on the indicators of component c1 at failure time t1 and the indicators of component c2 at failure time t2, respectively. Based on observations of actual data, the present application summarizes four abnormal modes: spike, trough, sustained rise, and sustained decline.

[0054] As an example, for spike and trough type anomalies, the present application uses the moving average method for detection. Let any one indicator be (i.e., a function mapping time to indicator value), and the failure time t to be detected and a time window length parameter W are given. The present application calculates the following anomaly score CDF M (t):

[0055]

[0056] where, and represent the average value and standard deviation of the values of the indicator at the W time points before t, respectively. If CDF M (t) is greater than a given threshold δ MA (generally set to 0.05), the present application considers that a spike type anomaly has occurred; if CDF M (t) is less than 1-δMA If so, the application considers that a trough-type anomaly has occurred.

[0057] As an example, for the anomaly of continuous rise and fall, the application uses t-test to detect. The application takes the values of W latest time points before and after the t time point of the index to be detected, and uses t-test to calculate the p-value. If the p-value is less than 0.05, then the anomaly of continuous rise or fall has occurred. Then the application calculates the mean values of the W values before and after, and if the mean value after is larger, then it is a continuous rise anomaly; otherwise, it is a continuous fall anomaly.

[0058] If none of the above four anomalies occurs, the index to be detected is normal.

[0059] Specifically, the application divides the monitoring indexes in each component category into different index categories according to the component sub-functions represented by the monitoring indexes. For example, for the monitoring indexes on the server, the application can be divided into CPU-related indexes (including CPU usage, CPU I / O wait, etc.), memory-related indexes (total memory bytes, available memory bytes, etc.), and network-related indexes (total bytes received and sent, failed bytes, etc.). The indexes of component c1 at fault t1 and the indexes of component c2 at fault t2 are matched with the proportion of anomaly patterns in each category of indexes, and then the total average matching rate is calculated between categories. In summary, the application calculates the index matching distance using the following formula:

[0060]

[0061] where Γ represents the number of total index categories, γ represents an index category, respectively represent the indexes of component c1 at fault t1 and the indexes of component c2 at fault t2, and the anomaly patterns of the part belonging to index category γ. The matching rate between the two parts of index anomaly patterns is calculated by the following formula:

[0062]

[0063] where COUNT_NONZERO represents the number of non-zero elements in a vector.

[0064] Then, the application calculates the graph matching distance by graph edit distance:

[0065] D G (c1,c2; t1,t2) = GED(AG(c1,t1;k), AG(c2,t2;k))

[0066] wherein GED denotes the graph edit distance between two graphs, AG(c1, t1; k) denotes the connected subgraph with only abnormal components within k-hop centered at c1 in the component dependency graph of the system at the failure time t1 AG(c2, t2; k) denotes the connected subgraph with only abnormal components within k-hop centered at c2 in the component dependency graph of the system at the failure time t2 AG(c2, t2; k) denotes the connected subgraph with only abnormal components within k-hop centered at c2 in the component dependency graph of the system at the failure time t2

[0067] Further, the application recommends the root cause of the current failure according to the calculated distance of the current failure and each historical failure, and the root cause corresponding to the historical failure. For each historical failure, the application records its root cause as (c, φ), wherein c represents the component where the failure position is located, and φ represents the failure category. According to the corresponding optimal mapping function m calculated when calculating the failure distance, the application maps it to (m(c), φ), which is the recommended root cause corresponding to the historical failure. The corresponding recommended weight of the application is set as wherein i represents the ranking of the historical failure in all historical failures according to the distance from small to large, and frequency(c) represents the frequency of c appearing in the root cause position of all historical failures. According to this, the application calculates all the recommended root causes and the corresponding weights, and the application sorts all the recommended root causes according to the total weight from large to small, which is the output result required by the application.

[0068] The interpretable graph-based root cause positioning method according to the embodiment of the application can utilize massive monitoring indicators and historical failure data, accurately find similar historical failures and perform root cause positioning in a complex and large-scale online service system through graph representation of complex dependency relationships in the system, can match failures occurring at different positions, thereby improving the generalization of the method, and can effectively process massive monitoring indicators on each component and accurately match similar components.

[0069] In order to realize the above-mentioned embodiments, as Figure 3 shown, the embodiment further provides an interpretable graph-based root cause positioning device 10, which comprises an indicator acquisition module 100, an anomaly detection module 200, a distance calculation module 300, and a root cause positioning module 400.

[0070] The indicator acquisition module 100 is used to acquire monitoring indicators in online service system components and corresponding indicator dependency graphs; wherein the monitoring indicators include monitoring indicators at a current failure time and historical failure times;

[0071] The anomaly detection module 200 is used to perform multi-anomaly mode anomaly detection on the monitoring indicators at the current failure time and the monitoring indicators at each historical failure time, to obtain a detection result of whether the monitoring indicators are abnormal.

[0072] The distance calculation module 300 is configured to calculate the indicator matching distance and the graph matching distance of each pair of same type components based on the detection result for the current fault and each historical fault, and obtain the distance of each pair of same type components; calculate the optimal component allocation based on the distance of each pair of same type components, and calculate the fault distance between the current fault and each historical fault according to the allocation result of the component distance of each pair of components;

[0073] The root cause positioning module 400 is configured to obtain the most similar historical fault based on the fault distance between the current fault and each historical fault, and locate the root cause of the current fault based on the most similar historical fault.

[0074] Further, the monitoring indicators in the components include server monitoring indicators and container monitoring indicators, wherein the server monitoring indicators include at least one of the following indicators: CPU related indicators, memory related indicators and network related indicators.

[0075] Further, the abnormality detection module 200 is further configured to:

[0076] determine the abnormality score of the monitoring indicators at the preset fault time according to the monitoring indicators at the preset fault time and the preset time window length parameter;

[0077] determine the abnormality detection result of the peak and valley abnormality mode according to the abnormality score and the preset threshold; and

[0078] determine the abnormality detection result of the continuous rise and continuous decline abnormality mode according to the calculation result of the indicator value corresponding to the monitoring indicators at the preset fault time.

[0079] Further, the root cause positioning module 400 is further configured to:

[0080] determine the root cause of each historical fault according to the component where the fault position is located and the fault category, and obtain the optimal mapping function according to the calculation of the fault distance between the current fault and each historical fault;

[0081] obtain the most similar historical fault based on the root cause of each historical fault and the optimal mapping function, and calculate the recommended root cause corresponding to the most similar historical fault and the corresponding recommended weight;

[0082] sort the recommended root causes according to the size of the recommended weight, and locate the root cause of the current fault according to the sorting result.

[0083] The explainable graph-based root cause positioning device according to the embodiment of the present application can utilize massive monitoring indicators and historical fault data, accurately find similar historical faults and perform root cause positioning by graph representation of complex dependency relationships in a complex and large-scale online service system, and can match faults occurring at different locations, thereby improving the generalization of the method and effectively processing massive monitoring indicators on each component and accurately matching similar components.

[0084] In order to implement the method of the above embodiment, the present application further provides a computer device. Figure 4 As shown in the figure, the computer device 600 comprises a memory 601 and a processor 602; wherein the processor 602 runs a program corresponding to an executable program code stored in the memory 601 by reading the executable program code, so as to implement each step of the above-described explainable graph-based root cause positioning method.

[0085] In order to implement the method of the above embodiment, the present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the explainable graph-based root cause positioning method.

[0086] In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second", etc. can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0087] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms is not necessarily for the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0088] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. An explainable graph-based root cause localization method, characterized in that, The method comprises the following steps: Obtaining monitoring indicators in online service system components and corresponding indicator dependency graphs; wherein the monitoring indicators include monitoring indicators at a current fault time and historical fault times; Performing multi-abnormal mode anomaly detection on the monitoring indicators at the current fault time and the monitoring indicators at each historical fault time to obtain detection results of whether the monitoring indicators are abnormal; Calculating indicator matching distances and graph matching distances for each pair of similar components based on the detection results for the current fault and each historical fault, and obtaining distances for each pair of similar components; calculating an optimal component allocation based on the distances for each pair of similar components, and calculating fault distances between the current fault and each historical fault according to the allocation results of the component distances for each pair of components; Obtaining the most similar historical fault based on the fault distances of the current fault and each historical fault, and locating the root cause of the current fault based on the most similar historical fault; The method further comprises calculating the indicator matching distance using the following formula: wherein denotes the number of total indicator categories, denotes an indicator category, denotes the components at the time of the failure the indicator at the time of the failure, and the component at the time of the failure the indicator at the time of the failure, belongs to the part of the abnormal pattern of the indicator category ; denotes the matching rate between two-part indicator abnormal patterns, calculated by the following formula: wherein represents the number of non-zero elements in a vector; Then, the graph matching distance is calculated by the graph edit distance: where GED denotes the graph edit distance between two graphs, denotes the connected subgraph of only faulty components within k-hop from , the component dependency graph of the system, , at the time of failure. denotes the connected subgraph of only faulty components within k-hop from , the component dependency graph of the system, , at the time of failure. denotes the connected subgraph of only faulty components within k-hop from , the component dependency graph of the system.

2. The method of claim 1, wherein, The monitoring indicators in the components include server monitoring indicators and container monitoring indicators, wherein the server monitoring indicators include at least one of the following indicators: CPU-related indicators, memory-related indicators, and network-related indicators.

3. The method of claim 1, wherein, The detection results of whether the monitoring indicators are abnormal include: Determining an abnormal score of the monitoring indicators at the preset fault time according to the monitoring indicators at the preset fault time and a preset time window length parameter; Determining anomaly detection results of the spike and trough abnormal modes according to the abnormal score and a preset threshold; and Calculating the indicator values corresponding to the monitoring indicators at the preset fault time, and determining anomaly detection results of the sustained rise and sustained fall abnormal modes according to the calculation results of the indicator values.

4. The method of claim 1, wherein, The method of obtaining the most similar historical fault based on the fault distances of the current fault and each historical fault, and locating the root cause of the current fault based on the most similar historical fault comprises: Determining the root cause of each historical fault according to the components and fault categories where the faults are located, and obtaining an optimal mapping function according to the calculation of the fault distances between the current fault and each historical fault; Obtaining the most similar historical fault based on the root cause of each historical fault and the optimal mapping function, and calculating a recommended root cause corresponding to the most similar historical fault and a corresponding recommended weight; Sorting the recommended root causes according to the sizes of the recommended weights, and locating the root cause of the current fault according to the sorting results.

5. An explainable graph-based root cause localization apparatus, characterized in that, The method comprises the following steps: An indicator acquisition module is configured to obtain monitoring indicators in online service system components and corresponding indicator dependency graphs; wherein the monitoring indicators include monitoring indicators at a current fault time and historical fault times; An anomaly detection module is configured to perform multi-abnormal mode anomaly detection on the monitoring indicators at the current fault time and the monitoring indicators at each historical fault time to obtain detection results of whether the monitoring indicators are abnormal; a distance calculation module, configured to calculate an indicator matching distance and a graph matching distance of each pair of same type components based on the detection result of the current fault and each historical fault, and obtain a distance of each pair of same type components; calculate an optimal component allocation based on the distance of each pair of same type components, and calculate a fault distance between the current fault and each historical fault according to the allocation result of the component distance of each pair of components; a root cause positioning module, configured to obtain a most similar historical fault based on the fault distance of the current fault and each historical fault, and position a root cause of the current fault based on the most similar historical fault; The indicator matching distance is calculated using the following formula: wherein denotes the number of total indicator categories, denotes an indicator category, denotes a component at a fault moment, and a component at a fault moment, belongs to a part of an abnormal pattern of an indicator category denotes the matching rate between two-part indicator abnormal patterns, calculated by the following formula:​ wherein represents the number of non-zero elements in a vector, Then, the graph matching distance is calculated by a graph edit distance: where GED denotes the graph edit distance between two graphs, denotes the set of faulty components at the time of failure , on the component dependency graph (CDG) of the system, the connected subgraph within k-hop centered at , consisting of only faulty components, Similarly, denotes the set of faulty components at the time of failure , on the component dependency graph (CDG) of the system, the connected subgraph within k-hop centered at , consisting of only faulty components.

6. The apparatus of claim 5, wherein, The monitoring indicators in the components include server monitoring indicators and container monitoring indicators, and the server monitoring indicators include at least one of a CPU related indicator, a memory related indicator and a network related indicator.

7. The apparatus of claim 5, wherein, The anomaly detection module is further configured to: determine an anomaly score of the monitoring indicators at the preset fault time according to the monitoring indicators at the preset fault time and a preset time window length parameter; determine an anomaly detection result of a peak and a trough anomaly mode according to the anomaly score and a preset threshold value; and, determine an anomaly detection result of a continuous rise and a continuous decline anomaly mode according to a calculation result of an indicator value corresponding to the monitoring indicators at the preset fault time.

8. The apparatus of claim 5, wherein, The root cause positioning module is further configured to: determine a root cause of each historical fault according to a component where a fault position is located and a fault category, and obtain an optimal mapping function according to the calculation of the fault distance between the current fault and each historical fault; obtain a most similar historical fault based on the root cause of each historical fault and the optimal mapping function, and calculate a recommended root cause corresponding to the most similar historical fault and a corresponding recommended weight; sort the recommended root cause according to the size of the recommended weight, and position a root cause of the current fault according to a sorting result.

9. A computer device, comprising: include a processor and a memory; The processor runs a program corresponding to executable program code stored in the memory by reading the executable program code, so as to implement the interpretable graph-based root cause positioning method in any one of claims 1-4.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the interpretable graph-based root cause positioning method in any one of claims 1-4.

Images