Micro-service fault root cause positioning method based on graph convolutional neural network

By adopting a fault location method based on graph convolutional neural network in the microservice architecture, the weighted microservice heterogeneous topology is constructed and feature weighted, the problem of difficulty in finding the root cause of the abnormality in the microservice system is solved, and high accuracy is achieved Fault location.

CN119420629APending Publication Date: 2025-02-11STATE GRID ANHUI ELECTRIC POWER CO LTD +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411450178.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-17
Publication Date
2025-02-11

AI Technical Summary

Technical Problem

In the microservice architecture, due to the complexity of microservices' interconnection, long service call links, dynamic changes in service status, and many service deployment nodes, it is difficult to find the root cause of system exceptions, which ensures the reliability of large microservice systems. Sex has become an urgent and challenging issue.

Method used

The microservice failure root cause positioning method based on graph convolution neural network is adopted. The injection failure is simulated through the monitoring system, and the historical metric data is trained using graph convolution neural network MetricSage to collect real-time fault data and service call data, and build weighted microservices. Heterogeneous topology, performing personalized random walk algorithms and feature weighting, analyzing the final root cause and corresponding failure types.

Benefits of technology

It improves the accuracy of multi-level root cause positioning, can accurately identify the root cause of failure at the instance level, and significantly improves the efficiency and accuracy of fault positioning of microservice system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119420629A_ABST
    Figure CN119420629A_ABST
Patent Text Reader

Abstract

The invention relates to a micro-service fault root cause positioning method (Micro) based on a graph convolutional neural network, and the method analyzes the complex dependency relationship between micro-services through the graph convolutional neural network, helps to judge the fault type when a fault occurs, and comprises the steps: firstly, simulating various fault types through a monitoring system, and injecting the fault types into the system; the method comprises the following steps of: classifying and marking measurement data of a simulated fault, and then training based on historical measurement data through a graph convolutional neural network MericSage so as to realize instance-level fault root cause analysis. After a fault occurs, the system collects real-time fault data and service calling data, and constructs a micro-service heterogeneous topology subset through abnormal service calling data. Weights are allocated according to the real-time indicator data, a weighted heterogeneous topology is created, and a personalized random walk algorithm is performed on the weighted heterogeneous topology to generate a root cause candidate set. And finally, inputting the candidate set and the real-time data features into Metri Sage for feature weighted analysis, and determining the final fault root cause and type.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to technical fields such as microservice architecture, operation and maintenance technology, graph neural network and anomaly detection, and specifically to a method for locating the root cause of microservice failures based on graph convolutional neural network. Background Art

[0002] Microservice architecture is becoming more and more popular due to its ability to provide high availability, scalability, and elasticity for dynamically changing workloads of software applications. As a result, more and more network-based systems are designed and built using microservice architecture. However, due to the complexity of interconnection between microservices, long service call links, dynamic changes in service status, and multiple service deployment nodes, it is difficult to find the root cause of system anomalies. Therefore, ensuring the reliability of large-scale microservice systems has become a pressing and challenging issue. To meet this challenge, various root cause analysis methods have been proposed. These methods use different types of data to achieve observability, including metrics, logs, and traces. Among them, metric data has received the most attention. Existing metric-based root cause analysis methods can be roughly classified as graph-based methods.

[0003] After containerization technology was widely adopted, microservice systems are usually operated and managed using container orchestration tools such as Kubernetes. These tools help to quickly deploy multiple instance copies for a single microservice, route requests to appropriate replicas through load balancing, and adjust the number of replicas based on request volume and available resources. Typically, different replicas are deployed on different physical machines, so it is necessary to perform comprehensive troubleshooting on all replicas of a microservice to determine the root cause of the failure. Due to the load balancing strategy, even the failure of a single instance can affect the entire microservice system. Service-level root cause location methods lack detailed indicator data for a single instance, so fault diagnosis needs to be performed instance by instance to find the root cause. Risk of new anomalies. The emergence of new anomalies in microservice systems has brought challenges to existing root cause location methods, especially those that rely solely on historical data and chaos engineering techniques for model training.

[0004] Challenges brought by dynamic creation and destruction of multiple instance replicas. In order to ensure service quality under different load pressures while minimizing resource consumption, microservice systems usually use automatic expansion to dynamically create and destroy instance replicas. In addition, in order to reduce the impact of instance failures on the system, strategies such as restart or expansion are usually implemented. However, these practices will cause the microservice topology to constantly change, making the task of tracking dynamic changes related to the root cause of the failure and accurately locating the root cause of the failure more complicated. In view of the problems existing in the above-mentioned technology, the purpose of the present invention is to provide a method for locating the root cause of microservice failures based on a graph convolutional neural network. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows: A method for locating the root cause of microservice failures based on a graph convolutional neural network, the method comprising the following steps:

[0005] Step 1: Various fault types are injected into the system through monitoring system simulation. Then, the metric data of these simulated faults are classified and labeled according to different fault types. Then, the graph convolutional neural network MetricSage is used to train the historical metric data. This training process aims to achieve instance-level root cause analysis of different fault types.

[0006] Step 2: Collect real-time fault data and service call data, build a microservice heterogeneous topology subset based on abnormal service call data, and then use real-time indicator data to assign weights to create a weighted microservice heterogeneous topology.

[0007] Step 3: Perform a personalized random walk algorithm on the obtained weighted heterogeneous topology to obtain a root cause candidate set, and then input the candidate set and real-time data features into MetricSage for feature weighting. The analysis result is to determine the final root cause and the corresponding fault type.

[0008] Preferably, in step 1:

[0009] When the microservice system is running, simulate the microservice traffic and use chaos engineering tools to inject different types of multi-dimensional faults into the service cluster. Collect multi-dimensional indicator data (including service level, instance level, and host level) of the entire cluster in different fault injection time windows. Define each fault injection time window and the time interval that captures the complete fluctuation of the indicator data as the abnormal time window. In each abnormal time window, the measurement data will be collected separately at a fixed time interval (for example, once every five seconds) as the time series measurement source data of the current fault. All subsequent method modeling and root cause location analysis will be performed within these defined abnormal time windows. For service-level indicator data, the microservices at both ends of each response message are defined as s1 and s2. Then, the response time data of a single microservice is represented as [s1, s2], and the response time data of all microservices with call relationships will form a vector x(S), as shown in the following formula.

[0010] x(S)=<[s1,s2],…,[s k ,s l ]>

[0011] The time series matrix composed of microservice response time data vector x(S) at the time interval t({t1,t2,…,t p}) sampling, denoted as S t , as follows,

[0012]

[0013] Regarding instance-level indicator data, it will belong to the microservice S m Each service instance is set to Si mn , collect the CPU, memory, network, QPS and success rate of each microservice instance, and record them as Example Si mn The metric data vector at a certain moment is expressed as

[0014]

[0015] The measurement data is measured at time intervals t({t1,t2,…,t p}) and the feature matrix SI is measured by the time series of all service instances t express,

[0016]

[0017] As for host-level metrics data, each host node is designated as n k , and collects its CPU, network and memory indicator data respectively as n kC ,n kM ,n kN At each given moment, a single time series metric data vector of a node is represented as

[0018] x(n k )= <n kC ,n kM ,n kN >

[0019] Metric feature matrix N t The measurement data of all host nodes are sampled within the time interval t, and its expression is:

[0020]

[0021] The global time series indicator within the abnormal time window is expressed as

[0022]

[0023] in represents a set of time windows with different injected faults,

[0024] Graph neural networks have the ability to model local and global patterns, explore time series indicators, propagate and aggregate information between nodes, so that global metric features of the entire microservice system can be identified, as well as local metric features between abnormal services and their multiple instances associated with specific fault types. In order to extract the features of a single node, the node at t in the abnormal time window is selected. p A single sample point at is input for normalization and constitutes a node in the graph network. Each node The characteristics include all indicators of the entire microservice system at the current moment, recorded as When the service topology is stable, the feature dimensions are consistent at all times In the graph convolution process, a multi-instance pool is introduced as an adaptive extension to serve s m Multiple instances of p The measurements of the moments are combined and expressed as

[0025]

[0026] in τ is the instance pool mask function, which is responsible for setting the instance features of the absent type to 0 to avoid the non-existent instance features affecting the adjustment of the model. Represents the most likely abnormal features among the multiple instances retained after the aggregation layer. For all services S, node The overall features after the pooling layer will be expressed as follows

[0027]

[0028] The pooling layer will allow subsequent metrics to represent the dimension of the number of services That is, the dimension is independent of the number of instances, so it is suitable for scenarios with dynamic topology changes. The feature nodes in the same abnormal time window are connected in chronological order. Create an edge E connecting the set of all nodes ({V t |t>t q ∧t∈{t1,t2,...,t p}}), generated later than the current time t in the graph network q , generate a neighbor set for each node The edges between nodes within the same anomaly time window enable the graph network structure to capture and preserve time series features. Node encoding process It can be expressed as

[0029]

[0030] Among them, normalize(·) means to normalize the measurement of each dimension using the L2 norm. Represents the metric features after the input encoding process. When all sampling points within a single abnormal time window are considered, the input data X t The MetricSage model will be used for encoding, which will generate the microservice cluster metric feature M corresponding to the abnormal time window. t , inspired by word embedding models, the encoder receives metric features,

[0031] In MetricSage, the feature vector of each node and the feature vector set is input into the aggregator for convolution aggregation. The aggregation process is repeated in sampling n (one of the graph network parameters). The k-th round aggregation of its weight matrix W can be expressed as

[0032]

[0033] Where N represents the set of adjacent nodes. n nodes are randomly selected from , σ represents the tuning linear unit activation function, and each recursive process is called a "convolutional layer". The number of convolutional layers can be selected according to the size of the microservice cluster. Due to the complexity of the measurement data, a balance must be struck between the model training time and the degree of node feature aggregation.

[0034] The node metric features are aggregated and trained to segment the root causes according to the different fault types caused by fault injection. The resulting MetricSage model is represented as Used for feature weighting in root cause segmentation.

[0035] Preferably, in step 2:

[0036] In a real-time system, the heterogeneous topology needs to be rebuilt every time according to the change in the number of instances. The first step is to collect the real-time service call data C in the system, and then use this data to build the system call topology graph. Describes the current operating status of the microservice system. When a fault is suspected, the service response time is used to detect anomalies based on the clustering algorithm. If a fault is found, the fault subgraph is extracted around the services on both sides of the abnormal call delay. Then, a heterogeneous service topology graph is constructed by integrating service-instance and instance-host connections. The generated topology map includes microservices, their instances, and host nodes that host instance containers. The focus of this step is to identify abnormal services and their instances, thereby simplifying the process of locating the root cause.

[0037] When an abnormal service call delay is detected, a fixed value Bias is specified as the maximum correlation value of the service call edge to obtain the topological node and The edge between The maximum correlation value of microservices Response time and service instances As input, when a service instance is running, its indicator data will be affected by the entire host resource data. In order to calculate the instance's metric data and its node metrics The correlation between them is calculated, the maximum value of the correlation is calculated, and the topological nodes are calculated. and Between The maximum correlation value of is then used as the edge weight of the instance and its nodes in the fault subgraph. The maximum correlation value (MC) is expressed as

[0038]

[0039] where corr(·) represents the Pearson correlation function, which calculates and selects the largest correlation among all metric dimensions as the correlation between node pairs.

[0040] Preferably, in step 3:

[0041] In the fault subgraph After weighting all the edges in , we get a weighted fault subgraph

[0042] Set the instance mn The corresponding fault services m The MC between them is assigned to the personalized array of instance nodes, for service nodes s m , and compare it to each instance {si m1 ,…,si mn} is assigned to its personalized array. Similarly, for host node n k , and compare it to each instance {si m1 ,…,si mn The average value MC between} is assigned to its personalized array to consider the implicit relationship between the instance and the host. The personalization process is expressed as

[0043]

[0044] Then, the personalized random walk algorithm is executed. The calculation formula of personalized random walk is:

[0045] v=(1-c)Pv+cu

[0046] Among them, v represents the final scoring result of the node, that is, the ranking of the root cause location results, c is the probability of the random walk continuing forward, P is a personalized array, which is used as a parameter to affect the probability of restarting the random walk from each node, and u is the scoring result of the next node. After multiple rounds of random walks, the scoring results of each node will tend to be consistent, thus generating a set of candidate root causes.

[0047] In order to enhance the accurate identification of fault types and true root causes from the indicator characteristics of the entire microservice system, the MetricSage model is used Root candidate list R C and real-time metrics data X in-time are input into the model for further feature weighting.

[0048] The time series metric features within the real-time abnormal time window are input into the graph neural network model to generate segmentation weights (SW, which are the similarity weights between historical fault classifications and real-time metric data. The final root cause weight is then calculated by multiplying the candidate set weight by (SW), as shown below.

[0049] R=R C ×SW

[0050] Among them, R C is a candidate list of fault root causes. After the feature weighting process is completed, the sorted root cause list R provides information on instance-level root causes and corresponding fault types.

[0051] In a microservice system, S represents the set of microservices, SI represents the set of all instances in these microservices, and N represents the set of all hosts. In addition, the various fault types injected into the system are classified into a set A. The root cause localization process aims to evaluate and rank each service node (s∈S) and instance node (si∈SI) in the fault subgraph according to their likelihood of being the root cause. The node with the highest score is determined to be the root cause, denoted by r, and its associated fault type is denoted by a. Therefore, the final root cause result is expressed as:

[0052]

[0053] Compared with the prior art, the advantages of the present invention are as follows:

[0054] The present invention has high accuracy in multi-level root cause location, and is compared with the most advanced methods in experiments. The fault scenarios in the dataset selected for the experiment include classic fault types observed in actual systems, covering multiple levels such as services, instances, and hosts, as shown in Table 1.

[0055]

[0056] Table 1 Dataset statistics

[0057] In order to evaluate the performance of this method in the experiment, three evaluation indicators, PR@K, AVG@N, and Acc, were used to measure the experimental results. PR@K represents the top K results in the root cause ranking list that contains the true root cause. The larger the value of PR@K result and the smaller the PR@ result, the higher the accuracy. In a given group G, the PR@ of root cause i in the root cause ranking list R can be expressed as:

[0058]

[0059] AVG@N is defined as the average metric of PR@Parliament:

[0060]

[0061] Acc indicates the position of the true root cause in the sorted list. The higher the ranking and the smaller the root cause list, the greater the Acc value. The Acc of root cause i in the sorted results of group G is calculated as follows, and the p-value (t-test) is calculated based on Acc.

[0062]

[0063] Table 2 shows the instance-level accuracy metrics for the four datasets. On average, the accuracy of the present invention in PR@1, PR@3 and PR@5 is 77.25%, 87.43% and 93.1%, respectively. The results show that MicroIRC outperforms the baseline method in instance-level root cause localization, with an average improvement of 11.5% in PR@1, 13.43% in PR@3 and 16.73% in PR@5 in the four datasets. The p-value of the t-test is used to evaluate whether the improvement of the present invention over other competing methods is due to chance. Except for the comparison with DejaVu in datasets A and B, all p-values ​​are less than 0.05, indicating that the difference between the present invention and the competing methods is statistically significant. It shows that the present invention is superior to other methods.

[0064]

[0065] Table 2 Comparison of positioning accuracy BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0067] In order to deepen the understanding of the present invention, the present embodiment is described in detail below with reference to the accompanying drawings.

[0068] Example: See Figure 1 , a method for locating the root cause of microservice failures based on a graph convolutional neural network, the method comprising the following steps:

[0069] Step 1: Various fault types are injected into the system through monitoring system simulation. Then, the metric data of these simulated faults are classified and labeled according to different fault types. Then, the graph convolutional neural network MetricSage is used to train the historical metric data. This training process aims to achieve instance-level root cause analysis of different fault types.

[0070] Step 2: Collect real-time fault data and service call data, build a microservice heterogeneous topology subset based on abnormal service call data, and then use real-time indicator data to assign weights to create a weighted microservice heterogeneous topology.

[0071] Step 3: Perform a personalized random walk algorithm on the obtained weighted heterogeneous topology to obtain a root cause candidate set, and then input the candidate set and real-time data features into MetricSage for feature weighting. The analysis result is to determine the final root cause and the corresponding fault type.

[0072] When the microservice system is running, simulate the microservice traffic and use chaos engineering tools to inject different types of multi-dimensional faults into the service cluster. Collect multi-dimensional indicator data (including service level, instance level, and host level) of the entire cluster in different fault injection time windows. Define each fault injection time window and the time interval that captures the complete fluctuation of the indicator data as the abnormal time window. In each abnormal time window, the measurement data will be collected separately at a fixed time interval (for example, once every five seconds) as the time series measurement source data of the current fault. All subsequent method modeling and root cause location analysis will be performed within these defined abnormal time windows. For service-level indicator data, the microservices at both ends of each response message are defined as s1 and s2. Then, the response time data of a single microservice is represented as [s1, s2], and the response time data of all microservices with call relationships will form a vector x(S), as shown in the following formula.

[0073] x(S)=<[s1,s2],…,[s k ,s l ]>

[0074] The time series matrix composed of microservice response time data vector x(S) at the time interval t({t1,t2,…,t p}) sampling, denoted as S t , as follows,

[0075]

[0076] Regarding instance-level indicator data, it will belong to the microservice S m Each service instance is set to Si mn , collect the CPU, memory, network, QPS and success rate of each microservice instance, and record them as Example Si mn The metric data vector at a certain moment is expressed as

[0077]

[0078] The measurement data is measured at time intervals t({t1,t2,…,t p}) and the feature matrix SI is measured by the time series of all service instances t express,

[0079]

[0080] As for host-level metrics data, each host node is designated as n k , and collects its CPU, network and memory indicator data respectively as n kC ,n kM ,n kN At each given moment, a single time series metric data vector of a node is represented as

[0081] x(n k )= <n kC ,n kM ,n kN >

[0082] Metric feature matrix N t The measurement data of all host nodes are sampled within the time interval t, and its expression is:

[0083]

[0084] The global time series indicator within the abnormal time window is expressed as

[0085]

[0086] in represents a set of time windows with different injected faults,

[0087] Graph neural networks have the ability to model local and global patterns, explore time series indicators, propagate and aggregate information between nodes, so that global metric features of the entire microservice system can be identified, as well as local metric features between abnormal services and their multiple instances associated with specific fault types. In order to extract the features of a single node, the node at t in the abnormal time window is selected. p A single sample point at is input for normalization and constitutes a node in the graph network. Each node The characteristics include all indicators of the entire microservice system at the current moment, recorded as When the service topology is stable, the feature dimensions are consistent at all times In the graph convolution process, a multi-instance pool is introduced as an adaptive extension to serve s m Multiple instances of p The measurements of the moments are combined and expressed as

[0088]

[0089] in τ is the instance pool mask function, which is responsible for setting the instance features of the absent type to 0 to avoid the non-existent instance features affecting the adjustment of the model. Represents the most likely abnormal features among the multiple instances retained after the aggregation layer. For all services S, node The overall features after the pooling layer will be expressed as follows

[0090]

[0091] The pooling layer will allow subsequent metrics to represent the dimension of the number of services That is, the dimension is independent of the number of instances, so it is suitable for scenarios with dynamic topology changes. The feature nodes in the same abnormal time window are connected in chronological order. Create an edge E connecting the set of all nodes ({V t |t>t q ∧t∈{t1,t2,...,t p}}), generated later than the current time t in the graph network q , generate a neighbor set for each node The edges between nodes within the same anomaly time window enable the graph network structure to capture and preserve time series features. Node encoding process It can be expressed as

[0092]

[0093] Among them, normalize(·) means to normalize the measurement of each dimension using the L2 norm. Represents the metric features after the input encoding process. When all sampling points within a single abnormal time window are considered, the input data X t The MetricSage model will be used for encoding, which will generate the microservice cluster metric feature M corresponding to the abnormal time window. t , inspired by word embedding models, the encoder receives metric features,

[0094] In MetricSage, the feature vector of each node and the feature vector set is input into the aggregator for convolution aggregation. The aggregation process is repeated in sampling n (one of the graph network parameters). The k-th round aggregation of its weight matrix W can be expressed as

[0095]

[0096] Where N represents the set of adjacent nodes. n nodes are randomly selected from , σ represents the tuning linear unit activation function, and each recursive process is called a "convolutional layer". The number of convolutional layers can be selected according to the size of the microservice cluster. Due to the complexity of the measurement data, a balance must be struck between the model training time and the degree of node feature aggregation.

[0097] The node metric features are aggregated and trained to segment the root causes according to the different fault types caused by fault injection. The resulting MetricSage model is represented as Used for feature weighting in root cause segmentation.

[0098] In a real-time system, the heterogeneous topology needs to be rebuilt every time according to the change in the number of instances. The first step is to collect the real-time service call data C in the system, and then use this data to build the system call topology graph. Describes the current operating status of the microservice system. When a fault is suspected, the service response time is used to detect anomalies based on the clustering algorithm. If a fault is found, the fault subgraph is extracted around the services on both sides of the abnormal call delay. Then, a heterogeneous service topology graph is constructed by integrating service-instance and instance-host connections. The generated topology map includes microservices, their instances, and host nodes that host instance containers. The focus of this step is to identify abnormal services and their instances, thereby simplifying the process of locating the root cause.

[0099] When an abnormal service call delay is detected, a fixed value Bias is specified as the maximum correlation value of the service call edge to obtain the topological node and The edge between The maximum correlation value of microservices Response time and service instances As input, when a service instance is running, its indicator data will be affected by the entire host resource data. In order to calculate the instance's metric data and its node metrics The correlation between them is calculated, the maximum value of the correlation is calculated, and the topological nodes are calculated. and Between The maximum correlation value of is then used as the edge weight of the instance and its nodes in the fault subgraph. The maximum correlation value (MC) is expressed as

[0100]

[0101] where corr(·) represents the Pearson correlation function, which calculates and selects the largest correlation among all metric dimensions as the correlation between node pairs.

[0102] In the fault subgraph After weighting all the edges in , we get a weighted fault subgraph

[0103] Set the instance mn The corresponding fault services m The MC between them is assigned to the personalized array of instance nodes, for service nodes s m , and compare it to each instance {si m1 ,…,si mn} is assigned to its personalized array. Similarly, for host node n k , and compare it to each instance {si m1 ,…,si mn The average value MC between} is assigned to its personalized array to consider the implicit relationship between the instance and the host. The personalization process is expressed as

[0104]

[0105] Then, the personalized random walk algorithm is executed. The calculation formula of personalized random walk is:

[0106] v=(1-c)Pv+cu

[0107] Among them, v represents the final scoring result of the node, that is, the ranking of the root cause location results, c is the probability of the random walk continuing forward, P is a personalized array, which is used as a parameter to affect the probability of restarting the random walk from each node, and u is the scoring result of the next node. After multiple rounds of random walks, the scoring results of each node will tend to be consistent, thus generating a set of candidate root causes.

[0108] In order to enhance the accurate identification of fault types and true root causes from the indicator characteristics of the entire microservice system, the MetricSage model is used Root candidate list R C and real-time metrics data X in-time are input into the model for further feature weighting.

[0109] The time series metric features within the real-time abnormal time window are input into the graph neural network model to generate segmentation weights (SW, which are the similarity weights between historical fault classifications and real-time metric data. The final root cause weight is then calculated by multiplying the candidate set weight by (SW), as shown below.

[0110] R=R C ×SW

[0111] Among them, R C is a candidate list of fault root causes. After the feature weighting process is completed, the sorted root cause list R provides information on instance-level root causes and corresponding fault types.

[0112] In a microservice system, S represents the set of microservices, SI represents the set of all instances in these microservices, and N represents the set of all hosts. In addition, the various fault types injected into the system are classified into a set A. The root cause localization process aims to evaluate and rank each service node (s∈S) and instance node (si∈SI) in the fault subgraph according to their likelihood of being the root cause. The node with the highest score is determined to be the root cause, denoted by r, and its associated fault type is denoted by a. Therefore, the final root cause result is expressed as:

[0113]

[0114] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A method for locating the root cause of microservice failures based on graph convolutional neural networks, characterized in that: The method comprises the following steps: Step 1: Various fault types are injected into the system through monitoring system simulation. Then, the metric data of these simulated faults are classified and labeled according to different fault types. Then, the graph convolutional neural network MetricSage is used to train the historical metric data. This training process aims to achieve instance-level root cause analysis of different fault types. Step 2: Collect real-time fault data and service call data, build a microservice heterogeneous topology subset based on abnormal service call data, and then use real-time indicator data to assign weights to create a weighted microservice heterogeneous topology. Step 3: Perform a personalized random walk algorithm on the obtained weighted heterogeneous topology to obtain a root cause candidate set, and then input the candidate set and real-time data features into MetricSage for feature weighting. The analysis result is to determine the final root cause and the corresponding fault type.

2. According to claim 1, a method for locating the root cause of microservice failure based on graph convolutional neural network is characterized in that: In step 1: When the microservice system is running, simulate the microservice traffic and use chaos engineering tools to inject different types of multidimensional faults into the service cluster. Collect the multidimensional indicator data of the entire cluster in different fault injection time windows. Define each fault injection time window and the time interval that captures the complete fluctuation of the indicator data as the abnormal time window. In each abnormal time window, the measurement data will be collected separately at a fixed time interval as the time series measurement source data of the current fault. All subsequent method modeling and root cause location analysis will be performed within these defined abnormal time windows. For service-level indicator data, define the microservices at both ends of each response message as s1 and s2. Then, the response time data of a single microservice is represented as [s1, s2], and the response time data of all microservices with call relationships will form a vector x(S), as shown in the following formula, x(S)=<[s1,s2],...,[s k ,s l ]> The time series matrix composed of the microservice response time data vector x(S) at the time interval t({t1, t2, ..., t p }) sampling, denoted as S t , as follows, Regarding instance-level indicator data, it will belong to the microservice S m Each service instance is set to Si mn , collect the CPU, memory, network, QPS and success rate of each microservice instance, and record them as Example Si mn The metric data vector at a certain moment is expressed as The measurement data is measured at time intervals t({t1, t2, ..., t p }) and the feature matrix SI is measured by the time series of all service instances t express, As for host-level metrics data, each host node is designated as n k , and collects its CPU, network and memory indicator data respectively as n kC , n kM , n kN , at each given moment, a single time series metric data vector of a node is represented as x(n k )=<n kC ,n kM ,n kN > Metric feature matrix N t The measurement data of all host nodes are sampled within the time interval t, and its expression is: The global time series indicator within the abnormal time window is expressed as in represents a set of time windows with different injected faults, Graph neural networks have the ability to model local and global patterns, explore time series indicators, propagate and aggregate information between nodes, so that global metric features of the entire microservice system can be identified, as well as local metric features between abnormal services and their multiple instances associated with specific fault types. In order to extract the features of a single node, the node at t in the abnormal time window is selected. p A single sample point at is input for normalization and constitutes a node in the graph network. Each node The characteristics include all indicators of the entire microservice system at the current moment, recorded as When the service topology is stable, the feature dimensions are consistent at all times In the graph convolution process, a multi-instance pool is introduced as an adaptive extension to serve s m Multiple instances of p The measurements of the moments are combined and expressed as in τ is the instance pool mask function, which is responsible for setting the instance features of the absent type to 0 to avoid the non-existent instance features affecting the adjustment of the model. Represents the most likely abnormal features among the multiple instances retained after the aggregation layer. For all services S, node The overall features after the pooling layer will be expressed as follows The pooling layer will allow subsequent measurements to represent the dimension of the number of services That is, the dimension is independent of the number of instances, so it is suitable for scenarios with dynamic topology changes. The feature nodes in the same abnormal time window are connected in chronological order. For a feature node Create an edge E connecting the set of all nodes The generation time is later than the current time t in the graph network q , generate a neighbor set for each node The edges between nodes within the same anomaly time window enable the graph network structure to capture and preserve time series characteristics. Node encoding process It can be expressed as Among them, normalize(·) means to normalize the measurement of each dimension using the L2 norm. Represents the metric features after the input encoding process. When all sampling points within a single abnormal time window are considered, the input data X t The MetricSage model will be used for encoding, which will generate the microservice cluster metric feature M corresponding to the abnormal time window. t , inspired by word embedding models, the encoder receives metric features, In MetricSage, the feature vector of each node and the feature vector set is input into the aggregator for convolution aggregation. The aggregation process is repeated in the sampling n. The k-th round aggregation of its weight matrix W can be expressed as Where N represents the set of adjacent nodes. n nodes are randomly selected from , σ represents the tuning linear unit activation function, and each recursive process is called a "convolutional layer". The number of convolutional layers can be selected according to the size of the microservice cluster. Due to the complexity of the measurement data, a balance must be struck between the model training time and the degree of node feature aggregation. The node metric features are aggregated and trained to segment the root causes of different fault types caused by fault injection. The resulting MetricSage model is represented as Used for feature weighting in root cause segmentation.

3. According to claim 1, a method for locating the root cause of microservice failure based on graph convolutional neural network is characterized in that: In step 2: In a real-time system, the heterogeneous topology needs to be rebuilt every time according to the change in the number of instances. The first step is to collect the real-time service call data C in the system, and then use this data to build the system call topology graph. Describes the current operating status of the microservice system. When a fault is suspected, the service response time is used to detect anomalies based on the clustering algorithm. If a fault is found, the fault subgraph is extracted around the services on both sides of the abnormal call delay. Then, a heterogeneous service topology graph is constructed by integrating service-instance and instance-host connections. The generated topology map includes microservices, their instances, and host nodes that host instance containers. The focus of this step is to identify abnormal services and their instances, thereby simplifying the process of locating the root cause. When an abnormal service call delay is detected, a fixed value Bias is specified as the maximum correlation value of the service call edge to obtain the topological node and The edge between The maximum correlation value of microservices Response time and service instances As input, when a service instance is running, its indicator data will be affected by the entire host resource data. In order to calculate the instance's metric data and its node metrics The correlation between them is calculated, the maximum value of the correlation is calculated, and the topological nodes are calculated. and Between The maximum correlation value of is then used as the edge weight of the instance and its nodes in the fault subgraph. The maximum correlation value (MC) is expressed as where corr(·) represents the Pearson correlation function, which calculates and selects the largest correlation among all metric dimensions as the correlation between node pairs.

4. According to claim 1, a method for locating the root cause of microservice failure based on graph convolutional neural network is characterized in that: In step 3: In the fault subgraph After weighting all the edges in , we get a weighted fault subgraph Set the instance mn The corresponding fault services m The MC between them is assigned to the personalized array of instance nodes, for service nodes s m , and compare it to each instance {si m1 ,...,si mn } is assigned to its personalized array. Similarly, for host node n k , and compare it to each instance {si m1 ,...,si mn The average value MC between} is assigned to its personalized array to consider the implicit relationship between the instance and the host. The personalization process is expressed as Then, the personalized random walk algorithm is executed. The calculation formula of personalized random walk is v = (1-c)Pv+cu Among them, v represents the final scoring result of the node, that is, the ranking of the root cause location results, c is the probability of the random walk continuing forward, P is a personalized array, which is used as a parameter to affect the probability of restarting the random walk from each node, and u is the scoring result of the next node. After multiple rounds of random walks, the scoring results of each node will tend to be consistent, thus generating a set of candidate root causes. In order to enhance the accurate identification of fault types and true root causes from the indicator characteristics of the entire microservice system, the MetricSage model is used Root candidate list R C and real-time metrics data X in-time are input into the model for further feature weighting. The time series metric features within the real-time abnormal time window are input into the graph neural network model to generate segmentation weights (SW, which is the similarity weight between historical fault classification and real-time metric data. Then, the final root cause weight is calculated by multiplying the candidate set weight by (SW), as shown below. R=R C ×SW Among them, R C is a candidate list of fault root causes. After the feature weighting process is completed, the sorted root cause list R provides information about instance-level root causes and corresponding fault types. In the microservice system, S represents the set of microservices, SI represents the set of all instances in these microservices, and N represents the set of all hosts. In addition, the various fault types injected into the system are classified into set A. The root cause location process aims to evaluate and rank each service node (s∈S) and instance node (si∈SI) in the fault subgraph according to the possibility of becoming the root cause. The node with the highest score is determined as the root cause, denoted by r, and its related fault type is denoted by a. Therefore, the final root cause result is expressed as:

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for locating the root cause of a microservice fault based on a graph convolutional neural network as described in any one of claims 1 to 4 above is implemented.

6. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, the method for locating the root cause of a microservice fault based on a graph convolutional neural network as described in any one of claims 1 to 4 is implemented.

Citation Information

Cited By

  • Multi-dimensional root cause positioning method and device based on multi-modal learning

    CN120151186A

  • Chaotic engineering system risk prediction method based on graph attention network

    CN120197959A

  • Control method and system for remote monitoring of Internet of Things

    CN120512353A